C.A. Voss. London Business School, UK. V. Chiesa. CNR-ITIA, Milano, Italy, and ..... contributor to the performance of the Japanese automotive industry. Keeping.
Developing and Testing Benchmarking and Selfassessment Frameworks in Manufacturing
Self-assessment Frameworks in Manufacturing 83
C.A. Voss London Business School, UK
V. Chiesa CNR-ITIA, Milano, Italy, and
P. Coughlan Trinity College, Dublin, Eire Benchmarking and self-assessment is being used increasingly by industry as a tool to help identify “best practice”, and to identify areas for improvement. In the quality management area, the impact has been particularly striking. In the US and Europe the Malcolm Baldrige National Quality Award has been particularly effective. Many companies have used it both for benchmarking and self-assessment leading to great improvements in quality practices and performance[1]. To date these approaches have been practice led, with limited academic rigour of the kind that is found in other areas and tools for improvement in manufacturing. The research in this article set out to examine two questions, first, can an academic based approach contribute to more rigorous benchmarking and self-assessment in manufacturing and, second, can tools be developed for use in areas in manufacturing, in particular those to do with technology management. Benchmarking Benchmarking has been described as “a continuous, systematic process for evaluating the products, services and work processes for organizations that are recognized as representing best practices for the purpose of organizational improvement”[2]. It has evolved from an approach that focused mainly on measures of performance to that which focuses on the management activities and practices that lead to superior performance. The concept of benchmarking has been used by companies for many years. Until recently, it was used to compare measures of business performance and to compare product The authors would like to acknowledge the support of the UK Department of Trade and Industry who sponsored this work, the ACME directorate of the Science and Engineering Research Council who funded much of the research, and the Confederation of British Industry and its members who contributed to the testing.
International Journal of Operations & Production Management, Vol. 14 No. 3, 1994, pp. 83-100. © MCB University Press, 0144-3577
IJOPM 14,3
84
performance. Following pioneer work at the Xerox corporation[3], it has been extended to business processes. Typically companies start by identifying a business process that they want to improve, or is seen as having high leverage on their competitive capability. For example the first process benchmarked by Xerox was warehousing. The next step is internal data collection, studying and understanding their own process in this area, often with the use of tools. The next step is external data collection; this can be through use of databases, visits to a number of other companies which are seen as leaders in the area, to study their processes as well as performance, and through this to identify industry best practice. The final steps are to compare internal with external data, to identify gaps between current and best practice and to develop plans to improve the business process. A recent study[4], found that self-assessment and benchmarking in quality management had begun to be adopted very effectively in Europe. European companies had found the American Baldrige award framework a good starting point, but had moved to using the framework developed by the European Foundation for Quality Management. In addition, the study found that a number of tools had been developed to support benchmarking of total quality management. The companies studied had used this very effectively, leading to real quality improvement. Manufacturing is an area which holds great promise for the application of benchmarking. Benchmarking requires companies to compare themselves with best in class practice, not just in their own countries, or even in Europe, but best world practice. The problems faced by manufacturing companies in Europe wishing to benchmark are, first, that there is no generally accepted instrument for benchmarking of manufacturing practices. Second, there are no tools available to support benchmarking; third, there are no adequate databases of manufacturing practice for companies to benchmark themselves against. There is a clear need to develop instruments and supporting tools for companies to support benchmarking of manufacturing to enable them to: ● assess their policies and management practices in manufacturing; ● help them to identify the gap between their current practices and the best in class and world-class practices; ● develop plans of action to eliminate these gaps. Process-based Approaches Benchmarking and self-assessment differs from previous approaches to performance evaluation in that the main approach is to examine the business or managerial processes. It is therefore instructive to examine different perspectives on this area. There is a growing school of research and application that views the business or administrative process as an alternative to a functional focus for management. In this sense an organization can be viewed as a set of physical, administrative, managerial and investment processes[5], the
design and the management of which need to be aligned with a company’s strategy and have an explicit role in its competitiveness. In the strategy area there is increasing interest in process based research (in contrast to content-based research). Chakravarthy and Doz[6] state that fundamental questions for (strategy) process research are: ● What are the relationships between a firm’s administrative systems and decision processes and its competitive and/or resource positions? ● How does a firm achieve and maintain effectiveness in the above relationships? ● How does it modify its administrative systems and decision processes both in response to environmental changes and through its own proactive actions. In the technology management area, Pavitt[7] has stated that “a major criticism of the content view of technological strategy is that it neglects the context within which – and the process whereby – technological strategies are generated, chosen and implemented. These processes are bound to involve more than the technical function”. This theme is repeated in many reviews of process based approaches. Processes cross both functional and disciplinary boundaries. Clark and Fujimoto[8] provide an excellent example of a process based approach in technology management. They conducted an in depth study of the processes associated with the development of new cars in a wide range of companies and countries. They identified a large number of elements of the process that were associated with successful development. It has led to normative recommendations related to the business objectives of shortening lead times, improving product quality and reducing development costs. Operations management academics have always paid considerable attention to processes. Thus, there has been much published research into operational processes such as materials resource planning, quality systems, just-in-time processes, lean production and project management. The traditional focus of operations management academics has been on the design of processes and their implementation rather than their relation to corporate performance. However, increasingly writers in the area are concerned with the relationship between operational processes and performance. Process based approaches can play a considerable role in developing and testing theory. Chakravarthy and Doz[6] argue that strategy research has to become more normative if it is to be of relevance to the general managers. “The challenge we see for strategy process research is to become more relevant to practice.” Research into the processes of firms has the potential to develop knowledge of value to practitioners. The Research The objectives of the research on which this article is based were twofold: first, to develop a methodology for generating benchmarking frameworks and tools.
Self-assessment Frameworks in Manufacturing 85
IJOPM 14,3
86
Second, to develop and test this framework in the area of innovation or technology management. Within the area of technology management is included the management of process technology in manufacturing[9]. To develop a process-based framework, the first step was to develop an overall model of the processes involved. This could be described as a “top down” approach. The second step was to focus on each of the major or core processes. This required a “bottom up” approach-based on the literature and knowledge concerning each process. From this could be developed definitions of sub-processes and tools for self-assessment and benchmarking each process and sub-process. This is equivalent to content validation used in the development of research scales. Content validity refers to whether the scale has appropriate meaning in relation to the concept being measured. It is determined by reference to the literature and to experts. In this article we will focus on one major area, process technology innovation and development. Tools have to be effective in use and a final key stage of the research was testing the definitions and tools for usability and usefulness. This was done with a sample of six UK companies drawn from a range of industries. The following sections describe the process used for developing and testing a benchmarking framework. A Business Process-based Model The starting point is to identify the core business processes to be benchmarked. This leads to the development of a process-based model, outlining the key processes, the enabling factors and the outcome of the business processes. To do this it was found that a top-down approach was needed. Taking a top down approach, technology management in manufacturing can be viewed as a set of four main business processes: (1) product innovation: the process of bringing together technology and market needs to develop new product concepts; (2) product development: the process of bringing a new product concept through development and manufacturing to the market; (3) production process innovation: the process of innovating and developing new production processes; (4) technology acquisition: the process of acquiring the technology necessary for product and process innovation through internal R&D and/or other means. These core business processes address both the formulation of technology strategies and long-term plans (product innovation and technology acquisition processes); and the implementation of technology strategies and of product and process innovations (product development and process innovation processes). To be effective, these core processes have to be supported in a number of areas (enablers): resource management process, leadership process and the availability and use of methodologies, systems and tools.
A process will have objectives and will lead to an outcome in operational or business terms. Much research has focused on the relationship between single aspects of innovation and performance of the organization. We argue that the process of technology management as a whole can lead to increased business performance and ultimately to increased competitiveness. Therefore, to complete our process based model of technology management we relate the process to an organization’s competitiveness and performance in the marketplace. The resulting process based model has been discussed in more detail by Chiesa et al.[10]. Figure 1 provides an overview of the process-based model of technology management developed in this research. Having built an overall model using a top-down approach, the next step was to develop a detailed view of each particular process and its sub-processes using a bottom up approach. This involved examination of the literature on each business process, to identify research or assertions concerned with good practice and/or success and failure in order to develop a model of the subprocesses and the areas to examine when benchmarking these sub-processes; and to develop performance measurement and self-assessment tools. In the research this was carried out for each of the business processes identified in Figure 1, but in this article we focus on the process most related to production, process innovation. There is a need for both development and innovation of processes as a source of competitive strength in their own right[11] and to support product innovation. Production processes have been seen as the central focus of manufacturing strategy formulation[12]. A particular issue in production process innovation is implementation[13], and even after implementation incremental innovation is seen as necessary to ensure continuous improvement.
Self-assessment Frameworks in Manufacturing 87
Process of innovation Resources Product innovation L E A D E R S H I P
Product development
Innovation performance Business performance Systems and tools
Enablers
Process innovation
Technology acquisition
Process
Outcome
Figure 1. A Process-based Model of Technology Management
IJOPM 14,3
88
Process innovation includes key areas of sub-process and good practice, which can be used as a basis for developing tools for companies to use in benchmarking and self-assessment. These are summarized below: (1) Generating process innovations: ● evaluating the capabilities of existing production processes; ● matching process capabilities to the requirements of the marketplace; ● linking process innovation to product innovation; ● developing new process technologies. (2) Implementation of new processes: ● matching technology complexity to the capability to adopt; ● managing the links with suppliers in the development and implementation; ● making the appropriate changes to the organization. (3) Continuous improvement: ● identifying opportunities for improvement in processes; ● integrating process improvement with quality control; ● benchmarking production process performance; ● involving manufacturing process developers in improvement after installation. We will discuss each sub-process in turn. Generating Process Innovations Process innovations represent a major contributory factor to the competitiveness of firms and, thus, nations[14]. In their book The Machine that Changed the World, Womack et al.[15] argued that the aggregation of a series of process innovations that they entitled “lean production” was a major contributor to the performance of the Japanese automotive industry. Keeping informed of new process technologies and testing them with a view to subsequent adoption is a key managerial task. A number of authors such as Hill[12] have argued that process choice is central to a firm’s manufacturing strategy, and that this choice should be the result or outcome of a process of manufacturing strategy development. A manufacturing strategy should help define the required manufacturing capabilities and new processes. Hayes et al.[11] state that “one can build a competitive advantage through superior manufacturing, but sustaining it over time requires comparable skills in creating a continual stream of new products and processes. The effective combination of product and process development increases the returns from each.” They argue that product and process innovation should interact because of the competitive power that results from combining proprietary processes
with proprietary products. This in turn leads to the need for two-way interaction between product innovation and process innovation. These links do not just require links at the innovation stage, but also the development of appropriate skills, flexibility and resources to support transfer of new products into production and their ramp-up[16].
Self-assessment Frameworks in Manufacturing
Implementation of Process Innovations Leonard-Barton[17] showed that process innovations often fail to meet their initial targets even when, subsequently, they become successful. A familiar example is a software program crash on initial use. The process of getting from installation, through any initial failure to success is implementation. Voss[13] has argued that implementation of process innovation is a field of study in its own right and set out a framework to describe the process of implementation. There have been a number of studies of factors impacting on the success and failure of implementation[14]. Tidd[18] and Jaikumar[19] have all found that a mismatch between process state of knowledge and capability to adopt can lead to failure in implementation. Tidd, for example, found that flexible assembly robot installations in the UK were more sophisticated technologically than the equivalent Japanese installations and were less flexible in use. A similar pattern was found by Jaikumar[19] when studying FMS in Japan and the US . Winch et al.[20] have identified a number of other factors concerned with the organization of the implementation, the involvement of relevant functions and suppliers. It is widely recognized that implementing new process innovations requires appropriate skills and training. Tidd[18] found that poor implementation in the UK was associated with poorly trained, low skilled operators, as well as little communication between manufacturing and sales functions. In contrast the Japanese firms in his study were characterized by highly skilled operators, good communications between design and manufacturing and sales and close relationships between suppliers and customers. Chiesa and Jovane[21] have found many examples of poorly managed relationships between end-users and technology suppliers in production process innovations.
89
Continuous Improvement The literature of product innovation refers to incremental innovation as an important factor in successful innovation. In process innovation this is equally true, though it is often referred to as continuous improvement. Quality management has always paid particular attention to processes and, in recent years, has seen this as central to its application. Continuous improvement of processes is now seen as a central part of any quality process (see for example the Baldrige Quality Award structure). The Japanese have also seen this as central to management of manufacturing, and the term kaizen is commonly used. Continuous improvement is consistent with the need for continuing adaptation in the implementation of process innovations. Many problems that occur in implementing process innovations arise from a poor match between
IJOPM 14,3
90
the innovation, the organization, and the strategy and objectives. LeonardBarton[17] has developed a model that brings these together. She views implementation as a process of mutual adaptation of technology and organization. When an innovation is first used, there is likely to be a number of misalignments. These misalignments can be between the innovations and the organization, the innovation and its objectives etc. To overcome these, a series of cycles of adaptation of both the technology and the user environment is required. Implementation of process innovations thus becomes a continuous process of mutual adaptation. Metrics When assessing oneself, or benchmarking a process, it is necessary to be able to develop metrics or measures of performance. The process based model developed above is a powerful way of structuring the development of metrics. Having identified each core process and their intended outcomes, we can postulate potential measures of performance for each of the core processes and enabling processes. For process innovation, performance can be measured in terms of the effectiveness, development speed and cost. Process innovation effectiveness can be measured by the number of new processes and significant enhancements per year. Moreover, innovations in production are often a matter of improving continuously; thus, performance measures of the process of continuous improvement are: the number of suggestions per employee, the percentage of them that have been implemented, the average annual improvement in the process parameters (quality, cost, lead time, WIP, reliability, downtime, capability). Use of metrics can identify processes where performance is poor and, thus, provide focus for benchmarking. The metrics developed are shown in Table I. Developing Benchmarking and Self-assessment Tools Both self-assessment and benchmarking focus on questions such as whether the individual processes necessary for technology management are in place and the degree to which best practice is used and implemented effectively. We have proposed a process-based model for process technology and innovation. Such a model can be developed into two sorts of tools for companies, a self-assessment tool and a process framework. Each essentially provide qualitative analysis, though standard scores can be developed. They provide information on strengths and deficiencies in current processes, indicate where good processes might be found and provide a rich set of data for learning and managerial action for improvement. Self-assessment Tool The development of such a tool requires identification of good and best practice in each area. An extensive literature review (see for example Chiesa et al.[10]) provided a basis for identifying the characteristics of each process associated with success and failure in technology management and, thus, the practices
Measures (versus plan, absolute, trend, versus competitors): (1)
Process parameters, cost, quality, WIP levels, lead time etc. ● performance versus competitors ● percentage improvement over 12 months, three years
(2)
Installation lead times ● start to trouble-free working ● percentage of new processes/process innovations considered successful
(3)
Number of new production processes, significant process enhancements in year
(4)
Number of business processes redesigned/re-engineered in last year
(5)
Continuous improvement number of improvement suggestions per employee ● percentage implemented ● average annual improvement in process parameters (quality, cost, lead time, WIP, reliability, downtime, capability)
Self-assessment Frameworks in Manufacturing 91
●
(6)
Progress to lean production, WIP, lead times, quality
leading to successful innovations. We took the above research-based statements as indicators of the characteristics of good practice in processes. From this, we constructed a hierarchy of characteristics of a good process. To these we added indicators of lack of good management processes in each area. From this, we constructed a simple score-card to be used by companies to assess their own processes. The score-card used a scale of 1-4, 1 being unsatisfactory, 4 being good. A score-sheet was developed for each of the processes and sub-processes of technology management. As the score-cards were designed to be used on a stand-alone basis without expert support it was found that a balance needed to be struck between detail and simplicity. Table II shows the score-card for the process innovation sub-processes described above. This was used by companies as a tool for self-assessment of process innovation. It will produce a raw score for each sub-process. In the terms commonly used by companies in benchmarking, the difference between the actual and ideal score was the gap between current and good processes. The assessor having identified a gap, then used this as a framework for examining in more detail the reasons for the gap, and developed plans of action. In line with the approach used by the Baldrige award, assessment can be done both in terms of the depth of any application and in terms of its breadth of use. This first tool was designed to be a simple assessment process that can be done easily and quickly. As such it can provide an entry into the self-assessment
Table I. Possible Measures of Performance for Process Innovation and Technology Management
No attention to implementation
If it isn’t broken leave it alone
2. Implementation of new processes
3. Continuous improvement
Table II. Score-sheet for Self-assessment
Serious lagging in process technology and process capability
Focus on maintenance of processes, not improvement
Implementation seen as installation
Process technology bought off the shelf
No manufacturing strategy
2
Use of a wide range of data both internal and external to support process improvement
Improvement primarily the responsibility of process engineering function
Involvement of equipment suppliers in continuous improvement
Work teams are encouraged to identify opportunities for improvement
Active involvement by suppliers in implementation
Implementation teams stay together into full production to ensure learning and improvement
Information on new process technology actively sought and new processes tested to gain experience
Manufacturing processes seen as a source of competitive advantage
Capabilities of current processes fully understood
Strong links between product development and process development
4
Need for continuous improvement of processes recognized
Identification of new skills required, development of training programmes
Cross-functional implementation teams
Prepared to invest in improving existing and developing new process technologies
A manufacturing strategy ensures that process capabilities support market needs
3
92
1. Generating process innovations
1
IJOPM 14,3
and benchmarking process, and an introduction to the processes to be assessed and benchmarked. It can indicate, at an aggregate level, where a company is doing well and where there are gaps, and can help set up an agenda for more indepth investigation.
Self-assessment Frameworks in Manufacturing
A Benchmarking Framework A second way in which processes can be reviewed is through benchmarking, that is, comparison of a company’s process with that of others, preferably best practice companies. Benchmarking, as described earlier requires understanding of the processes to be benchmarked. One of the most successful uses of benchmarking has been through the use of the Malcolm Baldrige Quality Award framework[1]. The reasons for its success include:
93
●
It provides a uniform framework that enables companies to benchmark the key processes.
●
It requires companies to understand themselves before benchmarking against others.
●
It is a good balance between depth, requiring real questions to be asked, and simplicity, not being too complex to be used.
In this research it was decided to recognize these characteristics and develop a framework using Baldrige as a model. A similar framework was developed for each core and enabling process. The framework was developed on the following principles: ●
Statement and definition of the main business process to be benchmarked. This has already been discussed in the earlier sections of this article.
●
Statement and definition of each sub-process to be benchmarked. The description is based on a request to describe the particular process.
●
A set of possible questions to consider. These were the equivalent of the Baldrige areas to consider. They were based on the literature review described above on best practice and determinants of success and failure. These questions were designed to guide companies to ask appropriate questions when benchmarking other organizations or assessing themselves.
The framework for process innovation and technology management is shown in Table III. The framework was designed to support the team assessing and benchmarking. It was to provide a common focus and direction, and was designed to force companies to ask relevant questions during the benchmarking and selfassessment process.
Describe how production processes are continually improved
Continuous improvement
Describe the process for assuring effective implementation of new process technology
Implementation of new processes
Describe the process for developing new manufacturing and business processes
How is process improvement integrated with day to day quality control? Are work teams encouraged to identify opportunities for improvement in processes? Is there use of internal data such as process control and external data such as customer feedback, competitive benchmarks and product performance to improve process performance?
What is the organization for implementation? Are there: (1) teams involving all relevant functions and suppliers; (2) appropriate skills and training available for new technologies; (3) links with equipment suppliers in the development of new processes?
Is choice of complexity of process technology matched with the capability to adopt?
What is the organization for developing new process technologies? Does it include two way linking of product innovation to process innovation? Are the resources, skills and flexibility available to support new product testing and ramp-up?
Is there a written, market-led, manufacturing strategy? Does this define the need for new manufacturing capabilities and processes?
How does the company keep informed of sources of new process technology and test new processes? How are the capabilities of existing processes determined?
Generating process innovations
Table III. Process Innovation and Technology Management Areas to address
94
Process
IJOPM 14,3
Testing the Tools In development of research-based tools, the literature emphasizes the importance of testing the content validity, that is, whether the scales in the tools have appropriate meaning in relation to the concept being measured. To assure content validity of the tools described here we based them on extensive literature research as described above. However if the tools are to be of use by companies they need further testing. The final stage of development of research-based benchmarking tools was to test them for both usability (can it be used in practice) and for usefulness (did practitioners see it as being of real benefit to themselves). In order to test the framework, six firms, all active members of the Confederation of British Industry, volunteered to try using both the benchmarking framework and the self-assessment tool. They were drawn from a range of industries and sizes, see Table IV.
Self-assessment Frameworks in Manufacturing 95
Testing Usability Testing usability took place in three phases, the first two of which involved individual meetings with the companies and analysis by the companies, the final phase a benchmarking meeting of all six participating companies. All the processes of technology indicated in Figure 1 were tested. Phase 1 was designed to test the understanding of the framework and its terminology. As might be expected from a framework developed by academics, there were a considerable number of areas where the companies stated that they did not understand the language or the concepts. All of these comments were taken on board and a considerably revised framework was produced. In addition, the companies commented on areas that they thought were missing. This highlighted the area of metrics, and additional attention was paid to debating with them what metrics might be usable. In phase 2, the companies were asked to use the framework unaided by the development team. Two experiments were carried out. Two versions of the framework, a simple and a more thorough version were tested on different
Firm
Industry
Processes
Size
A
Petrochemicals
High-volume process
Very large
B
Electrical fittings
High-volume line
Medium
C
Systems integration
Low-volume batch
Medium
D
Fibres and chemicals
High- and medium-volume process
Large
E
Adhesives
Medium-volume batch
Medium
F
Speciality paper
Medium-volume process
Small
Table IV. Firms Testing the Frameworks
IJOPM 14,3
96
companies. In addition, some companies used the score-cards first, others used the benchmarking framework first. The outcome of this test was to indicate that (as might have been expected) the simpler version was found much easier to use. It was also found that the self-assessment score-card was seen to be a tool to use before full benchmarking, as it was very easy to use, and allowed companies to focus in on priority areas. The final stage of testing was to dry run a full benchmarking process with all six companies. This was done in the format of a benchmarking club, the group of companies got together to share knowledge of their own performance and processes in series of meetings. This proved successful, with each company having used the benchmarking framework to analyse their own processes, and being able to contribute to the exchange and learn from the others. Of particular interest was the degree to which a single set of tools and frameworks could be used by a wide range of companies. It was found that all could use the tools equally well. Differences seemed to be in the focus of their concerns. Process industries tended to have a greater concern for process technology development than others, engineering and software-based companies were more concerned with products and transfer of products into manufacture. Tools varied with context as well. There was surprisingly little difference based on company size, small companies were as concerned with technology management and innovation and were as capable in using benchmarking as large companies. Overall the tools and frameworks were found to be remarkably robust and generic. Testing Usefulness Testing usefulness is, by its nature, difficult and qualitative in the short-term. Explicit questions were asked of each of the participating organizations. The responses were very positive. All companies stated in various ways that they found the tools and frameworks challenging and useful. The chief executive of one made a statement to this effect in public to a meeting of 500 companies at the Confederation of British Industry. Another company stated that he had had many tools and questionnaires, but that this one had really made him think. Yet another manager was using it to start a debate at the top level in the company. Some of the statements solicited by a third party from the companies are listed below: Use the…self-assessment guide in the boardroom, and in business team meetings, it is a powerful tool for understanding and correcting problems in the innovation process. We all owe it to our shareholders to get this right. BP Chemicals In the growing company, the maintenance of competitive edge through innovation is of paramount importance. The self-assessment guide used honestly forces you to confront your real, as opposed to perceived, performance and forms a good guard against complacency. Whatman Paper Innovation in products and processes is a key strategy that both management and employees must take on board. The guide provides a valuable tool for internal assessment and comparisons with competitors. Hughes Rediffusion Simulation
Transfer to Industry Further evidence of usefulness was that when the testing was over, all companies wanted the benchmarking club to continue. A final indicator of usefulness was that the sponsor of the research, the UK Department of Trade and Industry (DTI), and the collaborating organization, were very keen to take up the work and distribute the tools nationally. During 1993, a full package for self-assessment and benchmarking was developed for the DTI. This included a workbook comprising both the framework described above and a full set of training materials in its use. The workbook contained the tools as a score-card within the workbook[22]. To operationalize the use of the framework a six-part process was developed; this is outlined in the Appendix[22]. Managerial Implications Benchmarking, while becoming increasingly used by companies, has benefited little from academic contribution or comment. We contend that to be successful, benchmarking participants must have a common model of the processes to be benchmarked and the metrics to be used. Too often benchmarking is no more than a fishing expedition, looking to catch something interesting, but often only landing a minnow. Increasingly, organizations are finding that benchmarking can be done most effectively through benchmarking clubs or clearing houses. These mechanisms place a particular demand on the participants to work from a rigorous base. Frameworks and tools based on good reference to literature and research, with proper attention to content validity, are more likely to be successful. A second contention, based in particular on experience of the use of the Baldrige award, and supported through our field research is that selfassessment should precede benchmarking. Indeed in many cases good selfassessment has a major value in its own right. As a result, any process, model or tool for benchmarking should pay particular attention to its use for selfassessment. This article has demonstrated a structured approach to benchmarking and self-assessment based on a business process approach. A particular issue that many companies face is that of measurement of performance of processes in innovation and technology management. We have developed a set of metrics that can be used in this area. As discussed earlier, they can provide a common base for comparison across companies and to provide focus for subsequent benchmarking activity. From this experience a generic six step set of procedures is proposed. These steps are summarized below: (1) Identify business processes to be benchmarked. (2) Use a “top-down” approach to develop an overall framework of the processes to be benchmarked. (3) Use a “bottom-up” approach, based on literature and knowledge of best practice to identify sub-processes and characteristics of best practice.
Self-assessment Frameworks in Manufacturing 97
IJOPM 14,3
98
(4) Develop metrics for each process. (5) Develop tools, self-assessment score-cards and benchmarking frameworks. (6) Test the frameworks and tools for usability and usefulness. This six-step process has been illustrated and tested in the area of technology management, with detailed illustration through a process innovation framework and a detailed framework and tools for process technology and innovation management. The evidence from this research is that this has proved to be an effective methodology. Basing benchmarking on more rigorous academic foundation leads to frameworks and tools that are not just useable by companies, but go beyond the simple to something that can lead to real improvement. There is no reason why such a procedure cannot be used for developing benchmarking and selfassessment tools for any business process for which there is an established set of knowledge of good practice and success factors. Finally, it can provide a valuable tool linking academic research to implementation and improvement. References 1. Garvin, D.A., “How the Baldrige Award Really Works”, Harvard Business Review, November-December, 1991, pp. 80-93. 2. Spendolini, M.J., The Benchmarking Book, AMACOM, New York, NY, 1992. 3. Camp, R.C., Benchmarking, Quality Press, Milwaukee, WI, 1989. 4. Carter, G. and Voss, C.A., “Self Assessment Using Quality Awards”, paper presented at European Foundation for Quality Management conference, Amsterdam, April 1993. 5. Nicholson, A. and Coughlan, P., “Representing Business Operations Management Responsibilities without Getting Lost in All the Detail”, in Proceedings of the EIASM Production Conference, London, April 1993. 6. Chakravarthy, B.S. and Doz, Y., “Strategy Process Research: Focusing on Corporate Selfrenewal”, Strategic Management Journal, Vol. 13, 1992, pp. 5-24. 7. Pavitt, K., “What We Know About the Strategic Management of Technology”, California Management Review, Vol. 32, 1990, p. 3. 8. Clark, K. and Fujimoto, T., Product Development Performance, HBS Press, Boston, MA, 1991. 9. Chiesa, V., Coughlan, P. and Voss, C.A., “Auditing the Firm’s Innovation Capability”, Operations Management, Working Paper, 1993-02, London Business School, 1993. 10. Chiesa, V., Coughlan, P. and Voss, C.A., “Innovation as a Business Process”, paper presented at the British Academy of Management conference, Milton Keynes, September 1993. 11. Hayes, Wheelwright and Clark, K., Dynamic Manufacturing, Free Press, New York, NY, 1988. 12. Hill, T., Manufacturing Strategy, Macmillan, London, 1985. 13. Voss, C.A., “Implementation, a Key Issue in Manufacturing Technology, the Need for a Field of Study”, Research Policy, Vol. 17, 1988, pp. 53-63. 14. Voss, C.A., “Successful innovation and implementation of New Processes”, Business Strategy Review, Spring 1992, pp. 29-43.
15. Womack, J.P., Jones, D.T. and Roos, D., The Machine that Changed the World, Macmillan, London, 1990. 16. Wheelwright and Clark, K., Revolutionising Product Development, Free Press, New York, NY, 1992. 17. Leonard-Barton, D., “Implementation as Mutual Adaptation of Technology and Organisation”, Research Policy, Vol. 17, 1988, pp . 251-67. 18. Tidd, J., Flexible Manufacturing Technologies and International Competitiveness, Pinter, London, 1991. 19. Jaikumar, R., “Postindustrial Manufacturing”, Harvard Business Review, NovemberDecember 1986, pp. 69-76. 20. Winch, G. and Voss, C.A., Process of Implementation, Achieving Business Success, London Business School operations management working paper OM11, 1991. 21. Chiesa, V. and Jovane, F., “The European Supranational Programmes: A Key Mechanism for Developing Innovative CIM Systems”, Proceedings Prolamat 1992, Tokyo, June 1992. 22. Voss, C.A., Coughlan, P. and Chiesa, V., Innovation Your Move – Self Assessment Guide and Workbook, Department of Trade and Industry, 1993.
Self-assessment Frameworks in Manufacturing 99
Appendix: Six-part Process for Using the Innovation Framework Step 1: Form an Assessment Team A team including people from all the functions or departments involved in innovation is more likely to be able to identify current practice accurately, and to take and be responsible for subsequent action. The team needs to understand the objectives of self-assessment in your organization, and will require appropriate briefing and training. The team should be charged with: ●
assessing your current innovation practice and performance;
●
identifying the gaps between current performance and best practice;
●
developing and implementing action plans and setting performance targets.
Step 2: Initial Self-assessment The assessment team should conduct an initial assessment of all areas of the innovation process in the organization. The first step should be for all members of the team to complete the scorecards and plot the scores on the accompanying profile. This provides an initial indication of the firm’s innovation performance as viewed by each individual team member. ●
The difference between your performance and the maximum possible score can be considered a gap to be closed. The scores allotted to each section by each team member may differ in some cases. Discussion of these differences may provide a more accurate insight into the firm’s performance.
Step 1 Team formation
Step 2 Initial self-assessment
Step 4 In-depth self-assessment
Step 3 Choice of focus
Step 6 Action: closing the gaps
Step 5 Benchmarking
Figure A1. The Six Steps of Innovation Selfassessment
IJOPM 14,3
Step 3: Choosing What to Focus On Following the initial assessment you may choose to go into greater detail on the whole process of innovation or to start with those aspects which are considered most critical. You may wish to focus on where your problems or needs are. If the choice is made to focus initially on particular areas, it is strongly recommended that all other areas are also examined in depth at a later stage.
100
Step 4: In-depth Review of Innovation Practice and Performance All innovation areas, or each area identified as important by your team in step 3, should now be reviewed in depth by the team. Current practice and performance should be identified and documented. This review should use the in-depth self-assessment questions. To ensure maximum benefit, the review should be grounded in reality and honestly reflect current practice rather than be just statements of policy. Much can be based on real case histories. For example, analysing a development project will create a stimulating discussion and will give a much more realistic picture of your performance than a theoretical analysis. The result of this should be a thorough understanding of the firm’s performance and accurate identification of gaps to be closed: ●
Are we doing what is required and do we have the right processes?
●
Are we doing it in all areas?
●
Are we doing it well?
After completing the in-depth self-assessment you should revisit Step 2 – Initial Self-assessment – and “re-score” your innovation performance. You should now be able to identify the main reasons for a gap between your score and a score of 4. Step 5: Benchmarking This optional step provides a powerful way of gaining a fresh perspective on the gaps and identifying action areas through comparison with best practice. Benchmarking is carried out in two ways: (1)
collaborative benchmarking and benchmarking clubs;
(2)
benchmarking through visiting leading companies.
Step 6: Action: Closing the Gaps Having identified gaps and the causes of those gaps through self-assessment and/or benchmarking, the prime task is identifying and implementing action to close them. This can be done through an internal workshop where the findings of the self-assessment and benchmarking process are presented and reviewed by the assessment team and other interested parties. At such a workshop: ●
The gaps should be agreed.
●
Reasons for the gaps should be identified.
●
How best practice can be achieved should be discussed.
●
Actions for closing gaps should be identified.
●
New goals for performance and practice should be set.
Action plans for improvement should be identified and implemented. To support action plans, quantitative measures of performance can be implemented – possibly drawn from the lists provided –- so that improvements can be measured.
Self-assessment Frameworks in Manufacturing 101
