Evaluating Intellectual Capital and Measuring ...

Paper presented at “The Third International Conference on Performance Measurement and Management 2002”

Evaluating Intellectual Capital and Measuring Knowledge Management Effectiveness Oliver Gupta, Stephen Pike and Göran Roos Intellectual Capital Services Ltd., England Abstract Building on the complexities of practical measurement of knowledge management (KM) in companies, we describe a practical approach to evaluating knowledge flows and managing knowledge more effectively. A pan-European survey revealed what people believe about truth, knowledge and the optimum approach to KM from their perspective. Practical results highlight the differences in KM requirements in epistemological terms and show typical differences between companies units and between knowledge workers and their perceptions of what companies are attempting to achieve. We claim these differences lie at the heart of problems companies experience with extracting the value they could from knowledge management investments. Introduction Two common themes of current performance management thinking are one, that a traditional view of measurement (i.e. as a means of control) as being naïve and two, that there should be greater emphasis on measuring the financial and non-financial value contributions of intangibles. Despite the introduction of numerous frameworks such as the Balanced Scorecard (Kaplan and Norton, 1996) and IC-Index (Roos et al. 1997) there still exists a large discrepancy between theory and practice as concerns the measurement of intellectual capital in action, in particular concerning knowledge management (KM) initiatives. A pan-European study of epistemological approaches revealed overarching problems are the inability of companies to measure the benefits of their investments in knowledge management or even to find the best way of managing the knowledge in their companies. One of the most common inhibitors revealed in the study was a lack of understanding related to the value of KM programs. KM was described as being “difficult to sell as it is a fuzzy concept.” There is often confusion about the nature of KM efforts, specific program elements and anticipated outcomes. Getting people “on the same page” is difficult and can inhibit not only KM success but also cause the complete demise of KM programs. Building on the complexities of practical measurement of knowledge management in companies, this paper describes a practical approach to evaluating knowledge flows and from this, shows how to manage knowledge more effectively. Drawing heavily upon a practice-oriented methodology, the proposed approach and the supporting theory has been developed in an iterative process between academia and consulting experience. The underlying theory is a synthesis of current thinking in the resource-based view of the firm, the intellectual capital perspective, axiology, epistemology and performance measurement. In the survey companies were investigated to determine what people believe about truth, knowledge and the optimum approach to knowledge management from their perspective. Practical results highlight the differences in knowledge management requirements in epistemological terms, using three categories to group individuals and further show typical differences between various parts of companies. The epistemological approach is also used to show the differences between what people want (as reflecting the sophisticated reality of their world) and what management provides in terms of KM. We claim that these differences lie at the heart of problems companies experience with knowledge management investments and the difficulties they face in extracting the value they could. The practical evidence, underpinned by the theoretical approach, is then extended using an intellectual capital navigator to further highlight the diversity of knowledge management activities and place them in the context of value creation across all the key activities of the company. Even with the diversity of knowledge use in single companies, the navigator is able to link knowledge intensive activities within a company to the value creating steps effectively. This enables managers to target and enhance value creation more effectively while meeting the knowledge management requirements of the employees. With the insights gained from the practical evidence and the management insights from the navigator, it then becomes a relatively simple task to make a meaningful assessment of the contribution of knowledge management to value creation in the company. Modern intellectual capital methodologies can provide the means to visualise the contribution meta-activities like knowledge management make to company value. Careful selection of critical indicators can then be attempted with the aim of addressing the knowledge intensive steps as shown on the navigator. The indicators can then be combined using the navigator pathways and algorithms appropriate to the structure of that pathway to produce an index or indices relevant to the company and its use of knowledge. While simple, the development of indices can give managers valuable insight into the effectiveness of their knowledge management and the optimum approaches to improve company performance. Basic Knowledge Management Under KM we understand the collective name for a group of processes and practices used by companies to increase their value by improving the effectiveness of the generation and application of their intellectual capital. KM processes are meta-processes which cannot be uniformly observed like physical processes and differ according to their means of creation, nature, recording, transmission and mode of use (Boisot, 1995). According to our definition, no two KM implementations will be the same since the infinitely variable human being is central to it. The reasons why companies invest in KM are that it gives them either a temporal, effectiveness or an efficiency advantage over their competitors, or they do it to try to negate the advantages of others. Many dress up the decision to invest in KM in softer terms but the underlying motives remain simply advantage or survival (Lyles et al., 1992; Huber, 1990). Unfortunately, the majority of KM implementations are unsatisfactory as standardised and inappropriate approaches are forced upon the varying predicament of the uninformed by the uninformed. It is generally accepted from reviews such as that by KPMG (2000) that KM is not a fad and is therefore taken seriously by companies and

Page 1 of 8


governments. Given this attitude, it is disappointing to find that according to surveys such as that by CTP (1999), few companies think that their investment in KM was a success while between 16% and 36% felt it was a failure. To succeed with KM in a company, an understanding of exactly what it is that is being managed is required and, in addition, an understanding of who is using it to do what. It is usual to find that different parts of companies use KM systems in quite different ways to fulfil their allotted function. Should these different parts have significantly different requirements of the KM systems and should the KM system be targeted at one or neither specifically then it follows that the KM system is not adequately serving at least one of the groups. Thus in an analysis of the value a KM system provides a company, three basic steps emerge. These are: 1. 2. 3.

For the key functional groups in the company, determine their belief system with regard to knowledge and how it compares with the company KM system. Determine the roles that knowledge plays in creating value for the company and how it changes the company through organisational learning. Develop a mechanism to track or measure the contribution of knowledge-intensive activities.

There is considerable latitude amongst the human race as the nature of knowledge, what it means and how it should be managed, if at all. If managers try to impose a model of knowledge and then attempt management on a basis foreign to the workforce they will fail. Typical modes of failure are that the KM system is ignored, new information is not entered into the system and consequently the system is ultimately ignored. To understand people’s beliefs and use of knowledge, a grounding in epistemological issues, the study of knowledge, is required. This is not an esoteric excursion, it is fundamental. Epistemological classes are described by Venzin et al. (1998) and in more detail by Habermas (1984) as they relate to KM. Figure 1 describes an interpretation of these epistemological classes in a KM setting. Cognitivist Cognitivists consider the identification, collection and central dissemination of information as the main knowledge development activity. Organisations are considered as open organisations that develop increasingly accurate pictures of their pre-defined worlds through the assimilation of new information. Knowledge is developed according to universal rule, hence the context of the incoming information is important.

Connectivist There are many similarities here to the cognitivist viewpoint but a difference being that there are no universal rules. As rules are teambased and vary locally, organisations are seen as groups of self-organised networks dependent on communication. The connectionists believe that knowledge resides in the connections and hence focus on the self-organised dispersed information flow.

Autopoietic Here the context of information inputs is unimportant as it is seen as data only. The organisation is a system that is simultaneously open (to data) and closed (to information and knowledge). Information and knowledge cannot be transmitted easily since they require internal interpretation within the system according to the individual’s rules. Thus autopoietics develop individual knowledge, and respect that process in others.

Figure 1 Three Cognitive Distinctions between Knowledge Viewpoints About the Study To determine the value beliefs of the various functional groups in companies and their attitudes to knowledge, a questionnaire comprising 16 questions devoted to epistemological issues has been developed. The questions are repeated in that respondents are first asked to describe themselves and secondly, how they see the capabilities of their company’s KM system. This methodology has been applied to a number of European companies from various industries. Proceeding in accordance with an agreement for non-specific attribution, the questionnaire results were coded and analysed using algorithmic content review to place the respondents’ answers on the surface as shown in Figure 2.

Page 2 of 8


Autopoetics Individual interpretation of knowledge

Cognitivists

Connectivists

Transferability of knowledge

Team ownership of knowledge

Figure 2 Classes of Epistemology The survey process was exploratory and thus not designed to enable statistical treatment of results, nor to test or confirm any specific hypothesis. The epistemological results are shown in the diagrams below. In these diagrams, the horizontal-lined areas depicts where the responses fell when respondents were considering themselves. The cross-hatched areas depict where the responses fell when respondents were considering their company’s KM system.

UK branch of a global consulting company

A Northern European company concerned with the exploitation of primary resources and first stage processing

Page 3 of 8


A European high-tech company seeking to concerned with reaching new customers and internal economies

A group of people from several UK companies charged with designing the KM systems and processes for their companies

Figure 3 Results for Selected European Companies Discussion of the Results The results from the UK consulting company show that there is a significant gap between the beliefs of the respondents with regard to truth and knowledge and the characteristics of the system they use in the company. Clearly the people tend to an autopoietic epistemology but are also likely to function well as a team. This is a common representation of a consultant who is usually part of a team on an engagement but who has to be mentally adaptable to the situation of the engagement. The view of their company KM system showed considerable agreement by virtue of the small size of the bubble and was placed mid-way between the cognitivist and connectivist approaches. In the Northern European primary industry company a considerable alignment between the people and their KM system can be found. Again, the KM system of the company straddles the cognitivist/connectivist interface but this time it closely reflects the people. The reason for this is that the work of people in the company is dominated by processes and safety critical systems and hence the implementation of a KM system that gives this is good. In the European high-tech company, the KM system is in its usual place but this time the responses of the people covered a very broad area. For some of the people the KM system was well suited to them but the sample group contained a number of highly individualistic autopoietic people, often though of as valuable problem solvers. Clearly the KM system was unsuitable for them. The fourth example does not depict a single company but the KM representatives of a number of UK companies that were surveyed in a networking forum. Each person was responsible for the KM activities in their parent company and this plot shows the diversity of people in this type of post. The people with the strong cognitive/autopoietic focus came from law firms. This is not unexpected given the highly codified nature of case law and the dependence on precedent combined with an ability to creatively extend precedent into new case examples. One feature that has dominated all our survey findings has been the placement of the company KM systems in the minds of people from widely different companies. This result is not surprising in practice since investigation revealed that in all companies surveyed, the KM system was dominated by a document management / information retrieval system. It should be noted, however, that the questionnaire is not aimed at determining the software characteristic but rather the KM environment of the company so that KM practices and management attitude should also be reflected. The implication of this is that the Northern European primary industry company has the most appropriate management attitude. While the results and the analysis show varying degrees of fit, the unanswered question is: are there other parts of the organisation’s value creating pathways for which the company KM solution is better or worse fitted? The Use of Knowledge in Companies and its Measurement With an understanding of what the workforce believes about knowledge, it is possible to look at how organisations deal with the practicalities of creating, managing and using knowledge. The problem faced by all people when designing knowledge management systems is that inside any organisation, knowledge is used in different ways in different places. Furthermore, the nature of the knowledge is also different. For example, the

Page 4 of 8


use of knowledge in parts of companies responsible for the generation or delivery of the product is likely to be dominated by explicit knowledge embodied in processes and procedures. Self-generated modifications to that knowledge based are likely to be controlled by processes to ensure that there are no unexpected or unwanted consequences of the changes. The dominant epistemology is likely to be a cognitive one. On the other hand, in the same company, the research group is likely to be very different in nature. The people are much more likely to operate under an autopoietic epistemology and tacit knowledge and exchanges of it will dominate knowledge exchange. Development and management could easily follow another model. The conclusion to be drawn is that the use of knowledge in a company will be complex and so its management and measurement will not be amenable to simple solutions. The next steps are then to find out where the critical knowledge using steps in a company are and then find a way to measure them. It is convenient to look at the mechanistics of knowledge management using the intellectual capital approach developed by Roos (Roos et al., 1997; Gupta and Roos, 2001). Early work (Sveiby, 1997; Edvinsson and Malone, 1997) referred to as first generation intellectual capital, showed differences in language and measurement approaches but most importantly was static in that transformations between categories of intellectual capital were not addressed. Second generation intellectual capital methodologies, devised by Roos et al. (1997), have served to harmonise language and fully account for transformations and hence can be related to real business processes. To further understand KM processes and to see where knowledge-intensive activities exist in the company a model is required to determine the value creating transformations. Second generation intellectual capital methodologies offer the means of determining this. In this approach, intellectual capital resources are described as belonging to one of three categories: human, organisational and relational. Each of these can be broken down into finer distinctions. In practical terms, the tangible and intangible resources in the company are identified and weighted in terms of importance in creating value according to the company’s strategic intent and then the importance of transformations between them are discovered and weighted accordingly. The results of this are expressed mathematically in a matrix and visually in a conceptual map known as a ‘navigator’ (see Figure 4). The size of the ovals represents the importance of the resources while the thickness of the arrows represents the importance of the transformations between the resources. As the navigator is essentially conceptual in nature, it can easily be formatted to highlight meta-processes such as those which characterise knowledge creation and use. This approach has its inherent advantages. Early discussions of KM were difficult and confusing because each person was envisioning a different concept. Consequently, we have found that developing an IC navigator as a visual model creates a basis for dialogue around knowledge flow. Although we recognize the model as a simplification, this simplicity has proved to be very valuable, for it stimulates almost everyone to work it in some way to improve it. The ensuing discussions invariably help make differences in perspective apparent or coalesce. For example, even at the coarse level of granularity such as in the example depicted it can be seen that the key influencing pathways aside from the end generation of cash are firstly the transformations from “tacit knowledge” to “explicit knowledge” and “image” and from “explicit knowledge” to “image” and secondly the “cognitive competence” to “cash” transformation. This could be interpreted as a company which depends on knowing how to deliver a service through excellent standard processes but is also innovative in its product. These two pathways have no direct link and are to a significant degree, independent. With such a clear difference, it is unlikely that a single KM solution or simple KM behavioural environment will function effectively. There also arises a question of measurement.

Cognitive competence Financial metric Creative problem solving metric

Premium rate index

Codification metric

Market research metric

Mental agility

Tacit knowledge

Publication rate & quality metric

Cash & equivalents

Customers

Company image Explicit knowledge

Figure 4 Example of an IC Navigator and Knowledge Flow Metrics As shown, KM is dominated by the dynamic use of knowledge. Recent attempts at measurement methodologies include those of Preiss (1999), but no direct measure is possible and the advantages of the holistic approach of intellectual capital are absent. Glazer (1998) attempts

Page 5 of 8


measurement from a different perspective and measures “the knowers” and the contribution of their activities. As with Preiss’ approach, this is limited and non-holistic and so cannot be used to solve the value equation for the company. At this stage, there are two related pathways that can be followed to measure the benefits of KM in companies. The first is to obtain insights into the role of knowledge and its management with a strategic perspective and consequently develop indices to measure critical features. This approach is developed in this paper but before describing the methodology, it is worth considering the aims of the second approach. The second is to attempt a wholly rigorous approach, attempting to solve the value equation for the company: Value = f (influence, flow). In this equation, influence is provided by the conceptual map of the navigator in that it shows the influence of intellectual capital resource transformations in the creation of value. Flow can be provided by an activity-based technique such as systems dynamics. The two are combined, and the equation solved, by a conjoint measurement system (Pike and Roos, 2000) in what is seen as the only extant third generation intellectual capital approach. This paper follows the first approach to the measurement of KM, that is, the construction of a set of indices which may be used to assess the changing contribution to the company’s strategic aims played by KM. Although quantitative metrics for KM remain under development in the literature there are many possible sources of metrics. The most readily used are metrics arising from an analysis of critical paths, critical cost drivers, critical customer value drivers and industry standards. For example, process metrics include the usual tracking of costs versus budget and frequency of hits on KM websites. Output metrics include a year-to-year analysis of the value of the company’s intellectual resources, including an estimate of the worth of its tacit knowledge base. A more specific technique is to assess the quality of the knowledge base, that is, how current, accessible and easily updated it is. In our study, one respondent engages an external consultant to periodically assess the readiness and the progress of the knowledge base within the company. Another company measures the reduction of manufacturing cost as a result of sharing technical advanced among its plants. All of these remain valid but essentially the navigator, with its focus on the influence resource transformations have on meeting strategic intent, must be seen as the principle guide of metrics for KM or for the whole company. An inspection of the navigator will reveal the most influential transformations, these have been described above and shown in Figure 4. A minimum set of metrics derived from metric sources is then developed. From a practical management perspective, however, multiple attribute metrics are of little use. Much more useful is the combination of these attributes into a set of indices that can be tracked over time—this is however always limited by the period of validity of the strategy. To effect the combination of metrics into a small set of indices, the combination logics also have to be determined by the connections in the navigator. Traditional combinatorial approaches (such as simple additions or multiplications) are of no use here. It is also wholly wrong to combine all measures of say human resources alone. Instead, combinations have to follow the navigator pathways since these reflect how value is created in strategic terms–this will undoubtedly include transformations between resources in different intellectual resource categories. Crucial for the measurement of knowledge management is that one or more of the combined metrics reflects the performances of transformation pathways that are knowledge intensive. By tracking the changes in the output values for these metrics, insight into the ability of the KM systems to deliver a fitting contribution to strategic value is obtained. Metrics combined using combinatorial mathematics Other metrics in the minimum set 1 0.8 Flow indicator 1 Flow indicator 2 Flow indicator 3 Flow indicator 4 Stock indicator 1 Stock indicator 2

0.6 0.4

Financial metric

0.2

Creative problem solving metric

Premium rate index

Codification metric

Market research metric

0 1999

Publication rate & quality metric

2000

2001

2002

The result is a set of metrics which address the critical pathways of the company including those which are knowledge intensive and make a real contribution to company value creation Figure 5 Making an Index

Conclusions This paper shows that utilisation knowledge for companies is important but far from simple due to the complex nature of knowledge in humans, the means to store and transfer it, ad the learning effect it has on those who use it. Managers must recognise the different ways in which knowledge is created and used in their companies if they are to manage it successfully and create value for stakeholders. Measurement and

Page 6 of 8


management of such intangible resources and meta-processes can only really be undertaken successfully by using modern intellectual capital processes coupled with an appropriate set of combined indices derived from a navigator which address knowledge intensive processes. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

Boisot, M. (1995), “Information Space”, International Thompson Business. CTP (1999), “The knowledge Paradox: How to Manage your Most Strategic Asset”. Edvinsson, L. and Malone, M.S., (1997), “Intellectual capital: The proven way to establish your company’s real value by measuring its hidden brainpower”, Piatkus. Glazer R. (1998), “Measuring the Knower: Towards a Theory of Knowledge Equity”. California Management Review Vol. 40. No. 3 pp175-194 Gupta, O. and G. Roos (2001), “Mergers and Acquisitions Through an Intellectual Capital Perspective” Journal of Intellectual Capital, Vol.2, Nr.3. Habermas, J. (1984), “Translator’s introduction”, in Habermas, J., The Theory of Communicative Action, Vol. 1., Polity Press Huber, George P. (1990), “A Theory Of The Effects Of Advanced Information Technology” The Academy Of Management Review, Briarcliff Manor; Jan 1990; Vol. 15, Iss. 1; pg. 47, 25 pgs Kaplan, R. and D. Norton (1996), “The Balanced Scorecard: Translating Strategy into Action”, Harvard Business School Press KPMG (2000), “Knowledge Management Research Report 2000” KPMG Consulting, London. Lyles, M. R. and C.R. Schwenk (1992), “Top Management, Strategy and Organisational Knowledge Structures”, Journal of Management Studies, No 29. Pike, S. and G. Roos (2000), “An introduction to intellectual capital”, Works Institute Journal, Vol. 42, Oct, Pages 21-27, Tokyo. Preiss, K. (1999), “Modelling Knowledge Flows and their Impact”, Journal of Knowledge Management, Vol. 3, No. 1, pp36-46 Roos, J., Roos, G. Dragonetti, N.C. and Edvinsson, L. (1997), “Intellectual Capital: Navigating the New Business Landscape”, Macmillan Press, London. Venzin, M., Krogh, G. and Roos, J. (1998), “Future research into knowledge management” in Krogh, G., Roos, J. and Kleine, D. “Knowing in Firms”, Sage Publications, London. Sveiby, K.E. (1997), “The new organisational wealth: Managing and measuring knowledge based assets”, Berrett-Koehler. Von Krogh, G., Roos, J., Kleine, D. (1998), “Knowing in Firms”, Sage Publications, London.

Page 7 of 8


Evaluating Intellectual Capital and Measuring Knowledge Management Effectiveness Oliver Gupta Dr. Stephen Pike Ind. Prof. Göran Roos Intellectual Capital Services Ltd., England This is a practitioner submission intended for the action stream. Keywords: Intellectual Capital, Knowledge Management, Measurement Address: 46 Grays Inn Road London, WC1X 8LR United Kingdom. Tel: Fax: E-mail: Web:

+44 20 7067 2920 +44 20 7067 2921 intcap@intcap.com http://www.intcap.com

Biographies: Oliver Gupta is a researcher and advisor at Intellectual Capital Services, London, England. He holds a M.Sc. in Industrial Engineering and Business Management from the Technical University of Ilmenau, Germany. His work primarily concerns developing and implementing the Intellectual Capital perspective to help individuals and organisations achieve more effective and efficient value creation. His research focuses on inter-organizational networks, intellectual capital modelling, knowledge management and strategy for the digital business. His award winning research has been published in the Journal of Intellectual Capital. Dr. Steve Pike is Director of Research at Intellectual Capital Services researching, developing and implementing intellectual capital management, KM and measurement methodologies for the emerging business environments. He has particular expertise in KM having published articles and book chapters on KM. He has presented world-wide, most recently on KM and Organisational Learning at the World Knowledge Forum. He also has trained in measurement approaches based on measurement theory, axiology and multi-attribute value theory. He is co-author of several books chapter and numerous articles and papers on intellectual capital, intellectual capital measurement and disclosure. Göran Roos is Chairman of Intellectual Capital Services, London, England, and one of the founders of the modern intellectual capital science. Mr Roos is a part time visiting Professor at the Helsinki School of Economics as well as a part time visiting Intellectual Capital Associate at Mt. Eliza Business School in Melbourne. He has worked as consultant in most OECD countries and has served in management positions in several European and US-based corporations. He is author and co-author of numerous books and articles on Intellectual Capital, Strategy and Ebusiness.

Page 8 of 8