Developing knowledge management metrics for measuring ...

The current issue and full text archive of this journal is available at http://www.emerald-library.com

JIC 1,1

54

Developing knowledge management metrics for measuring intellectual capital Jay Liebowitz

University of Maryland-Baltimore County (UMBC), Baltimore, Maryland, USA, and

Ching Y. Suen

Centre for Pattern Recognition and Machine Intelligence (CENPARMI), Concordia University, Montreal, Quebec, Canada Keywords Intellectual capital, Metrics, Knowledge management, Human capital Abstract Measuring intellectual capital is a growing area of interest in the knowledge management field. Metrics are being developed and applied by some organizations, but there needs to be more research throughout the international community to better define these measures. One limitation of the current measures is that they do not necessarily address the ``knowledge level'' and the types of value-added knowledge that individuals obtain. This paper takes a look at the current measures, discusses some possible limitations, and suggests some additional measures that could be used in the intellectual capital area to complement existing measures.

Introduction Many individuals have stated that if you can't measure, you can't manage. Lord Kelvin once said: When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure, when you cannot express in numbers, your knowledge is of a meager and unsatisfactory kind. It may be the beginnings of knowledge, but you have scarcely, in your thoughts, advanced to the stage of a science.

Journal of Intellectual Capital, Vol. 1 No. 1, 2000, pp. 54-67. # MCB University Press, 1469-1930

In the emerging field of knowledge management (Liebowitz, 1999; 2000; Von Krough et al., 1999), metrics are needed to further convince management and stakeholders as to the value of knowledge management initiatives. In software management, metrics are typically ``direct measures'' or ``indirect measures''. Direct measures will normally be size-oriented metrics like thousand lines of code/person-month, defects/(thousand lines of code), and others. Indirect measures are function-oriented metrics such as the number of user inputs and outputs, function points per month, years of experience, number of dollars invested/used, years of education/research/management, and the like. There can also be human-oriented metrics that look at process measures. Some of these metrics fall into the knowledge management field, as they attempt to quantify and measure human capital and knowledge assets. Michael

Malone (1997) of MIT stressed the need for new metrics for a new age. He and Leif Edvinsson of Skandia developed intellectual capital measurements covering the financial, customer, human, renewal and development, and process areas. Baruch Lev (1997) of NYU, who is the Director of the Center for Intangible Assets Research, believes that the accounting profession needs new standards for capitalizing intangibles. Nick Bontis (Bontis and Girardi, 1998) of McMaster University and Director of the Center for Intellectual Capital indicates the strong need for measuring intellectual capital. Additionally, the Canadian Management Accountants (CMA, 1999) is completing a report on ``measuring knowledge assets'' as they feel intangible assets must be better measured. Lastly, the ICM Group (1998) conducted a study on measuring intellectual capital and found that collectively, companies are still measuring under the ``tangible'' assets scenario. Thus, from the studies indicated, a need exists for developing metrics to measure intellectual capital or knowledge assets. For the focus of this paper, we will concentrate on creating innovative metrics for helping to measure knowledge assets. We feel that these metrics, as proposed, have not really been discussed in the literature. Current measures for intellectual capital in use From the ICM Group study (1998) of what companies are currently measuring with respect to intellectual capital, the following metrics were found: (1) Value extraction: . Profits resulting from new business operations. . Return on net asset value. . Total assets. . Revenues resulting from new business operations. . Market value. . Patents pending. . Return on net asset resulting from new business operations. (2) Customer capital: . Market share. . Customer rating. . Satisfied customer index. . Number of new customers/new market/leads, etc. . Annual sales/customer. . Average customer size. . Average time from customer contact to sales response. . Ratio of sales contacts to sales closed.

Knowledge management metrics 55

JIC 1,1

56

(3) Structural capital: . Administrative expense/total revenues. . Processing time, outpayments. . Computers/employee. . Contracts filed without error. . Corporate quality performance. . Investment in IT. (4) Value creation: . Training expense/employee. . Average customer duration with the company (months). . R&D invested in basic research. . R&D invested in product design. . Investment in new product support and training. . Satisfied employee index. . Relationship investment/customer. . Training expense/administrative expense. . R&D invested in applications. (5) Human capital: . Average years of service with the company. . Number of employees. . Number of managers. . Revenues/employee. . Employee turnover. . Number of female managers. . Profits/employee. . Average age of employees. . Number of exempt full-time employees. . Average age of full-time exempt employees. . Percent of company managers with advanced degrees. Roos et al. (1998) developed the following indicators for components of intellectual capital: (1) Human capital (competence, attitude, intellectual agility): . Percent of employees with advanced degrees. . IT literacy.

Hours of training/employee. Average duration of employment. . Hours spent in debriefing. . Hours spent by senior staff explaining strategy and actions (overlap expertise). . Leadership index. . Motivation index. . Savings from implemented employee suggestions. . New solutions/products/processes suggested. . Background variety index (individual and group level). . Company diversification index. (2) Structural capital (relationships, organization, renewal and development): . Percentage of supplier/customer business accounted for. . Length of relationship. . Partner satisfaction index. . Customer retention. . Administrative expenses/total revenues. . Revenues from patents/software/data/databases/etc. . Processes completed without error. . Cycle/process times. . Percentage of business from new products. . Training efforts ± expense/employee, hours/employee. . Renewal expenses/operating expenses. . New patents/software/etc. filed. . .

In the CMA (1999) draft report on ``measuring knowledge assets'', other measures for intellectual capital were also cited, such as: . Number of new products. . Number of new customers. . Success ratio by dollars. . Percentage of customer business. . Productivity index. . Number of processes reviewed. . Number of processes changed. . Percentage rated acceptable at first review.

Knowledge management metrics 57

JIC 1,1

58

.

Ratio of temporary/total employment.

.

Number of patents filed.

.

Number of ideas implemented from the suggestion box.

.

Traditional quality indicators.

.

ISO and customer satisfaction.

The last major set of intellectual capital measures were compiled as a ``Universal Intellectual Capital Report'', developed by Leif Edvinsson of Skandia and Michael Malone of MIT. These metrics are: (1) Financial focus: .

Total assets.

.

Total assets/employee.

.

Revenues/total assets.

.

Profits/total assets.

.

Revenues resulting from new business operations.

.

Revenues/employee.

.

Customer time/employee attendance.

.

Profits/employee.

.

Lost business revenues compared to market average.

.

Market value.

.

Return on net asset value.

.

Return on net asset resulting from new business operations.

.

Value added/employee.

.

Value added/IT employees.

.

Investments in IT.

.

Value added/customer.

(2) Customer focus: .

Market share.

.

Number of customers.

.

Annual sales/customer.

.

Customers lost.

.

Average duration of customer relationship.

.

Average customer size.

.

Customer rating.

Customer visits to the company and the number of hits on the company's Web site. . Days spent visiting customers. . Customers/employees. . Revenue generating staff. . Average time from customer contact to sales response. . Ratio of sales contacts to sales closed. . Satisfied customer index (e.g. customer contact/support/service through electronic means, number of items of merchandise returned, number of refunds made, etc.). . IT investment/salesperson (and perhaps dollars used in advertisements and their effectiveness). . IT investment/service and support employee. . IT literacy of customers. . Support expense/customer. . Service expense/customer/year. . Service expense/customer/contact. (3) Process focus: . Administrative expense/total revenues. . Cost for administrative error/management revenues. . Processing time, outpayments. . Contracts filed without error. . Function points/employee-month. . PCs and laptops/employee. . Network capability/employee. . Administrative expense/employee. . IT expense/employee. . IT expense/administrative expense. . Administrative expense/gross premium. . IT capacity (CPU and DASD). . Change in IT inventory. . Corporate quality performance (e.g. ISO 9000). . Corporate performance/quality goal. . Discontinued IT inventory/IT inventory. . Orphan IT inventory/IT inventory. .

Knowledge management metrics 59

JIC 1,1

60

IT capacity/employee. IT performance/employee. (4) Renewal and development focus: . Competence development expense/employee. . Satisfied employee index. . Relationship investment/customer. . Share of training hours. . Share of development hours. . Opportunity share. . R&D expense/administrative expense. . Training expense/employee. . Training expense/administrative expense. . Business development expense/administrative expense. . Share of employees under age 40. . IT development expense/IT expense. . IT expenses on training/IT expense. . R&D resources/total resources. . Customer opportunity base captured. . Average customer age; education; income. . Average customer duration with company in months. . Educational investment/customer. . Direct communications to customer/year. . Non-product-related expense/customer/year. . New markets development investments. . Structural capital development investment. . Value of EDI system. . Upgrades to EDI system. . Capacity of EDI system. . Ratio of new products (less than two years) to full company product family. . R&D invested in basic research. . R&D invested in product design (e.g. dollars invested in changes of quality, quantity, and variety of products/designs/etc.). . R&D invested in applications. . .

.

Investments in new product support and training.

.

Average age of company patents.

.

Patents pending/software, data, databases developed.

Knowledge management metrics

(5) Human focus: .

Leadership index.

.

Motivation index.

.

Empowerment index.

.

Number of employees/employee shares of the company (percent of shares owned by employees, program for employees to buy company shares, etc.).

.

Employee turnover.

.

Average years of service with company.

. . .

Number of managers. Number of female managers. Average age of employees and number with pertinent experience in trade and IT.

.

Time in training (days/year).

.

IT literacy of staff.

.

Number of directors.

. .

Number of female directors. Number of full-time or permanent employees.

.

Average age of full-time or permanent employees.

.

Average years with company of full-time or permanent employees.

.

Annual turnover of full-time permanent employees.

.

.

. .

. . .

Per capita annual cost of training, communication, and support programs for full-time permanent employees. Full-time or permanent employees who spend less than 50 percent of work hours at a corporate facility. Percentage of full-time permanent employees. Per capita annual cost of training, communication, and support programs. Number of full-time temporary employees. Average years with company of full-time temporary employees. Per capita annual costs of training and support programs for fulltime temporary employees.

61

JIC 1,1

.

.

62

Number of part-time employees or non-full-time contractors, average duration of contract. Company managers with advanced degrees: business, science and engineering, liberal arts.

Discussion on current metrics and need for additional intellectual capital metrics In reviewing the metrics that have been proposed and are in use for measuring intellectual capital, several observations can be made. First, many of the metrics cited in Section 2 can readily be obtained. Most of the metrics presented in the above listings are quantifiable and are readily available. Second, many of the cited metrics lack ``creativity'' in terms of determining the size and growth of the organization's knowledge base. Most of the metrics are fairly straightforward and do not necessarily address the types of knowledge that produce the most value-added benefits for the organization. Last, various assumptions (some perhaps erroneous) may be made in terms of the current list of metrics. For example, assuming the average age of an employee to be ``young'' (let us say around 30) may not necessarily mean that the organization is a vitally strong, innovative company. To compensate for some of these potential drawbacks, new metrics should be created to address these concerns. For example, metrics to determine ``return on vision'' versus ``return on investment'' are being developed by such companies as Andersen Consulting. This is a different mindset than producing metrics to assess returns on investment. The following is a listing of possible new metrics to be part of the pot-pourri for measuring intellectual capital: . The number of new colleague to colleague relationships spawned ± this will hopefully encourage the exchange of tacit knowledge between knowledgeable individuals. The World Bank's and Johnson & Johnson's knowledge fairs/exchanges are geared to promoting an increase in these types of relationships and transfer of tacit knowledge. . The reuse rate of ``frequently accessed/reused'' knowledge. . The capture of key expertise in an online way (i.e. the number of key concepts that are converted from tacit to explicit knowledge in the knowledge repositories and used by members of the organization). . The dissemination of knowledge sharing (i.e. distribution of knowledge) to appropriate individuals. . The number of knowledge sharing proficiencies gained ± at Andersen Consulting, they have developed six levels of knowledge sharing proficiencies whereby one cannot be promoted unless one reaches at least level 5. . The number of new ideas generating innovative products or services.

.

.

.

.

.

.

The number of lessons learned and best practices applied to create value-added (i.e. decreased proposal writing/development time, increased customer loyalty and satisfaction, etc.). (The number of patents/trademarks produced + number of articles or books written + number of talks given at conferences or workshops or trade shows)/number of employees ± the higher the number, the better. (Professional development/training dollars + R&D Budget dollars + Independent R&D dollars)/number of employees ± the greater the number, the better. The number of ``serious'' anecdotes presented about the value of the organization's knowledge management systems. The number of ``apprentices'' that one mentors, and the success of these apprentices as they mature in the organization. Interactions with academicians, consultants, and advisors.

Application of a knowledge check for an organization Before applying some of these intellectual capital measures to an organization, a quick knowledge audit/check was conducted to determine the value-added potential of the organization. The KPMG Netherlands Online Assessment, called The Value Enhancer (http://kpmg.interact.nl/assessment/index.htm), was used to first determine whether the organization is still at base camp, a market-driven organization, a competency and process-based organization, or a continuous learning organization. The organization in question is the Department of Information Systems at the University of Maryland-Baltimore County (UMBC). Since one of the major goals of a university is to produce and create knowledge, we thought a major department at a university would be an ideal candidate to apply some of these intellectual capital measures. The Department of Information Systems is one of the largest departments on campus at UMBC with about 1,000 undergraduate majors, 150 Master's students, and 35 doctoral students in information systems. The department has 16 faculty and six full-time lecturers. Table I shows the questions and author responses, as related to The Value Enhancer and the department. Based upon these responses, The Value Enhancer indicated that the department has reached base camp and is about to transform itself to a marketdriven organization. This value-adding potential depends on both the service level and knowledge intensity of the organization. It turned out that those companies that score average will show knowledge value-adding performance. The score for the department was between sub- and average knowledge valueadding performance. Keeping this level of the ``knowledge organization'' in mind, several of the proposed intellectual capital measures could be applied to the department (at least in the case of the author). They are as follows (of course, other measures

Knowledge management metrics 63

JIC 1,1

64

In my company, authority is decentralized to the (business) unit level

Agree

The (business) units in my company have a great deal of freedom to act and have bottom-line responsibility for those actions

Disagree

The functional disciplines in my company are adding value to my business

Agree

The functional disciplines in my company are team-based rather than jobbased

Disagree

In my company, working in teams is the rule instead of the exception

Agree

The composition of teams in my company is governed by creating the right mix of competencies needed for the work at hand

Agree

My company has a clear understanding of the knowledge needed to achieve our company objectives

Agree

The creation of new knowledge in each area of my company is guided by a written, overall knowledge strategy

Disagree

My company knows what combinations of knowledge should be made to create value

Agree

My company has a structure in which the knowledge of individuals is easily shared and combined to take advantage of synergy

Disagree

My company has mapped its individual knowledge segments: everything individuals and systems know about a specific subject that relates to achieving our business goals

Disagree

My company holds a knowledge profile on every individual

Agree

The operational processes in my company are aligned with the goals of individual (business) units

Disagree

The operational processes in my company effectively deliver products and services

Agree

The corporate headquarters in my company adds value to my business

Agree

The corporate headquarters in my company deals with processes rather than functions

Agree

My company has identified the processes that are needed to achieve our long-term business objectives

Agree

Every individual in my company knows how he or she contributes to our long-term business objectives

Disagree

My company has mapped its current core competencies

Disagree

My company is enhancing its competencies to create new products and services

Agree

The people in my company are able to effectively work in teams

Agree

In my company, each team starts a new project by defining the needed competencies

Agree

My company has a complete competency profile for each individual

Agree

In my company, individuals are stimulated to develop new competencies based on future business demands

Agree

Table I. Quick knowledge check Individuals in my company are empowered to act on their own accord using The Value Enhancer

Agree (Continued)

My company encourages individuals to take charge of their own career path

Agree

In my company, individuals are held personally responsible for creating value

Disagree

In my company, systems are in place which provide ongoing feedback on individual performance

Agree

In my company, people act both as team leaders and as team members

Agree

Teams in my company possess a contagious team spirit

Disagree

People in my company relate to the company's overall purpose in such a way that they gain personal fulfillment

Agree

My company's culture encourages individuals to fulfill their personal values

Agree

My company provides individuals with exciting opportunities that challenge business-as-usual performance

Disagree

My company encourages individuals to jump outside their box

Agree

My company allows individuals to share in the wealth created by their efforts

Disagree

In my company, individuals receive ample recognition for value-adding performance

Disagree

like the dollar amount of sponsored research in the department, the number of students, the student/faculty ratio, number of scholarships, the amount of industry contacts, and others are examples of measures that should also be used in addition to the measures shown below): .

.

.

.

.

.

Number of new colleague to colleague relationships spawned: 25 in the university and 20 in industry/government in a year. Reused about 30 percent of the knowledge on research projects and teaching. Started a laboratory for knowledge management to use expert systems and knowledge management systems for capturing online expertise (e.g. in course advising, business protocol, etc.) Dissemination of knowledge sharing: taught about 70 students (probably half of them had knowledge sharing experiences with me and with their peers; 20 faculty/lecturers/doctoral/Master's students probably had knowledge sharing experiences with me). Generation of new products and services: 15 new ideas as related to research proposals and department projects (e.g. creating in two weeks the agreed copy and layout for a new ten-page marketing brochure for the department). Applying lessons learned: as editor-in-chief of the Failure and Lessons Learned in Information Technology Management Journal (Cognizant Communication Corp, NY), applied lessons learned to improve my

Knowledge management metrics 65

Table I.

JIC 1,1 .

.

66

project management abilities in acting as the project leader on several research projects). Number of articles/books written + talks = about 30 for the author in the year. Mentoring activities: two faculty members and two doctoral students.

With these measures determined, how can we interpret what they mean? Are there any baselines or benchmarks to measure against these numbers? We could certainly measure these numbers against the other colleagues in the department or against colleagues in similar departments at the institution or at similar universities. We could check some professional or ``standards-granting'' organizations, like the American Association for University Professors or the American Assembly for Collegiate Schools of Business (AACSB). In fact, checking the AACSB Web site indicates the following information regarding business accreditation standards for ``Intellectual Contributions '': Producing intellectual contributions represents a core set of responsibilities of higher education for business. Such contributions improve management theory and practice, and support the present and future quality of instruction at all institutions. A wide variety of intellectual contributions is appropriate in academic institutions. For purposes of this standard, contributions have been grouped as follows: basic scholarship, applied scholarship, and instructional development. The school's mission influences the relative emphases among the types of intellectual contributions. All schools should have some of their intellectual contributions committed to instructional development. IC.1: Faculty members should make intellectual contributions on a continuing basis appropriate to the school's mission. The outputs from intellectual contributions should be available for public scrutiny by academic peers or practitioners.

Other studies have been conducted to look at faculty productivity such as: . According to the National Center for Education Statistics report on ``Profiles of Faculty in Higher Education Institutions 1988'', it indicates that full-time faculty members at all institutions worked an average of 53 hours a week and spent 56 percent of their time in instructional activities. . American Association of University Professors' 1994 report found that faculty members, on average, work from 47 to 57 hours a week. . The State Council for Higher Education in Virginia (SCHEV) in their faculty activity survey in March 1997 found, for example: faculty spend about 32.9 hours a week for teaching, 11.8 hours for research/week, and 10.2 hours for service/week; full professors spend 55.2 percent of the week toward teaching, 23.6 percent for research, and 21.2 percent for service for 56 median hours of work per week. In surveying the literature, very little has been done in terms of an intellectual capital audit on the faculty in order to determine the kinds of metrics we have proposed in this paper. Studies need to be conducted in order to determine baselines for comparing these types of measures and results.

Summary This paper tried to indicate the current types of intellectual capital measures being developed and used, as well as proposing some additional measures. These new measures were also applied to a university department in order to start to develop an intellectual capital audit. The knowledge management community needs to be responsive to the needs of management in the organization by trying to adequately measure the organization's intellectual capital and assess the worthiness of the knowledge management initiatives. Developing metrics and studies for measuring intellectual capital will help to consolidate the knowledge management field and give the discipline further credibility. References Bontis, N. and Girardi, J. (1998), ``Teaching knowledge management and intellectual capital lessons: an empirical examination of the tango simulation'', 3rd World Congress on Intellectual Capital, McMaster University, Canada. Canadian Management Accountants (CMA) (1999), ``Focus group draft: measuring knowledge assets'', April 16. ICM Group, Inc. (1998), ``What are companies currently measuring?'', http://www.icmgroup.com/ presentpub/LES_MEASUREMENT Lev, B. (1997), ``The old rules no longer apply'', Forbes Magazine, April 7. Liebowitz, J. (Ed.) (1999), The Knowledge Management Handbook, CRC Press, Boca Raton, FL. Liebowitz, J. (2000), Building Organizational Intelligence: A Knowledge Management Primer, CRC Press, Boca Raton, FL. Malone, M. (1997), ``New metrics for a new age'', Forbes Magazine, April 7. Roos, J., Roos, G., Dragonetti, N. and Edvinsson, L. (1998), Intellectual Capital: Navigating in the New Business Landscape, New York University Press, New York, NY. Von Krough, G., Roos, J. and Kleine, D. (Eds) (1999), Knowing in Firms: Understanding, Managing, and Measuring Knowledge, Altamira Press.

Knowledge management metrics 67