voice of evidence
Editor: Forrest Shull
N
Fraunhofer Center for Experimental Software Engineering, Maryland N
[email protected]
What Do We Know about Knowledge Management? Practical Implications for Software Engineering
Torgeir Dingsøyr, Finn Olav Bjørnson, and Forrest Shull
T
he past decade has seen much discussion regarding the knowledge economy, the importance of knowledge-intensive work, and how companies in the knowledge society can benefit from managing their knowledge better. Some of knowledge management’s (KM’s) main proponents, Thomas Davenport and Laurence Prusak, define it as “a method that simplifies the process of sharing, distributing, creating, capturing, and understanding a company’s knowledge.”1 Thus, KM is ideally an overarching idea with a large potential impact on the everyday practice of modern working life. Given that software engineering is definitely knowledge intensive,2 we would expect an especially large impact in this field. Important KM concepts have informed some key software engineering practices, such as early research on managing software engineering experience in an “experience factory.”3 More recently, in a 2002 special issue of IEEE Software, Ioana Rus and Mikael Lindvall argued that software companies can decrease the time and cost for development, increase quality, and make better decisions if they manage their knowledge better.4 Software companies require knowledge about new technologies, their customers’ domains, local processes and practices, and other information. They also need effective platforms for making knowledge available for software teams. 100 I E E E S O F T W A R E
Published by the IEEE Computer Society
But what do we know about these approaches and how they work in practice? What are the main findings from research on KM in software engineering, and what are these findings’ practical implications? To answer these questions, we first look at different KM approaches.
KM Approaches
Michael Earl has classified KM methods or approaches into three broad categories (technocratic, economic, and behavioral) and seven schools,5 as Table 1 shows. Each school has a particular focus or aim. The schools aren’t competitive or mutually exclusive; a robust KM strategy will combine aspects from multiple schools. The three technocratic schools focus on information technology in supporting employees in knowledge work: N The systems school focuses on information and knowledge-based systems. The central idea is to codify knowledge in databases, repositories, or expert systems. N The cartographic school makes knowledgeable people in an organization accessible to each other for advice, consultation, or knowledge exchange through skills management systems. N The engineering school focuses on processes— for example, by making descriptions of them available for employees through electronic process guides. The commercial school (in the economic category) focuses on how knowledge assets relate to income in organizations. An example of this would 0 74 0 -74 5 9 / 0 9 / $ 2 5 . 0 0 © 2 0 0 9 I E E E
VOICE OF EVIDENCE
be an organization tracking employee knowledge (intellectual capital) to capitalize on intellectual property rights. The behavioral schools focus on orchestrating knowledge sharing in organizations: N The organizational school focuses on describing the use of organizational structures (networks or communities of practice) to share or pool knowledge. N The spatial school focuses on designing office space to foster knowledge sharing. N The strategic school sees knowledge as the main source of value creation and organizes companies according to this criterion.
Table 1 Schools of knowledge management5 Category
School
Focus
Aim
Unit
Technocratic
Systems
Technology
Knowledge bases
Domain
Cartographic
Maps
Knowledge directories
Enterprise
Engineering
Processes
Knowledge flows
Activity
Economic
Commercial
Income
Knowledge assets
Know-how
Behavioral
Organizational
Networks
Knowledge pooling
Communities
Spatial
Space
Knowledge exchange
Place
Strategic
Mind-set
Knowledge capabilities
Business
0 3
Systems
6
3 0
1
The work on defining KM schools shows that multiple approaches have been taken. It’s also important to look at how they’ve been used in software engineering.
What We Know about KM
To determine the current state of research, we conducted a systematic review of empirical studies of KM in software engineering up to 2006.6 We identified 25 studies and 41 lessons-learned reports. The studies discovered during this review and analyzed in this article are listed by school in a Web appendix to this article at www.computer. org/software/webextra.html. Figure 1 categorizes these studies and reports according to the different schools. Most focused on the three technocratic schools (systems, cartographic, and engineering). Conversely, no studies or lessons learned focused on the commercial school. Future studies in this area could help provide important information about the return on investment from using KM systems in our field.
Technocratic-School Studies A closer examination of the studies lets us focus on the issues these evidence sources address. In the systems school, the discussion has generally focused on knowledge repositories’ usefulness. Several voices have claimed that such repositories are expensive to establish and often end up as information graveyards. However, the studies involving software engineering contain some positive results, showing that software engineers use knowledge repositories
9
Cartographic Engineering Organizational 12
Spatial
20
1 2
Commercial 0
9
Strategic
(a)
(b)
0
Figure 1. Knowledge management literature analyzed for this article. We examined (a) empirical studies and (b) lessons-learned reports, categorized by knowledge management school.
years after their introduction, and users report several ways of employing them. Some studies also present guidelines for developing knowledge repositories, which can be valuable for software companies. A single study in the cartographic school demonstrates that this school also has relevance for software companies. Software engineers found a particular skills-management tool to be useful for not just resource allocation but also identifying experts for problem solving, upgrading skills, and identifying new project opportunities. The many studies in the engineering school focus on several distinct areas. Five focus on managing knowledge about software development processes. One main finding is that to improve software development processes, both tacit knowledge (that is, the unwritten knowledge in the heads of experienced personnel) and explicit knowledge (knowledge that’s been written down and documented) must be managed. This is supported by another study on using defined processes
in software companies. This study found that such processes must be supported with collaborative social processes to promote organizational learning. Another study claims that a pure focus on knowledge codification will lead to inefficient knowledge sharing in software teams. Yet another group of studies describe various approaches to a central learning process for software teams— namely, project postmortem reviews or retrospectives. We found several guidelines on how to conduct such reviews and how to ensure that they support learning processes at individual, team, and organizational levels.
Behavioral-School Studies The three studies in the organizational school underline the importance of connecting humans via social networks when introducing new software engineering methods. They specifically emphasize that building company networks on existing social networks increases the chance of success. May/June 2009 I E E E S O F T W A R E
101
VOICE OF EVIDENCE 5.0
Current situation
4.5
Future importance
4.0
Gap
3.5 Score
3.0 2.5 2.0 1.5 1.0 0.5 0
System
Cartographic Engineering Schools
Organizational
Spatial
Figure 2. The current and future importance of selected knowledge management schools from a survey of 15 small and medium companies. The systems and engineering schools have the largest gap between current and future importance.
Current situation 1 2 3 4 5
Knowledge management approaches
Future importance 1 2 3 4 5
1.1 Knowledge repositories (knowledge database/wiki/intranet) are important for sharing knowledge in our company Figure 3. A question from our survey. We designed four questions to determine current practice and future importance of the systems school.
We found no studies on KM in the spatial school. As for the strategic school, the studies have widely different focuses. A study to find factors contributing to successful KM implies that a company must consider technological and organizational issues as well as human factors when deciding on a strategy. Another study suggests that to improve practice, the learning process must be an ongoing interaction between different learning processes. KM isn’t an “implement and be done with it” project; it must be sustained for organizations to reap the best results. Furthermore, we also found evidence in this school that software companies must manage both tacit and explicit knowledge properly to achieve an overall good KM strategy.
Practical Implications
Having reviewed the research literature, we surveyed 15 small and medium companies in Norway, Belgium, and Cyprus to get a 102 I E E E S O F T W A R E
w w w. c o m p u t e r. o rg /s o f t w a re
sense of how these ideas have penetrated into practice. We based the survey on the five schools most relevant to software development departments to identify where companies saw the largest potential for improvement. We developed a questionnaire that scored the companies on their current situation and how they imagined their future situation (see Figure 2), using four questions for each school. We didn’t mention the schools themselves in the questionnaire. Through this survey, we got an impression of each school’s perceived usefulness from (1) strongly disagree to (5) strongly agree (see Figure 3 for an example). Our survey showed that most companies already used some elements of each school. The results do indicate that some gap exists, at least among the sample members. For example, our literature search showed that the systems and engineering schools are relatively well studied. For the systems school in particular, the research
has shown several examples of benefits due to using knowledge repositories. In the engineering school, one of the most straightforward yet effective KM practices could be postmortem reviews or project retrospectives. Most companies do development through projects, and sharing experience across projects would be a natural first step toward a learning organization. Still, our survey results indicate that the largest gaps between current practice and desired future situations occur in the systems and engineering schools. So, although companies view the systems and engineering schools as the most important for their futures, those schools also have the largest potential for improvement. Other lessons emerged from our review. One practical implication was pure awareness about the variety of approaches. Many companies aren’t deliberately examining how many important issues are covered by the KM solutions they select. We hope that the identification of schools of thought in this area provides a helpful roadmap for practitioners in thinking about the most important issues they need to address. In our experience, managers tend to value explicit knowledge more than tacit knowledge, and one main finding expressed throughout the studies we surveyed is that focusing on one type of knowledge won’t likely succeed. So, companies should pick solutions that emphasize both tacit and explicit knowledge when planning KM initiatives to support software engineering. An important step is relating these findings to current software engineering trends. Looking more closely at traditional versus agile development practices, we see that these build on different approaches to KM.7 The technocratic schools are closely related to traditional software development, whereas the behavioral schools are more related to the agile approach. A main consideration for practitioners is thus that organizations developing software through traditional approaches will probably benefit more from the technocratic schools, whereas agile teams will benefit more from behavioral schools. Agile teams have often implemented good practices for KM through techniques such as retrospectives, frequent meetings, and colocated teams, but agile methods provide little support for KM beyond team level. Here, the organizational school can be a valuable resource,
VOICE OF EVIDENCE
building on experience other fields have with managing knowledge between crossfunctional teams.
W
e’ve seen many claims about KM’s benefits in software engineering, such as decreased time and cost for development, increased quality, and better decision-making abilities. Although we can find some success stories illustrating these claims, particularly on aspects related to the systems and engineering schools, more research is necessary to explore the intersection between each school and the software engineering field. Researchers should continue to emphasize the need for a broad focus across multiple KM schools to succeed in improving KM’s practical application in software engineering.
References 1. T.H. Davenport and L. Prusak, Working Knowledge: How Organizations Manage What They Know, Harvard Business School Press, 1998. 2. A. Aurum et al., Managing Software Engineering Knowledge, Springer, 2003. 3. V.R. Basili, G. Caldiera, and H.D. Rombach, “The Experience Factory,” Encyclopedia of Software Engineering, vol. 1, J.J. Marciniak, ed., John Wiley & Sons, 1994, pp. 469–476. 4. I. Rus and M. Lindvall, “Knowledge Management in Software Engineering,” IEEE Software, vol. 19, no. 3, 2002, pp. 26–38. 5. M. Earl, “Knowledge Management Strategies: Toward a Taxonomy,” J. Management Information Systems, vol. 18, no. 1, 2001, pp. 215–233. 6. F.O. Bjørnson and T. Dingsøyr, “Knowledge Management in Software Engineering: A Systematic Review of Studied Concepts and Research Methods Used,” Information and Software Technology, vol. 50, no. 11, 2008, pp. 1055–1168. 7. S. Nerur and V. Balijepally, “Theoretical Reflections on Agile Development Methodologies,” Comm. ACM, vol. 50, no. 3, 2007, pp. 79–83.
Torgeir Dingsøyr is a senior scientist at SINTEF Information and Communication Technology and an adjunct associate professor in the Department of Computer and Information Science, Norwegian University of Science and Technology. Contact him at
[email protected].
LOYAL OPPOSITION
Continued from p. 104 unauthorized access, whether accidental or deliberate, to programs and data. To investigate vulnerability, I analyzed data from the US National Vulnerability Database (http://nvd.nist.gov/nvd.cfm), which is hosted by the National Cyber Security division of the US Computer Emergency Readiness Team (US-CERT). The database integrates all publicly available US government vulnerability databases. It lists each vulnerability type once. For example, if CERT is notified 300 times of a potentially damaging type of computer vulnerability, it lists that vulnerability only once in the database. I aggregated the known vulnerability types for RedHat Linux and Windows systems reported during 1997–2005. The study included 1,048 vulnerability types for RedHat Linux and 552 for Windows 2000 and Windows 2003. The types fall into three categories: high, moderate, and low severity. I based each type’s scoring on the Common Vulnerability Scoring System (CVSS),10 a global standard. Between 2002 and 2005, the total number of vulnerabilities for Linux rose dramatically from 67 to 333, while that of Windows rose from 69 to 86. I also found that for high severity, Linux experienced a rise from 31 to 126 while Windows experienced a rise from 38 to 53. I found similar results for low and medium severity.
T
herefore, unless someone can show that Windows systems’ vulnerabilities are underreported, my study doesn’t support the assertion that open source soft-
ware, represented by Linux, is less vulnerable than Windows systems. It also casts doubt on the global assertion that Linux’s quality is better than that of Windows. Could these results hold true for other categories of open source and closedsource software? References 1. R. Glass, “A Look at the Economics of Open Source,” Comm. ACM, vol. 47, no. 2, 2004, pp. 25–27. 2. G. Gousios et al., “Software Quality Assessment of Open Source Software,” Current Trends in Informatics: 11th Panhellenic Conf. Informatics (PCI 07), vol. A, New Technologies Publications, 2007, pp. 303–315. 3. D. Spinellis, Code Quality: The Open Source Perspective, Addison-Wesley, 2006. 4. F. Shull, “Who Needs Evidence Anyway? IEEE Software, Sept.–Oct. 2007, pp. 10–11. 5. B. Weiner, Open Benchmark: Windows NT Server 4.0 and Linux, Mindcraft, 1999; www. mindcraft.com/whitepapers/openbench1.html. 6. Linux versus Windows NT: The Verdict, Bloor Research, 1999; www.bloor-research. com/research/white-paper/188/linux-vwindows-nt-the-verdict.html. 7. I. Samoladas et al., “Open Source Software Development Should Strive for Even Greater Code Maintainability,” Comm. ACM, vol. 47, no. 10, 2004, pp. 83–87. 8. H. Thompson, Reliability: Analyzing Solution Uptime as Business Needs Change, Security Innovation, 2005. 9. D. Spinellis, “A Tale of Four Kernels,” Proc. 30th Int’l Conf. Software Eng. (ICSE 08), ACM Press, 2008, pp. 381–390. 10. P. Mell, K. Scarfone, and S. Romanosky, A Complete Guide to Common Vulnerability Scoring System Version 2.0, Forum of Incident Response and Security Teams, 2007; www. first.org/cvss/cvssguide.html. Adenekan (Nick) Dedeke is a visiting lecturer
of information systems and operations management in Northeastern University’s College of Business Administration. Contact him at
[email protected].
IEEE Software (ISSN 0740-7459) is published bimonthly by the IEEE Computer Society. IEEE headquarters: Three Park Ave., 17th Floor, New York, NY 10016-5997. IEEE Computer Society Publications Office: 10662 Los Vaqueros Cir., PO Box 3014, Los Alamitos, CA 90720-1314; +1 714 821 8380; fax +1 714 821 4010. IEEE Computer Society headquarters: 2001 L St., Ste. 700, Washington, DC 20036. Subscription rates: IEEE Computer Society members get the lowest rate of US$51 per year, which includes printed issues plus online access to all issues published since 1988. Go to www.computer.org/subscribe to order and for more information on other subscription prices. Back issues: $20 for members, $163 for nonmembers (plus shipping and handling). Send undelivered copies and address changes to IEEE Software, Membership Processing Dept., IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854-4141. Periodicals Postage Paid at New York, NY, and at additional mailing offices. Canadian GST #125634188. Canada Post Publications Mail Agreement Number 40013885. Return undeliverable Canadian addresses to PO Box 122, Niagara Falls, ON L2E 6S8, Canada. Printed in the USA. Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for profit; 2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-party products or services. Authors and their companies are permitted to post their IEEE-copyrighted material on their own Web servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the first screen of the posted copy.
Finn Olav Bjørnson is a research scientist at SINTEF
Permission to reprint/republish this material for commercial, advertising, or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or
[email protected]. Copyright © 2009 IEEE. All rights reserved.
Forrest Shull is a senior scientist at the Fraunhofer Center for Experimental Software Engineering, Maryland, and director of its Measurement and Knowledge Management Division. Contact him at
[email protected].
Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per-copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.
Fisheries and Aquaculture. Contact him at finn.o.bjornson@ sintef.no.
May/June 2009 I E E E S O F T W A R E
103
