A Review-Report-Oriented Knowledge-Management ...

A Review-Report-Oriented Knowledge-Management System Yutaka Kudo *, Chiaki Hirai *, Yukari Furuhata **, Tadashi Watanabe **, and Osamu Ohno ** * Systems Development Laboratory, Hitachi Ltd., [email protected] ** Government & Public Corporation Information Systems Division, Hitachi Ltd.

Abstract: We have developed a model of a review-report-oriented knowledge spiral that aims at guiding projects to their goals steadily as well as promoting knowledge reuse in software projects. In order to attain these aims, we focused on reports of design review. The features of our model are (1) to manage the discussion processes in design review, as well as documents and program files, (2) to store the discussion processes written in the review reports and to share them in an organization, and (3) to provide evaluation standards to help a manager manage the discussion processes. We also developed a WWW-based knowledge-management system based on our model, and it is being applied in more than 50 projects. And the system has been qualitatively shown to improve the quality of software development. We also show a measure of quantitative evaluation for our proposal.

1.

Introduction

With the dramatic advances in Internet/Intranet technologies, the number of distributed software development projects has been increasing. And Internet and Intranet groupworking tools are becoming more widespread [1] and can often provide a low-cost, easy to implement solution to share and utilize knowledge in world-wide organization. On the other hand, in addition to the conventional technology of software components and its reuse technology around CASE tools, the research on knowledge acquisition, retention, and reuse in a software project has been more active. Maurer has been trying to improve the software development efficiency by putting the development process on workflow management system and presenting the necessary knowledge in each task to the members[2]. Conklin structuralized discussions in the upper stream of software development and proposed the IBIS (Issue Based Information Structure) model which enables understanding of the discussion context by visualizing the discussion process[4]. Komiya improved the IBIS model to make software design more efficient by recording and reusing the discussion process under a distributed development environment[3]. We assume that in order to improve software quality, discussion during design reviews must be managed. We thus developed a knowledge spiral model for utilizing recorded discussions. Then we developed a Web-based knowledge management system based on this model. And this system is being used in more than 50 software projects. In this paper, firstly we clarify the knowledge source in each phase of the software development process and summarize the characteristics of the review report from the viewpoint of the knowledge source. We propose a review-report-oriented model of a knowledge spiral and explain the functions and behavior of our knowledge management system with our model. Finally, we present the evaluation method of our activity and experiences obtained in our activity.

2. Proposal of Review-Report-Oriented Knowledge-Spiral Model 2.1. Development Process as Knowledge Creating Process Software development is essentially a knowledge-creating process. Designers should define problems clearly, 1

gather information to solve them, and choose proper solutions or create new ones. This process is the activity to keep producing knowledge toward the final goal. We use “problems” here as the items for assurance of software quality and feasibility. The designers lay the foundation for new knowledge creation by using past development experience or technology that has been piled in the organization. Conventional software-development models often measure progress according to the extent of artifact registration. This approach, however, does not guarantee that the design will be fully discussed. 2.2. From Artifact Management to Discussion Management Nonaka modeled the knowledge-creating process as a tacit and explicit conversion process. This knowledge conversion is called a “Knowledge Spiral” because new knowledge is piled upon existing knowledge[6]. When Nonaka’s knowledge spiral is adopted as a knowledge-creating process in a software project, it is important that an organization provides the discussion place and rotates the knowledge spiral in order to improve development efficiency. In other words, project members need a process model to encourage knowledge sharing and knowledge creation. In light of the above-mentioned requirement, we have added a mode for checking the discussion condition and have developed our original knowledge spiral model. And we have solved the problems that occur in conventional software-development-process models on the hypothesis that the quality and quantity of discussion rather than those of artifacts should be managed. 2.3. Definition of “KNOWLEDGE” There are several definitions of the word “knowledge”, and we adopt Davenport’s definition[5]. According to his definition, knowledge is information combined with experience, context, interpretation, and reflection. It is a high-value form of information and it can be readily applied to decisions and actions. In compliance with Nonaka’s definition[6], there are two types of knowledge: explicit and tacit. Explicit knowledge is objective and more formal while tacit knowledge is more personal and subjective. Explicit knowledge can be expressed in words and numbers and can be easily communicated and shared in the form of hard data, scientific formula, codified procedures or universal principles. Tacit knowledge is highly personal and hard to formalize, making it difficult to share with others. It is embedded in an individual experience and involves intangible factors such as personal belief, perspectives, and values. Subjective insights, intuitions, and hunches fall into this category of knowledge. Knowledge also exists in software projects in various forms. Know-how obtained “on the development spot” is tacit knowledge. Proposal documents, design documents, specifications, program sources, and review reports are explicit knowledge. 2.4. Knowledge in a Software Project Knowledge and its source Table 1 lists knowledge sources in each phase of software development. This is the result of the questionnaire targeting 30 developers in HITACHI and it shows that sometimes documents themselves, such as proposal documents and specifications, as well as experience from past developments, such as key persons and review reports, are reused. This questionnaire clearly shows that the review report is the knowledge source through all phases.

2

Table 1 Process and Knowledge in Use Development Phase

Wanted Information

Knowledge Source

System proposal

Preference of customer, Past business result

Proposals, Contracts, Review reports, Key-person

Cost estimation, Development organization

Plans, Review reports, Key-person

Planning Design & Development Maintenance & Enhancement

Design basis, Architecture, Influence in changing design

Design documents, Specifications, Review reports, Source programs Design Documents, Specifications, Review reports, Source programs

Testing method, Debug know-how

Test specifications, Bug reports, Review reports

Feasible design, Performance and quality assurance

Test

Characteristics of review reports as knowledge Review reports have the following characteristics: (1) They contain not only a conclusion but also its decision process. (2) They contain both successful ideas and unsuccessful ideas. (3) They are described in developers’ actual terminology. (4) They give links to the review materials and/or the mentioned information source. (5) They cover ALL phases of the software development process. In order to acquire information as knowledge, it is important to virtually experience the context from the record of the past experience[6]. This means that intentions and contexts must be understood in order to practically reuse knowledge. It is remarkable that review reports contain much information that represents intentions and contexts ((1), (2), (3), and (4) above); therefore we can easily extract the tacit knowledge even though review reports are explicit knowledge. 2.5. Knowledge Spiral Model Discussions during software development are made in the place of design review, and according to the result of questionnaire shown in Table 1, the review reports are recognized as the knowledge source for many designers. For this reason, we developed a review-report-oriented knowledge spiral model to manage the discussions in design review. This model consists of four modes as shown in Fig 1: knowledge mining, knowledge utilization,

knowledge

combination,

1. Mining

2. Utilization on the spot

Designers

Development experiences Designers

•Points of proposal •Design policy •…

•Proposal docs •Design docs Review Report DB

Review reports

Comparison with plan

and

Design and making Review materials

Review materials

Discussion and making decision on the review Review report registration

coverage check of discussions Which items have been reviewed? Manager

(1) Knowledge Mining Designers advance their designs by acquiring various types of information through their experiences. For instance, a designer

Design review

4. Coverage Check

With customers

3. Combination

1 review / 1 cycle

Fig. 1 ReviewReview-report -oriented Knowledge Spiral

acquires knowledge from meeting customers, from actual experiences on the spot, and from all sorts of documents and review reports.

3

(2) Knowledge Utilization A designer can design software by utilizing knowledge and reusing past documents that are acquired in knowledge-mining mode. (3) Knowledge Combination Software designs made in knowledge-utilization mode are discussed and some comments are added to them. The contents of discussion, comments, and referred information are recorded in the review report. (4) Checking the coverage of discussions A project manager confirms whether the items which should be discussed in the design review were actually reviewed or not. He coordinates design work for the next design review depending on this confirmation.

3.

Design of Support Functions for a Knowledge Spiral We developed a system for managing review reports so that we can support our model proposed in Chapter 2.

In this chapter, we describe some functions of our system. (1) Knowledge Mining In order to promote knowledge creation, the searching function of our system must enable the users to find target information quickly. A characteristic of design review is that one subject is discussed across the reviews and some subjects are discussed in only one review. Because discussions about a certain design exist across several review reports, users must open several review reports. So our system extracts each discussion part about a specified subject and combines them into one list so that the users can save time when searching target information. (2) Knowledge Utilization To learn knowledge efficiently, users must be able to virtually experience the context of designs from the record of a designer’s past experience[6]. It is thus necessary for users to know not only the conclusion of a discussion but also the decision process. The design process contains important clues in deciding whether a certain design should be reused or not. So our system provides a function that stores both documents and review reports with a link between them so that users can easily refer to both of them. (3) Knowledge Combination To enable our system to extract discussions about a specified subject from review reports, we established the format of a review report (Fig. 2). The review report consists of three parts: review information, discussion contents, and attached documents. The Review-information part explains the review itself and consists of date, attendance, and so on. Discussions in the review are written in the discussion-contents part. Hyper-links to the review materials are inputted in the attached-documents part. Each discussion is stored in each field of a database so that the

Review Info. Title: Date/Time: Reviewer: Discussion Contents

Attached Documents

Place: Reporter: Main Topic:

1. Subject Argument (Conclusion/Reason) 2. Subject Argument (Conclusion/Reason) 3. … 1. Document File or URL 2. …

Fig. 2 Format of Review Report

system can retrieve them independently and recompose them flexibly. The discussion-contents part is divided into discussion pieces and stored in each individual record. (The delimiter of the division is a headline with the item number.) 4

(4) Checking the coverage of discussions The table that defines what items should be discussed in a design review is called the review-item-definition table. So that a manager can follow the progress of discussions, it is necessary to make a distinction between discussed items and undiscussed ones. Our system thus checks each review item in the definition table and each headline in the review reports, and it indicates how many times each item has been reviewed.

4. Implementation of the Review-Report-Oriented Knowledge-Management System 4.1. System Outline Figure 3 gives an overview of storage and retrieval within our system. The left side of the figure illustrates review report management. Projects submit review reports to the system, and the manager checks the coverage of discussions and feeds back to the project in order to guide to its goal steadily. The right side of the figure shows knowledge sharing. Because the system has been developed using WWW-based technology, other projects and/or project member can retrieve discussion process as knowledge even though projects are distributed over the world. Review report management

Knowledge sharing Di s c u s s i on Pr o c e s s

Review-report-oriented Knowledge management system

Project

Review Review Report Report Regisgration Regisgration (Review (Review report report and and documents) documents)

Registration CGI prog.

Review Reports / Documents database

Feedback Manager

Checking Checking Coverage Coverage of of Discussions Discussions

Project Info. database

Subj e c t Concl u s i on Reason Subj e c t Concl u s i on Reason Subj .e .c .t Concl u s i on Reason ... ...... ... ...... ... ...

Coverage Check CGI prog.

Standard Review -item-difinition table

Knowledge Search CGI prog.

Other projects / Project member

Knowledge Knowledge Mining Mining

Fig. 3 Storage and retrieval within the reviewreview-reportreport-oriented knowledge management system Our system was implemented by using a WWW server, CGI programs, and RDBMS. It can extract target-discussion parts by storing each discussion in the review reports in the corresponding database field. Users can access our system through a Web browser. Knowledge is shared among the projects by storing all of the review reports and attached documents in an organization. 4.2. From Storing Discussion Data To getting Outcome During knowledge-combination mode, review reports are stored in a database by users. An example of discussion-contents part is shown in Fig. 4(1) and that of stored data is shown in Fig. 4(2). During knowledge mining mode, users use a search function. This database scheme enables our system to extract discussion pieces, including user-specified search words, and to combine them into one list. figure 4(3) shows an example of search result for “DirTree”. As shown in Fig. 4(3), each applicable discussion piece is extracted from two different review reports. The coverage of the review item (Fig. 4(4)) can be outputted by comparing "subject" in Fig. 4(2) and each item of the review-item-definition table.

5

(1)

(3)

Example of Review Report (Contents Part)

subject sub subject Review result

1. User Interface 1.1 GUI for Search Function Directory-type interface . Almost all users in HITACHI can’t find search keyword by themselves. Directory-type can provide some choices for user. 1.2 How to define Directory Directory information is stored in “DirTree” table. “DirTree” consists of “myNode”, “parentNode” and “Name”.... 2. DB Design ...

Search Results (Keyword: “DirTree”) 1 record

lABC Directory Service System (for ABC Co.,Ltd)(by A03) “Feasibility review for the Directory function” (1999/10/20)

1 record

1.2 How to define Directory Directory information is stored in “DirTree” table. “DirTree” consists of “myNode”, “parentNode” and “Name”....

1 record

lABC Directory Service System (for ABC Co.,Ltd)(by A03) “Performance review for the Directory DB” (1999/10/13) 3.4 Performance Design for “DirTree” "DirTree" will be accessed in every operation of opening and closing directory node. ...

Structuralization

(2) Review Report Database

(4) subject

sub subject

review result

Review Item Coverage Report

1. User 1.1 GUI Interface for Search Function

Directory-type interface . Almost all users in HITACHI can’t find search keyword by themselves. Directory-type can provide choices for user.

1. User 1.2 How to Interface define Directory

Directory information is stored in “DirTree” table. “DirTree” consists of “myNode”, “parentNode” and “Name”....

2. DB Design

…

Review sub-Item

User Inrerface

GUI for Search Function

6

GUI for Registration

0

How to define Directory

3

DB Design

…

Coverage rate : 68%

Review Item

count

Capacity Estimate

12

Performance Design

8

…

3

...

Fig. 4 Searching and Coverage Reporting by ReRe-composition of Discussion Pieces

5. User Interface 5.1. Review-Report Editor The screen image of the review-report editor is shown in figure 5. Users can edit review reports and submit them to the server using this interface. This editor screen consists of 3 frames. Command buttons such as copy/delete/submit is arranged in upper frame. Left frame shows a list of registered review reports, and users can select one report from this list to see or to edit it. Right frame is the review-report form. Users fill in blanks to make review report, and submit it.

• Submit • Title • Send-to • Date/Time /Place/Submitter • Reviewer(Customer) • Reviewer(Hitachi) • Outline • Contents (Result & Reason) • Referred Documents (Filename/URL) List of registered review reports

Registration form for review report

Fig. 5 Example of a reviewreview-report editor

6

5.2. Search function The screen image for search function is shown in figure 6. Users can make search conditions and search registered review reports for their necessary knowledge. The feature of this search function is shown below. (1) Showing only matched paragraphs from registered review reports on one page (2) Showing search results for only user-specified items The search screen consists of 2 frames. Upper frame is for making the search conditions, and users can specify search keywords and can select display items. The search result appears on lower frame. • Search Condition • Search Button • Display Condition 1 Record : matched paragraphs

title/send-to/date/reviewers/outline/ todo/attached-docs/contents

•Project Info. •Review Title (link to minute) •Conclusion(Left-side) •Reason(Right-side)

Fig. 6 Screen image for search function 5.3. Review-Item-Coverage Report The screen image of a review-item-coverage report is shown in figure 7. Users are mostly project managers and can see the progress of discussions. This screen consists of 2 frames. Users select a target project and a standard review-item-definition table in upper frame. Every project has its own standard table, so users usually select the standard table for the selected project. But there are often the cases that managers want to examine the coverage with different project’s standard review-item-definition table. The review-item-coverage report appears in lower frame. Our system compares each heading of registered review reports with each item of standard review-item-definition table, and shows how many times each item has been reviewed.

7

•Target Project •Standard review-item-definition table •Execute Button •Review-item-definition table

•Number of Reviews (Reports)

Fig. 7 Screen image of a reviewreview-itemitem -coverage report

6. Practical Evaluation 6.1. Current Status Our system is being applied in 50 software projects inside Hitachi. These projects vary from small-scale projects taking 6 person-months to develop an intra-office tool to large-scale projects taking 1000 person-months to construct information systems for public corporations. After application of this system started, the number of registered review reports exceeded 1000 in five months. Because it is only half a year since this system was applied to each project, we cannot evaluate it quantitatively. So we only describe the evaluation technique for this kind of system here. 6.2. Quantitative Evaluation Technique We can evaluate our system by examining whether the numbers of the bug occurrence and the delay of the development change depending on whether the project adopts our model or not. In the case that all project adopt our model, we can evaluate it by analyzing whether the outcome of a development project changes according to the degree of coverage of the review-item definition. 6.3. Qualitative Evaluation Reducing the number of problems Every project is evaluated monthly by examining the number of newly registered review reports. The rate of problem occurrence could be reduced by detecting unnatural events such as: (1) Number of review executions is small when a lot of design review must be carried out. (2) The order of the review process is unnatural (reverse). Standardization of the Review-Item Definition as the Intellectual Property of an Organization An increasing number of projects have tried to reuse the review-item-definition table of successful projects. This tendency is the result of our approach that helps designers to design software more efficiently. The review-item-definition table will become standardized for each domain and each platform. 8

7.

Conclusion We have proposed that the quality and quantity of a discussion rather than those of artifacts should be

managed. And we consequently developed a review-report-oriented knowledge spiral model and a support system that aims to promote reuse of discussions in design review as knowledge. The features of our model are (1) to manage not documents and program files but discussion processes in design review, (2) to store the discussions written in the review reports and to share them in the organization, and (3) to provide evaluation standards to help managers manage the discussion processes. Finally, we qualitatively evaluated the model and showed it to be effective. By managing review reports, discussions that were not managed in the conventional development process can be managed by our new system. And it is concluded that the new model and support system will effectively improve software productivity and quality.

Reference [1] R.Guensler,"Facilitating Workgroup Activities Through the Internet", ASCE Conference on Computing in Civil Engineering, June 17-19,1996,Anaheim,California, USA. [2] Frank Maurer , Barbara Dellen: Process Support for Virtual Software Organizations, SEKE’99 pp.175-179,1999 [3] Seiichi Komiya: A Model for Recording Software Design Decisions and Design Rationale, IEICE TRANS. INF. & SYST., VOL. E81-D, NO. 12 DECEMBER 1998,p.1350-1363 [4] Conklin J. and Begeman M. L.: gIBIS: A Hypertext Tool for Exploratory Policy Discussion, CSCW’88, 140-152,1988 [5] Davenport, Thomas H., Prusak Laurence: Working Knowledge,Harvard Business School Press, Boston, Massachusetts. ,1998, ISBN: 0875846556 [6] Ikujiro Nonaka, Hirotaka Takeuchi: The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation, Oxford University Press, Oxford. 1995,ISBN: 0195092694

9