Concepts for Reuse in the Experience Factory and ... - Semantic Scholar

Concepts for Reuse in the Experience Factory and Their Implementation for CBR-System Development Klaus-Dieter Althoff, Markus Nick, Carsten Tautz Fraunhofer Institute for Experimental Software Engineering (Fraunhofer IESE) Sauerwiesen 6, D-67661 Kaiserslautern, Germany email: {althoff, nick, tautz}@iese.fhg.de Abstract. An Experience Factory is an infrastructure for organizational learning in software development that includes an Experience Base as an organizational memory. We introduce a system architecture how such an infrastructure can be technically supported based on Case-Based Reasoning (CBR) technology. As a first instantiation of this architecture we present the CBR-PEB application, a publicly accessible WWW-based experience base for CBR-system development. Based on first experiments, some results about the evaluation of the success of this application are described. Keywords. Case-based reasoning, continuous learning from experience, experimental software engineering, experience factory, experience base, GQM-based measurement

1 Introduction In software development, reuse is regarded as a means to handle today's increasing quality, productivity, and time-to-market requirements [BCR94a, GFW94]. But reuse is not just a by-product of software development. To support reuse one has to establish an infrastructure aimed at capitalization and reuse of life cycle experience and products [GFW94, p62]. Basili and Rombach [BCR94a] call such an infrastructure experience factory (EF). The central element of the experience factory is the experience base in which the experience is stored in experience packages [BCR94a]. Griss et al. [GFW94] report that simple lists – like World-Wide-Web (WWW/Web) pages – can be used for a small experience base of 30–50 components, and for a larger library of 100–200 parts, a set of well organized and indexed catalogs has been used effectively. Beyond that, more powerful tools might be necessary. In [TA97, ABT97] it is shown that Case-Based Reasoning (CBR) is a very promising technology for the realization of an experience base, i.e., techniques, methods, and tools from the domain of CBR can be useful for building and maintaining an experience base. At the Fraunhofer IESE we are using CBR technology to develop experience bases not only for research purposes but also for industrial use. Thus, we are applying machine learning technology to real-life problems. In addition, the experience factory approach is very useful for operationalizing learning systems and/or knowledge-based systems in industrial/business environments [AW97, ABT97, BA98]. This might also be helpful in the context of building methodologies for

applying data mining (e.g., [ELS97, ELS98]) and machine learning in general [Icm98]. One focus of this paper is on describing a concrete experience base that has been developed for supporting CBR-system development. The emphasis is put on providing decision support for reusing existing CBR systems for the development of new systems. In order to make the system easily accessible by CBR developers all around the world, the system has been made publicly available over the WWW. By this, a service for the whole CBR community is offered at no charge. Maybe for other dynamic subfields of machine learning (e.g., data mining) such a kind of service would be helpful, too. This experience base is based on a number of research efforts: [Alt97, ABS96, AABM95] developed the classification criteria for CBR systems. [BSAM97, Mei96] conducted a survey about CBR systems and made these experiences reusable by means of CBR technology, i.e., each questionnaire has been represented as a structured case. As a part of [Nic98], this system has been made publicly accessible over the WWW and an evaluation program was developed in order to show the usefulness of the system from the viewpoint of its users. The rest of the paper is structured as follows. Section 2 describes how the experience factory as an organizational framework together with a process model for reuse and learning can support reuse and learning in a software business company and shows its relation to the basic CBR cycle (introduced by Aamodt and Plaza [AP94]) including the identification of points for tool support. Section 3 introduces a generic architecture for an experience factory tool using CBR technology that serves as a basis for the implementation of an experi-

Published in the Proceedings of the 11th German Workshop on Machine Learning

1 of 8

ence base. Section 4 describes the instantiation of the experience factory model and tool architecture for CBR-PEB. Section 5 summarizes the evaluation and its current status. Section 6 gives a summary and an overview of future work.

software business company development organization project organization development process

2 Support for Reuse and Learning in the Experience Factory

management

reuse process reuse

“A continuous build-up of knowledge requires an organizational structure” [ABT97]. In order to operationalize reuse and learning and provide tool support for it, a process model is required. In [BR91] such a process model for reuse-oriented software development is presented. In this section, the experience factory as an organizational framework for reuse and learning is integrated with the reuse-oriented software development model. The relation of the reuse-oriented software development model to the basic CBR cycle is made explicit. This shows how CBR is (and can be) used as a technology to support the reuse and learning processes. Furthermore the required features for an experience factory tool are identified. For such features where support through machine learning technology is useful. CBR-PEB is a concrete example that shows that our approach is applicable. Since there are some differences between CBR-PEB and the generic model for reuse and learning in the experience factory and, in addition, it is easier to understand on the generic level of description, it is worthwhile to know the generic model as well as its tailoring for CBR-PEB.

2.1 Generic Model The generic model is based on the experience factory paradigm and its comprehensive reuse mechanisms, also called reuse-oriented software development model [BR91]. The model focuses on the reuse process itself. Neither the project development with reuse is explicitly modeled nor the development for reuse. The generic model is presented in two parts. The organizational structure shows the relation between organizational unit – especially the information flow – and the number of instances for each process and each organizational unit in a typical software business company. The software development model, a slightly modified version of the reuse-oriented software development model [BR91], describes the process models concerning software development with reuse and experience base maintenance.

experience existing in the world at large

record

experience factory infuse experience base

information flow optional information flow influence

Figure 1: The organizational structure of a typical software business company.

2.1.1 Organizational Structure A typical software business company consists of a development organization, the EF organization, and the management (see Figure 1). The development organization consists of several project organizations. Each of them is developing or has developed the deliverables of a project. During the project development several reuse attempts1 (i.e., instances of the reuse process model) may take place. The experience factory organization aims at supporting reuse of life-cycle products and related experience (such as guidance for their development). Therefore, it creates and maintains the experience base and performs certain tasks involved in reuse and/or provides technology for the performance of these tasks. The management of the business company has an influence on both the development organization and the experience factory. Thus, the management can help introduce a reuse program by, e.g., rewards, incentives, financing an EF, or other forms of commitment.

2.1.2 Software Development Model To reflect our needs, the reuse-oriented software development model [BR91] is extended with the process 'create EB' and processes are associated with the organizational units (i.e., experience factory and project organization) – see Figure 2. Project organization's tasks. The project organization develops project deliverables. During a project 1. “Attempt” refers more to the “experimental” nature of the reuse process than the word “process” because reuse may fail.

Published in the Proceedings of the 11th German Workshop on Machine Learning

2 of 8

development process model (j) experience existing in the world at large

xk

create

process

Ai

modify

x k ; A i , …, A ik

modify

xk ; Ai

k

record project-specific experience

A ik

experience factory

m

experience base create

(re-)package (l)

Figure 2: The reuse-oriented software development model. requirements are specified for objects that shall be integrated. For each object a reuse attempt k may take place. A reuse attempt runs as follows: Given the project-specific reuse requirements xk for an object, a set of reuse candidates

A ik , …, A ik from 1

m

the experience base is identified. Their potential to satisfy the reuse requirements xk is evaluated, and the best-suited candidate

k1

k

1

m

m

A ik xk

Table 1: Input and output of the elementary reuse processes project organization

1

A i k , …, A i k

evaluate and select

reuse process model (k)

A ik

xk, EB

identify

evaluate and select

transfer into organizational ownership (“infuse”)

output

xk

identify

“world”

input

A ik is selected. Then either this

candidate is modified to the needs of the project, if required, or a new object is developed, if reuse is not considered to be appropriate. The resulting object is called xk. The reuse attempt may fail during identification, selection, modification, or integration. Failure means that the reuse process is aborted and then either, instead of reuse, the required object xk is created from scratch or a new attempt is started. Table 1 gives an overview of the input and output of the sub-processes of the reuse process. Experience factory's tasks. In the beginning the experience base must be created. This includes choosing or developing a tool, developing characterization schemes, and implementing the experience base. During operation a new object, if modification or creation took place during development, and/or experience gained in the reuse or creation (lessons learned,

etc.) may be recorded in the experience base. Recording includes checking whether the object and/or experience is worthwhile to store and, if so, storing the new experience in the experience base. Also experience from outside the organization (e.g., commercial off-the-shelf (COTS) software) may be infused into the experience base. To increase its reuse potential, experience can be (re-)packaged (i.e., tailored, generalized, or formalized) [BR91].1 Another way to repackage the experience is to change the characterization scheme or to rework the relations among artifacts (e.g., dependencies, variations, specializations). Repackaging also includes changing the tool used to store the experience, e.g., choosing a more advanced tool or improving retrieval mechanisms. Note that a more advanced tool usually requires a different representation, thus additional repackaging of the experience base with respect to characterization scheme and contents becomes necessary.

2.2 Using CBR in the Experience Factory The reuse-oriented software development model, which is part of the generic model, is quite similar to the CBR cycle, a common process model for case-based reasoning (see [AP94]). In fact CBR technology can be used to support reuse in the experience factory:2 • The case base can be seen as an experience base, and each case as an experience package. For example, the CBR system CBR-PEB includes such a case base. • The similarities between the processes are as follows (see Table 2 for an overview). The reuse requirements describe the problem. This problem description is used to retrieve one or more similar 1. “Packaging” refers to the first tailoring, generalizing, or formalizing of an artifact, “repackaging” to any additional tailoring, generalizing, or formalizing of the same artifact. 2. A more detailed view using the CBR task-method decomposition model, a refinement of the CBR cycle, is presented in [TA97].

Published in the Proceedings of the 11th German Workshop on Machine Learning

3 of 8

cases from the case base. Then the solution from one of the cases is reused and adapted, i.e., modified to fulfill the reuse requirements. The result is the required object, the solution to the problem. The application of the solution, which can be compared with integration, is not explicitly part of the CBR cycle. The revise and retain steps are similar to recording and packaging. CBR cycle

experience factory

support by a CBR tool

retrieve

identify, evaluate + select

querying

reuse

modify or create

retrieve experience packagea

revise + retain

record, (re-)package

create/update/ remove cases, update similarity and general knowledge

Table 2: How the CBR cycle matches the reuse-oriented software development model. a. The “experience package” is the actual reuse candidate, which is modified to fulfill the reuse requirements.

Thus, the minimum set of features of a CBR tool for an experience base are • querying the case base (similarity-based retrieval), • retrieval of the experience package that is characterized by the selected case, • adding a new case to the case base, • modifying an existing case in the case base, • removing an existing case from the case base. For each of these actions related to recording and packaging, criteria that trigger these actions should be developed. An experience base tool should support the maintenance by checking the criteria and/or triggering the actions.

3 A Generic Architecture for the Realization of an Experience Base The basic system architecture – as shown in Figure 3 – has been derived from today’s typical architecture for DB-oriented client-server applications [Hae95] consisting of three logical layers: presentation, application, and data storage layer. Here, the presentation is done by a WWW browser (e.g., Netscape Navigator). The application is represented by the EF server for querying the case base and on-line data collection. The data storage layer consists of the data bases and their

DBMSs1 (i.e., a CBR tool for the case base and an appropriate DBMS for the experience package DB). WWW browser

WWW browser HTTP

presentation layer application layer

EF server

data storage layer

CBR tool case base

ref

DBMS

DBMS

EP DB

measurement DB

Figure 3: Experience factory tool architecture. The EF server must be installed at a WWW server or include one. The components from the data storage layer can be installed at the same computer system or can be distributed over the network. The storing of the experience packages is managed by a separate DBMS. The CBR tool provides only an index for the experience package DB. Each case contains a reference to the experience package it describes (if an experience package is not classified then there will not be any case describing it). This implies that the EF server manages the distribution of the EB data across the case base and the experience package DB (EPDB). Of course, the same DBMS or a single DBMS may be used for measurement DB and experience package DB.

4 CBR-PEB – An Experience Base for CBR-System Development 4.1 Tailoring the Generic Model for CBR-PEB As already stated in section 2.2, a case base like CBR-PEB can be viewed as an experience base. Experience from CBR-PEB can be reused for CBR-system development. The (re-)users (i.e., CBR-system developers) can give feedback on CBR-system development to the CBR-PEB maintenance staff at the Fraunhofer IESE by filling out the form/questionnaire with information about their new system and keeping this information up-to-date. The maintenance staff acts as an experience factory that maintains the experience base CBR-PEB. Thus, the generic model as presented in Section 2.1 can be tailored to the needs of CBR-PEB. In the following the coincidences, if not obvious, and the differences concerning organizational structure and software development model are described. 1. DBMS = Data Base Management System

Published in the Proceedings of the 11th German Workshop on Machine Learning

4 of 8

4.1.1 Organizational Structure

• The only link between the CBR-PEB EF and the project organizations (i.e., the users of CBR-PEB) is the Internet.

The organizational structure in the context of CBR-PEB as shown in Figure 4 has been derived from the organizational structure in the generic model.

• There is no parent organization for all project organizations, who use CBR-PEB, like the development organization in the generic model.

software business company InterNet, etc.

• There is no organization like the management in the generic model that has an influence on both the experience factory and the project organizations.

project organization

• The reuse of CBR-PEB experience takes place only once at the beginning of the project (see development process description below).

CBR system development process

management

• Each software business company may also have its own experience factory as described in the generic model. So the CBR-PEB experience factory serves as an additional external source of experience that can be queried by the project organizations directly or by the company's EF depending on the company's organizational policies.

reuse process

experience existing in the world at large

EF reuse InterNet

record

Note that the difference between using CBR-PEB and infusing “experience existing in the world at large” is that there is a closed loop with feedback for CBR-PEB.

experience factory (Fraunhofer IESE)

experience base CBR-PEB

In the future, the generic model could also be extended with the concept of external experience factories. This is the typical role the Fraunhofer IESE plays when establishing an EF at a partner organization.

information flow optional inf. flow influence

Figure 4: The organizational structure with CBR-PEB.

4.1.2 Software Development Model The software development model is shown in Figure 5. This model has been derived from the generic model and is tailored to the needs of CBR-PEB, i.e., CBR-system development. The model also coincides with the CBR-PEB usage scenario as described by the creator of CBR-PEB [Mei96]. In the following the model will be described.

The differences with respect to the generic model are as follows: • CBR-PEB is an experience base outside the company. • Not only one but several companies can access CBR-PEB.

development process model criteria (domain, task, technical, ergonomic)

required CBR system

develop CBR system

lessons learned, etc.

without reuse

contribute new case (fill out questionnaire)

develop CBR system with reuse

identify knowledge

existing CBR system select

new CBR system infos

reuse process model

project organization experience factory (IESE) CBR system package create CBR-PEB

CBR appl package

CBR tool package

record (check+store)

(re-)package

Figure 5: The software development model for CBR systems with CBR-PEB.

Published in the Proceedings of the 11th German Workshop on Machine Learning

5 of 8

Project organization's tasks. A project organization (“user” of CBR-PEB) develops a CBR system as follows: First the project organization collects the decision support criteria considering the CBR system the organization wants to develop. The criteria are divided into domain, task, ergonomic, and technical criteria (see [Mei96] and [ABS96] for further information or have a look at the questionnaire in the WWW1). In order to receive decision support the user retrieves the most similar cases from the CBR-PEB case base according to the collected criteria. CBR-PEB lists the ten most similar cases together with their similarity rating.2 The user can browse through these cases to carry out further subjective investigations. Based on these evaluations3 he decides whether he wants to reuse an existing system or knowledge, and, if so, which system (application or tool) he wants to use as a starting point for his development efforts or which knowledge from the development of existing systems he wants to reuse. After making the decision he fetches or buys the existing system or knowledge (i.e., the existing solution). Then he develops his new system reusing the existing system or knowledge. Afterwards he answers the WWW questionnaire for CBR system development in order to provide a new case for CBR-PEB.

TCP/IP socket data storage layer

CBR-Works CBR-PEB case base

UNIX file system contact address

CBR-system developer

measurement data file

Figure 6: Data storage layer of CBR-PEB. At the moment, the case base contains 45 cases. An overview of the content of the case base is given in [BSAM97].

4.2 Tailoring the Generic Architecture for CBR-PEB

A CBR system developer who does not reuse existing work, but is willing to share his experience with others, may also answer the WWW questionnaire.

The generic architecture was simplified for CBR-PEB at two points in the data storage layer (see Figure 6): 1. CBR-PEB does not contain any experience packages, i.e., a CBR system. It only consists of the case base as an index and a contact address in each case that models the reference. 2. The measurement data base is implemented as a simple file using the system-library file-access functions. CBR-Works from TECINNO GmbH4 (Kaiserslautern, Germany) is used as CBR tool. CBR-Works is queried by the EF server over a TCP/IP socket using the CQL/Casuel interface [MBC+94].

Note that there is only one single reuse process considering CBR-PEB per CBR system development.

4.3 The Features of CBR-PEB

When a case should be updated (e.g., when a CBR system is improved), the developer provides the information for the update of the case to CBR-PEB (this has not been explicitly modeled in Figure 5). Experience factory's tasks. The case base of CBR-PEB [Mei96] contains experience packages that describe CBR systems (i.e., software products). There are two subtypes of CBR system packages: CBR application and CBR tool packages. Before storing a new CBR system package in CBR-PEB's case base the new information must be checked to see if it is valuable and complete or just nonsense input from a surfer. Further inquiring, if necessary, and checking and storing is done by the Fraunhofer IESE. This is the same for update information regarding existing cases. 1. http://www.iese.fhg.de/Competences/QPE/QE/CBR-PEB.html 2. The system uses local type-specific and global similarity measures and weighting. For more in-depth information see [Wes95]. 3. automatic evaluation by CBR-PEB and/or personal and subjective evaluation by the project organization

The system implements all features listed in Section 2.2, Table 2. The WWW interface of CBR-PEB5 especially supports the querying, creating, and updating of cases. As already stated, in the reuse step the system can only provide a contact address as a reference but not the system that shall be reused. The features dealing with learning (i.e., recording and packaging) are especially worthy of note: • Contribution of new cases: The new cases are reviewed by the EF staff in order to filter nonsense input from net surfers. • Update of existing cases: The case contributors can update their cases on-line. The authorization uses a login name and a password. The changes are written to a log file to allow the EF staff to undo an unauthorized or invalid repackaging. This updating by the case contributor is regarded as a recording of update infos and repackaging of the existing case (this is not explicitly modeled in 4. http://www.tecinno.de/ 5. http://www.iese.fhg.de/Competences/QPE/QE/CBR-PEB.html

Published in the Proceedings of the 11th German Workshop on Machine Learning

6 of 8

Figure 5). We propose that making on-line updating possible will also result in cases that are more up-to-date because the contributors can easily update the information. So, the whole system will be more up-to-date than, e.g., a book can ever be. This is fundamental because up-to-date information was regarded as very important in the evaluation program (see Section 5). • Removal of cases: Cases can be removed by the EF staff on request or if the evaluation program shows that a case is considered “not useful” by a certain percentage of users.

5 Evaluating an Experience Factory/Base 5.1 Choosing the Evaluation Focus Basili, Caldiera, and Rombach propose that software development must be seen and treated as a business. As already stated, an experience factory is established to improve this business by supporting and improving reuse [BCR94b]. Since the obvious ultimate goal of a business is success – especially economic success –, an evaluation of an experience factory and its experience base must focus on its success. Then, such an evaluation can be used to justify the establishment of an experience factory.

5.2 Choosing the Evaluation Technique For the evaluation a standard technique of the Fraunhofer IESE and the AG Software Engineering of the University of Kaiserslautern for goal-oriented measurement and evaluation – the Goal/Question/Metric (GQM) approach [BCR94b, GHW95, BDR96, HR97] – is used. GQM is an innovative technology for goal-oriented software engineering measurement. GQM helps defining and implementing operational and measurable software improvement goals. It has been successfully applied in several companies, such as NASA-SEL, Bosch, Digital, and Schlumberger [CEM96]. In GQM programs, the analysis task of measurement is specified precisely and explicitly by detailed measurement goals, called GQM goals. Relevant measures are derived in a top-down fashion based on the goals via a set of questions and quality/resource models. This refinement is precisely documented in a GQM plan, providing an explicit rationale for the selection of the underlying measures. The data collected is interpreted in a bottom-up fashion considering the limitations and assumptions underlying each measure. To create an effective measurement plan, a process model is needed to assign the measures to elementary

processes and products. The software development model for CBR-PEB (see Section 4.1.2) is such a process model.

5.3 Practical Evaluation Domain experts were interviewed in order to make their implicit knowledge about success criteria of CBR systems explicit, which led to detailed measurement criteria for CBR-PEB (from goals to questions to measures). The measurement criteria focus on the utility of the information supplied by CBR-PEB and the user friendliness of its WWW interface. Analyses of the measures showed that the success is closely related to the usability of an experience base as defined by [RIG93] and the effectiveness of an information retrieval system as defined by [Coo97]. The utility measures put an emphasis on the quality of the retrieved information and also helped identifying the important attributes of the characterization scheme and their proposed interrelationships, which can be regarded as general knowledge about the domain that is being validated by the GQM process. The interpretation of the results of the measurement program takes place in well-structured, organized meetings, the so-called feedback sessions.

5.4 Preliminary Results In order to make a first validation of the measurement program, usage trials with the experts were conducted. These usage trials not only supplied measurement data for the first feedback session, but also showed how different users actually interact with the system, and gave hints for improvements of the system. In the first feedback session, the system and the measurement program received a very positive rating by the experts. They also stated that the system at its current state is already useful (that was not expected by the experts in the beginning) and, therefore, should be installed as soon as possible. The first iteration of the GQM process for CBR-PEB also showed that the GQM approach is useful for evaluation of an experience base and led to a meaningful result.

6 Summary and Future Work A comprehensive model for the experience factory consisting of an organizational structure and a software development model was presented. The application of CBR technology in this context was shown. A generic architecture for an experience factory tool using CBR technology was described. The experience factory model and the architecture were successfully instantiated for CBR-PEB - an experience base for CBR-system development. CBR-PEB

Published in the Proceedings of the 11th German Workshop on Machine Learning

7 of 8

is now available over the World-Wide-Web and ready for your contribution of new cases. 1 First experience was gained regarding the use of the GQM approach for the practical evaluation of an experience base by its successful application to the evaluation of the experience base CBR-PEB. Data on real use is being collected. When an appropriate amount of data has been collected, another feedback session with this real data and an iteration of the GQM process will be carried out. We are also planning to develop a comprehensive evaluation technique for experience bases and test and validate this with our experience bases.

7 References [AABM95] K.-D. Althoff, E. Auriol, R. Barletta, and M. Manago. A Review of Industrial Case-Based Reasoning Tools. AI Intelligence. Oxford (UK), 1995. [ABS96] Klaus-Dieter Althoff and Brigitte Bartsch-Spörl. Decision support for case-based applications. Wirtschaftsinformatik, 38(1):8–16, February 1996. [ABT97] K.-D. Althoff, A. Birk, and C. Tautz. The experience factory approach: Realizing learning from experience in software development organizations. In Proceedings of the Tenth German Workshop on Machine Learning, University of Karlsruhe, 1997. [Alt97] Klaus-Dieter Althoff. Evaluating case-based reasoning systems: The Inreca case study. Postdoctoral thesis (Habilitationsschrift), University of Kaiserslautern, 1997. [AP94] Agnar Aamodt and Enric Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AICom - Artificial Intelligence Communications, 7(1):39–59, March 1994. [AW97] Klaus-Dieter Althoff and Wolfgang Wilke. Potential uses of case-based reasoning in experienced based construction of software systems and business process support. In R. Bergmann and W. Wilke, editors, Proceedings of the Fifth German Workshop on Case-Based Reasoning, LSA-97-01E, pages 31–38. Centre for Learning Systems and Applications, University of Kaiserslautern, March 1997. [BA98] R. Bergmann and K.-D. Althoff. Methodology for building case-based reasoning applications. In M. Lenz, B. Bartsch-Spörl, H. D. Burkhard, and S. Wess, editors, Case-Based Reasoning Technology - From Foundations to Applications, pages 299–328. Springer Verlag, 1998. [BCR94a] Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. Experience Factory. In John J. Marciniak, editor, Encyclopedia of Software Engineering, volume 1, pages 469–476. John Wiley & Sons, 1994. [BCR94b] Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. Goal Question Metric Paradigm. In John J. Marciniak, editor, Encyclopedia of Software Engineering, volume 1, pages 528–532. John Wiley & Sons, 1994. [BDR96] Lionel C. Briand, Christiane M. Differding, and H. Dieter Rombach. Practical guidelines for measurement-based process improvement. Software Process, 2(4):253–280, December 1996. [BR91] Victor R. Basili and H. Dieter Rombach. Support for comprehensive reuse. IEEE Software Engineering Journal, 6(5):303–316, September 1991. [BSAM97] Brigitte Bartsch-Spörl, Klaus-Dieter Althoff, and 1. http://www.iese.fhg.de/Competences/QPE/QE/CBR-PEB.html

Alexandre Meissonnier. Learning from and reasoning about case-based reasoning systems. In Proceedings of the Fourth German Conference on Knowledge-Based Systems (XPS97), March 1997. [CEM96] The CEMP Consortium. Customized establishment of measurement programs. Final report, ESSI Project Nr. 10358, Germany, 1996. [Coo97] William S. Cooper. On selecting a measure of retrieval effectiveness. In K.S. Jones and P. Willet, editors, Readings in Information Retrieval, pages 191–204. Morgan Kaufmann Publishers, 1997. [ELS97] Robert Engels, Guido Lindner, and Rudi Studer. A guided tour through the data mining jungle. In Proceedings of the Tenth German Workshop on Machine Learning, August 1997. [ELS98] Robert Engels, Guido Lindner, and Rudi Studer. Providing user support for developing knowledge discovery applications: A midterm report. Künstliche Intelligenz, 1:40–45, 1998. [GFW94] Martin L. Griss, John Favaro, and Paul Walton. Managerial and organizational issues - starting and running a software reuse program. In W. Schäfer, R. Prieto-Diaz, and M. Matsumoto, editors, Software Reusability, chapter 3, pages 51–78. Ellis Horwood Ltd., 1994. [GHW95] Christiane Gresse, Barbara Hoisl, and Jürgen Wüst. A process model for GQM-based measurement. Technical Report STTI-95-04-E, Software Technologie Transfer Initiative Kaiserslautern, Fachbereich Informatik, Universität Kaiserslautern, D-67653 Kaiserslautern, 1995. [Hae95] Theo Haerder. Skript zur Vorlesung Datenbanksysteme. AG Datenbanksysteme, Universität Kaiserslautern, 1995. [HR97] Susanne Hartkopf and Günther Ruhe. Goal-oriented learning in experimental software engineering by using rough sets. In Proceedings of the Tenth German Workshop on Machine Learning, August 1997. [Icm98] The Methodology of Applying Machine Learning. AAAI/ICML 1998 Workshop, Madison (WI), July 1998. [MBC+94] M. Manago, R. Bergmann, N. Conruyt, R. Traphöner, J. Pasley, J. LeRenard, F. Maurer, S. Wess, K.-D. Althoff, and S. Dumont. Casuel: A common case representation language. Technical Report Deliverable D1, Version 2.01, Esprit Project Inreca (P6322), 1994. [Mei96] Alexandre Meissonnier. A case-based information system for case-based applications and systems based on INRECA (in German). Diploma thesis, AI/Expert System Group, University of Kaiserslautern, October 1996. [Nic98] Markus Nick. Implementation and Evaluation of an Experience Base. Diploma thesis, Fraunhofer IESE, University of Kaiserslautern, 1998. [RIG93] RIG Glossary. Technical Report RTR-0001, Reuse Library Interoperability Group (RIG), April 1993. [TA97] Carsten Tautz and Klaus-Dieter Althoff. Using case-based reasoning for reusing software knowledge. In D. Leake and E. Plaza, editors, Proceedings of the Second International Conference on Case-Based Reasoning. Springer Verlag, 1997. [Wes95] S. Wess. Case-Based Reasoning in Knowledge-Based Systems for Decision Support and Diagnostics (in German). PhD thesis, University of Kaiserslautern, 1995; also: infix Verlag, Sankt Augustin.

Published in the Proceedings of the 11th German Workshop on Machine Learning

8 of 8