Towards the Application of Case Based Reasoning to ... - CiteSeerX

Towards the Application of Case Based Reasoning to Decision-Making in Concurrent Product Development (Concurrent Engineering). B. U. Haque, R. A. Belecheanu, R. J. Barson, K. S. Pawar School of Mechanical, Materials, Manufacturing Engineering and Management, University of Nottingham, Nottingham, United Kingdom. Abstract This paper describes the development and application of Case Based Reasoning (CBR) to provide decision support for project managers and engineers during the early phases of New Product Development (NPD) in a Concurrent Engineering (CE) environment. The paper discusses the reasons for using CBR, focussing on issues such as case collection, maintenance, terminology, adaptation, and similarity; and how the final system could contribute towards achieving a CE conducive culture. The main issues in using CBR in a CE environment, that is characterised by ill defined and ill structured information during early phases of product development, are textual consistency of terminology, validity of case similarity, and the difficulty in automating case evaluation and adaptation. Additionally the paper concludes that using technology like CBR, which can be costly to develop and implement, requires the company to train considerably their managers and team members to document their experiences and knowledge in a manner with which the system can work with and team members can understand. There needs to a commitment to maintain and improve the knowledge base- a ‘knowledge friendly’ culture hence needs to be instilled for CBR tools to succeed.

1 1.1

Problem Description Background (Concurrent Engineering)

New Product Development (NPD), is an interdisciplinary activity requiring contributions from nearly all the functions of a firm, whether it is an upgrade/improvement of an existing product or a new concept either to the company or the market. Traditionally NPD has been viewed as an organisational activity, which was the result of various functional activities performed in stages from

concept development to product delivery. The sequential operation of these functional stages resulted in long development times and many quality problems due to the lack of communication and understanding of the different product design, manufacturing and above all customer requirements. To avoid these problems Concurrent Engineering (CE) is being used by many companies and has resulted in companies making new products better and faster [1, 2, 3, 4]. CE or CNPD is characterised by the early involvement of the different functional disciplines and parallelism of hitherto sequential activities (i.e., bringing downstream activities forward). Decisions made in the early (design) phases of product development are hence often based on incomplete, ill-structured, and poor quality information. This is why decisions are sometimes made in an empirical manner, using only personal knowledge and experience, gained during past problem solving processes. It is widely cited that most mangers/designers refer back to previous solutions to related problems as a first step in the design process [5].

1.2

The Problem (End User Needs)

When engineers or managers call upon past experience or the “experts” opinion, the information or knowledge is prone to bias of the particular experiences of that individual (or the so-called “expert”). The wider collective company (corporate) knowledge is not always readily available in a structured and consistent format. A detailed review of project management procedures and processes in new product development at selected companies [6] identified that there exists a case history of past projects contained in disparate ‘data’ or ‘information’ sources such as project files, databases and most importantly individuals memory. This information was not only restricted to major decisions concerning the continuation of the project, but also included data about specific products or components, design decisions that have worked well in the past and those that have not worked so well, problem solving approaches etc. This data was usually difficult to access, especially where individual knowledge or memory was concerned, when making decisions in new projects. At the same time it was not always possible to find the most experienced or most knowledgeable personnel. Hence, there was the risk that a problem or difficulty that was found in an earlier project, and subsequently resolved, could be repeated in a new project. There was a lack of structured support, not only in formal management reviews (decisions), but also in many decisions made by team members outside the formal reviews with respect to the detail design and development. These decisions could be equally critical to the success of the project. The basic requirement of the industrial partners was that the past experience was presented in a constructive way at the time of making the decision, and also indicate the relevance of the data for that particular decision. The decision support data or knowledge should also be examined by other team members to assess from their viewpoint the acceptability of the decision. The requirement for the system was hence to build a knowledge base in which complete ‘decision cases’ or scenarios could be entered and then recalled or reused when similar problems arose again.

Another issue was that design is typically carried out in an iterative manner in terms of generating the initial design and then testing. Product or component design, in mechanical engineering for instance, is evaluated under numerous interrelated criteria such as machinability, quality, reliability, structural integrity, assembly, maintenance and so on. This process is referred to as Design For x (DFx), with x as one of the criteria or constraints. Time delays and costs can be incurred if such evaluations result in red-design or take too long. Though the CE philosophy attempts to bring DFx issues to the attention of the designer as soon as possible, the process would benefit greatly if support could be provided through a what-if study based on past experiences. Design changes or changes to specification arise from other sources too such as marketing, reacting to sudden changes in market needs or issues relating to industrial design; or purchasing, identifying supplier capability limitations, etc. Quite often engineers are not aware of the consequences of the changes. It would be quite useful if the consequences of such changes or similar changes could be identified or known in advance or prior to elaborate testing, simulations or waiting for the actual event to happen! The above problems or issues called for a knowledge based decision support system, providing the managers and engineers (in design and development) with structured, consistent, comprehensive and accurate information and knowledge. This would enable the early phases of NPD and hence CE to be more productive. Additionally the success of CE depends upon collaboration between the different functional expertise to arrive at a mutually agreeable decision. The decision support should encourage this by providing viewpoints from different experiences of different people. The development of the required system has been carried out in an EU funded project called CODESCO- A Practical Communication and Decision Support Environment for Managing Concurrent Product Development (ESPRIT project no. 25455) [7]. The overall objective of this research project was to develop and validate a communication as well as a decision support system for helping project managers and design/development engineers in their decision-making activities within a CE environment.

2 2.1

Application Description The Solution- Choice of CBR

The explicit request for reuse of knowledge and experience called for the application of Case Based Reasoning (CBR). CBR is a computer technique, which combines the knowledge based support philosophy with a simulation of human reasoning when past experience is used, i.e. mentally searching for similar situations happened in the past and reusing the experience gained in those situations [8]. In the same way, in CBR, the knowledge cases are structured and stored in a database, which the user queries when trying to solve a problem. The system retrieves a set of similar cases

and then evaluates the similarity between each case in the database and the query. The most similar case(s) are presented to the user as possible scenarios for the problem at hand. The user has to decide if the solution retrieved is applicable to the problem, i.e. the system doesn’t make the decision, it only supports the decision making process. If it cannot be reused, the solution is adapted (manually or automatically). When the user finds a solution, and its validity has been determined, it is retained with the problem as a new case in the database (the case is “learned”), for future reuse. The theoretical CBR cycle is, therefore, a retrieve-evaluate-adapt-learn process. However, a CBR system may very well implement only the retrieval stage of the process and does not need to implement the other stages. The retrieval stage is the basic stage and the expression of the concept of reuse of experience. There were a number of reasons for choosing CBR over conventional rule based systems. The main requirement for the system was that it should be able to support a variety of product and business domains. Additionally the system was intended to deal with a fairly wide range of technical and managerial problems. Traditional rulebased knowledge approaches were not found suitable for this requirement, as they required strong domain knowledge and representation, whereas decision problems in CE are generally difficult to define and structure. In CBR, as opposed to rule-based approaches, knowledge about the domain is acquired and maintained through unrelated but similar cases and does not need a domain expert or knowledge about the problem domain. The generic concept of our system hence moves away from a rule-based implementation, as for each type of product and company context, specific rule-sets would need to be encoded. Another requirement was that the system should be able to trace the effect of decisions made in upstream process on the down stream processes, i.e., perform a What-if Analysis. CBR enables this to be performed in a better way using the same searching mechanism and case structure as it would for a normal CBR query. This enables identification of different effects or consequences of change for different contexts and conditions without having the need to build a complex set of rules for different contexts and conditions. Additionally, a more detailed analysis can be performed, through sequential search, and one can carry on the search if the retrieved consequences are not acceptable after one iteration.

2.2

Collection of Decision Cases and Development of the Case Structure

One of the main development issues was the definition of the case structure, as its acceptance to both the end user and the system developer was paramount to the continuation of the project. In the CODECSO project two manufacturing organisations are taking part in the development, implementation and validation of the CBR system. The two industrial partners are:

1.

2.

Thomson CSF Service Industrie (TSI). TSI are located in France and produce high technology electrical and electronic hardware, and information technology systems (software), for defence and commercial applications. Their products are quite often ‘one of a kind’. General Domestic Appliances Ltd. (GDA). GDA, are a subsidiary company of a joint venture between industrial giants GE (USA) and GEC (UK). GDA Ltd., are a consumer goods company, producing domestic appliances in the UK.

Analysis of decision making literature [9], CBR literature [10] and mapping of real decision making activity during new product development at the industrial users lead to the development of a generic decision making process. Essentially four phases in decision making were identified; (1) (2) (3) (4)

Framing or problem understanding; Gathering intelligence and developing alternatives or solutions; Coming to conclusions and selecting a solution; and Learning from feedback.

These phases were used to define the structure of decision cases to be collected and entered into a knowledge base, for re-use during new product development activities. YES

Identification of the problem (issue)

Is the context known ?

NO

Examination of the context

Assessment of the problem

Fram ing or Problem Understanding

Creation/development of the solutions (alternatives)

Assessment of the solutions- advantages/ disadvantages

Learning from Case Base

NO

Are solutions valid alternatives? CO DESCO Case Case

Gathering Intelligence / Generating Alternatives / Developing solutions

Enter Decsion Case into Case Base

Y ES

Outcome identification Learning from feedback or experience

Im plementation of selected solution

Selection of solution using selection criteria Coming to conclusions / Solution Selection

Figure 1 The Generic Decision Making Process

In Figure 1 a model of the generic decision process is presented. Further, the different phases of the decision process are named. For the actual case structure to be used in the CBR application this process was simplified into three phases: • Problem Description • Solution Development • Outcome The Problem Description section, besides a textual description of the problem, incorporates details about the context in which the problem arose: product, project and development phase information, people involved or responsible for the decisionmaking, and decision parameters. The Solution Development section aims to record how the solution was found, what the available alternatives were, why one solution was preferred to the others. It was also found useful to record the Outcome of implementing that solution, in terms of performance, and positive or negative consequences. In this way, retrieving cases was also useful for identifying and analysing ‘what-if’ scenarios related to particular problems. Having defined the case structure further interviews, using the predefined case structures, were carried out to collect decision cases in both companies. Collection and analysis of decision cases lead to the identification of different types of decisions made in new product development. Essentially two main types were identified, managerial/strategic decisions or technical/design decisions. These types were further subdivided. Under managerial/strategic decisions the main groupings were: (1) Determining the risks of the project; (2) Cost Estimations; (3) Team building decisions; (4) Determining strategic indicators for the project; and (5) Choosing between strategic alternatives e.g., make versus buy decisions. Under technical decisions the main groupings were: (1) Choosing between different design alternatives or technical alternatives relating to production etc.; and (2) Identifying technical risks. The differences between the operating (trading) and cultural environments of the two companies meant that at a detailed level the case structure had to be customised to reflect the differences. For instance, in TSI the product is fairly complex (e.g. rugged computers, radars, and avionics display systems), the business is very much geared towards satisfying individual client needs and the contracts quite often include long-term maintenance and life-cycle support. So, ‘client information’ is included only in TSI cases. Additionally because of their business environment, the focus of TSI decisions were on risk analysis, cost estimation, determining strategic indicators for the project, and choosing between strategic alternatives. At GDA, with the products made for a mass market the focus was more on marketing, quality and technical issues, like choosing between different design or technical alternatives, and identifying technical risks.

PRO BLEM SOL VING ISSUES

CASE STRUCTURE

DECISIO N MAKIN G ISSUES

PR OBLEM • Problem description • Context • Constraints

Problem/Situation Description

SOLUTION DEV ELOPM ENT AN D SOLU TION SELECTION

Objectives

Alternatives

• Possible solutions:

Solution

- solution

Criteria

- advantages

Outcome

- disadvantages

Selected solution

• Selection criteria • Implemented solution

Consequences

• Method of assessment

OUTC OM E • Consequences • Comments

Figure 2 Details of the Case Structure

Also, the general decision parameters, referred in the cases as ‘contractual or company constraints’, have been found to be different, i.e. decision-makers consider different aspects in making a decision. Cost to the client, delivery time, technical performance and technology are specifically akin to TSI, whereas quality and safety standards, company costs, marketing (such as aesthetics) and production issues (such as tooling) are the major decision constraints in GDA. Nevertheless, the general process of making decisions was the same, independent of function, expertise, or type of company, and this is reflected in the common high level structure of the case. Figure 2 shows the main attributes or fields, common to GDA and TSI. At this level of detail, cases from both companies can be mapped onto this structure. Differences, as mentioned above, appear in the sub fields of fields like ‘context’ and ‘constraints’. For example ‘Context’ contains sub fields like ‘product details’, ‘project details’, which obviously differ from company to company. The fields and hence the case structure (and its specialisation for GDA and TSI) developed can be used for a wide range of companies similar in structure, product or NPD process characteristics. The NPD processes and management structure found in TSI and GDA are common to many companies of similar business or product type. This is mainly due to that fact that most companies’ quality assurance procedures are based on ISO9000 standards, and generic best practice procedures.

2.3 2.3.1

System Architecture and CBR Tool Selection Architecture

Figure 3 illustrates the architecture of the Decision Support System within the wider CODESCO system. At User Interface Level, Java has been used for developing the ‘forms’ for entering cases, case querying and query results. Distinct interfaces have been created for both GDA and TSI. At Decision Support Level, The Easy Reasoner (TER) CBR tool from Haley Enterprise, Inc. has been used. Using an existing CBR tool was appropriate for our project considering the time and effort available. Decision Support M odule User Interface Level

CBR D isplay

CBR m odule (C application)

The Easy R esoner (TER ) (C libraries)

dBase D atabase of Cases

D ecision Support Level

Figure 3 Architecture of the Decision Support System

2.3.2

Selection of CBR Tool:

A review of the CBR tool market identfied sixteen (16) suitable tools. An initial screening process using ten criteria (with constraints), see Table 1, reduced this list down to four potential contenders that could satisfy most of our criteria/constraints. Criteria Operating system platform Customisability of API Ability to represent cases collected to date Costs Reasoning speed Number of cases it can handle The reasoning mechanism Training support for developers Post development release/version support Hear say/comments

Constraints Windows NT and UNIX Java or C++ Less then $3.000 USD A no limit situation is desired Training should be provided Lower price for future s/w releases Degree of confidence in source

Table 1 CBR Tool Selection Criteria and Constraints

The Easy Reasoner

Kate

The Haley Enterprise Inc

Aknosoft

Ability to represent CODESCO structure could cases implemented

Score Case Cannot 5

C++ implementation Reasoning Dimension & speed increases the speed

Customization

Database support

Platforms Price

Total Scored

ReCall

TecInno GmbH

Isoft

Score Score Score Case CASUEL syntax for case Full case representation 0 3 3

represent structure for CODESCO structure. Kate cannot support adaptation

be

CBR-Works

representation

providing complex object hierarchy

3

2 seconds to retrieve a case out of 10.000 case-base on a PC

5

Retrieval in less than 1 second, out of approximately 1000 cases

3

Information Not Available

0

C, C++ API

5

Also Available as DLL allowing integration in user applications

5

Requires knowledge of SMALLTALK programming

3

5

Supports ODBC databases

3

Open to databases (doesn’t specify which ones) for import/export capabilities

5

DB2, Oracle, Sybase, GemStone, ODBC compatible RDBMS

5

Libraries C/C++ calls for embedding your ReCall application into others applications Connection to RDBMS via ODBC driver

5

Win 3.1,95,NT, soon Unix

3

5

Win 3.1,95,NT,Unix

5

4

13875 ECU Commercial License; 2313 ECU Academic License

1

Win 3.1,95,NT,Unix, OS/2, Mac, Solaris 2313 ECU

5

Information Not Available

0

access to and SQL

Win 3.1,95,NT,Unix RETE++ 1850 ECU TER 4000 ECU

25

19

Table 2 Comparison of Four CBR Tools

24

5

18

The four tools were evaluated against each other using six of the ten selection criteria described above, but this time applying a more rigorous scoring mechanism, see Table 2. TER was identified as the most suitable tool for our application. The tool comes as a set of Eclipse and C libraries, which implement the indexing and retrieval mechanism. The Searching Engine is based on Eclipse libraries (Clips-like language) but TER also provides the layer of C libraries upon these Eclipse libraries. Unlike the other tools researched, it is not a ‘ready-to-use’ software tool, and therefore requires more application programming effort. Summarising, the following advantages of TER were the basis for its selection: • The tool can be embedded in a C++ application and therefore the input/output interface which our system requires can be built (enabling the customisability feature) • The system works with an external database; the current version works only with the ‘dBase 4’ database format, but any other format can be converted to this one. Further releases will include ODBC functionality. • The tool allows for text pattern search, which is a critical need for our system as the cases contain almost only textual information. Despite the fact that the case structure described earlier was done in a hierarchical manner, decomposition of fields into sub fields (or sections) can be seen only at the interface level of the application. The representation used to store the cases in the database is a flat representation (as opposed to hierarchical). Two databases with different structures, have been created, one for TSI and one for GDA, each database consisting of one table. The database format was dBase 4, as required by the CBR tool. The fields in the database have been represented as text fields, hence, text similarity formulas have been used in the implementation of the CBR module, to calculate the similarity between cases. The underlying reason is the difficulty to further structure some of the fields, like Problem Description, Solution, Constraints.

2.4

Functionality (including What if Study)

The following functions will be provided by the CBR system: • Insert, modify, delete, and browse through cases in the database • The ability to customise the input or user interface according to the case structure required by the user company • CBR-like query of the database- to search for a particular problem or solution. • What-if analysis- i.e., what would happen if a particular decision is taken. The what if analysis is perhaps the most useful functionality as far as the end users are concerned. Below is a description of how we intend to support this.

The ‘what-if study’ function: To capture design or specification change scenarios the information and consequently knowledge needs to be presented in a way, which would enable retrieval. Therefore, in such scenarios, problems can be described in terms of a ‘change’ (in specification or design) and a ‘consequence’ (or the ‘outcome’). ST ART

Change in design or other parameter Query database for effects of change Results with + ve consequences

implement change

with -ve consequences

YES

Acceptable? NO

Search for solution in database

Can’t implement change

NO

Solution found? YES

ST OP

Can implement change under new conditions

NO

Is solution a design change?

Figure 4 What if Study

The CBR system can retrieve the consequence information by either searching (querying) the problem description field (which would contain the change and the outcome or consequence information) or the solution implemented and outcome fields, if the change was itself a solution to another problem. The flow chart in Figure 4 summarises the ‘what-if’ procedure. The starting point is of course a change in design, or other specification or issue such as increased costs, faulty component, manufacturing problem etc. The results of the query would reveal either positive or negative consequences, described either in the problem description field or the outcome field. If the consequences are positive then the change is implementable, otherwise a solution to the problem or change needs to be found. The results of the second search (for a solution) could reveal that no solution is available, or a solution is available under new conditions, or that a further design change is required, hence another what-if loop. The CBR system hence identifies

scenarios that have happened in the past, in similar contexts, and for similar design changes. Through a sequential querying of the case base, the user can see and analyse the dynamics of the design process, retracing upstream changes and finding downstream consequences. The scenario can continue depending on the availability of cases and on how positive the outcome is (negative consequences can be new problems to be investigated). This functionality would aid quite considerably the achievement of a more concurrent product development environment.

3

Application Building

3.1

The Development Process- overview

A CBR implementation, in any domain, requires a detailed analysis of the environment, as it is strongly related to the type of problems being solved or decisions being supported. For this reason, in the CODESCO project, the academic partners started by looking at the specific product development context of each company. User requirements were developed into system functional requirements through interviews, questionnaires and focussed workshops with project managers, design/development engineers, production, and quality engineers. Requirements Definition

Case structure definition and case collection

GDA Refinement

User Requirements System Requirements

TSI

Case Structure Development

Industrial environment

Case collection through interviews

Literature Research

More case collection

Testing in the user companies and getting user feedback

CBR system implementation

Case base development

CBR tool selection

Refinement

Identification of CBR tool requirements, CBR system requirements

Implementation and pilot projects

Figure 5 Development Phases

The system was developed based on industrial requirements formulated by the companies participating as partners in CODESCO. Figure 5 above illustrates the main phases of development. The diagram shows three phases Requirements definition: This involved collection of user requirements and establishment of the system requirements.

Case structure definition and case collection: The application of CBR requires a structured representation of knowledge in the form of cases. Implementation: This involved selecting the commercial CBR tool to be used for development of the application, and then customisation of the tool to meet both common and specific end user requirements. Below we shall describe in detail the development of the CBR system.

3.2

CBR System Development Tasks

The main tasks of the development phase are: 1. 2. 3. 4.

To collect cases and build the database of cases To implement the TER application using the TER built-in functions (TER Search Engine) for indexing and retrieval. To implement the input/output interface To implement the information retrieval for the ‘what-if’ analysis.

3.2.1

Database Development

In order to implement the first level of the development tasks, i.e., the database of cases, over 40 cases from TSI and GDA were collected and entered in a dBase 4 database. Both cases from TSI and GDA were entered using the same case template customised to meet individual organisation needs, in separate files. 3.2.2

TER Application Development

The CBR Module (see Figure 3) is a C application, in which the query data provided by the user through the CBR Display module is processed and the results consist of the ‘n’ most similar cases (‘n’ is user defined), or cases with similarity less than a user specified threshold. The implementation of the retrieval mechanism is provided with The Easy Reasoner, in the form of C libraries and consists of two main phases, a pre-query processing phase and a query-processing phase. In the first phase, an index containing statistical information is created for all the records in the database, before queries are made. In the query processing stage, information contained in the index is used to determine the case(s) most similar to the query, using a ‘Nearest Neighbour’ algorithm. The similarity between two cases is calculated pairwise, between pairs of fields. The fields in the database have been represented as free text fields; hence the similarity formulae for fields are textual retrieval specific. During calculation, each text field is considered a list of terms (words) and the information in the field is normalised, so that each field contributes evenly to the global distance of two cases. Therefore, a weight is determined for each term in the text field of the case (record), and a weight is determined for each term in the text field of the query. These weights make part of the index. The weight of the k-th term in the record i, is

Fik Log(nk/N) Wik = √Σj(Fij Log(nj/N))² , where N is the number of records, nk is the number of different records in which term k occurs, and Fik is the number of occurrences of term k in record i divided by the total number of terms in record i. Using these weights, a normalised distance between two fields is calculated according to the formula:

Si =

Σk WikWk √Σk (Wik)² Σk (Wk) ² ,

where Wk - the weight of the k-th term in the query. The similarities are calculated only for the fields with information. Therefore the results depend on how detailed the query as well the cases are. 3.2.3

User Interface Development

The user input interface or CBR Display module is a set of windows (entry forms) implemented in Java, where a query can be created by entering data in the corresponding fields (see Figure 6). A query contains data related to Problem, Context and Constraints. The results are presented as a list of similar cases, ordered by similarity, in a separate window, where details about the Solutions and how they have been developed can be viewed (see Figure 7). The CBR Display module interfaces with the CBR module through a temporary file containing the query. 3.2.4

‘What-If’ analysis implementation task

The knowledge structure was not only suitable for design problem solving task, but also for the analysis of the ‘what-if’ scenarios. As problems are described in terms of ‘cause – effects’, cases can be retrieved in such a way, that consequences of a design change can be identified, as well as the cause(s) for a particular problem.

Figure 6 Some Screen Shots from a CBR Query

Figure 7 Results of a CBR Query

3.3

The project team, costs and time scales

The project team, costs and time scales for each of the three phases identified above are described below (Table 3): Development Phase Requirements Definition

Case Structure Definition, Case Collection and Analysis Implementation -Development (excl. Pilot Projects) TOTAL (OVERALL)

Team

Duration (months) 6

Effort in Man Months 8.3

Total Development Costs (Euro) 89K

Lead by the industrial partners. Representatives from the two industrial end users (project managers), 2 academic institutes, and 2 software houses/consultancies Lead by one academic institute with support from the other academic institute, and the industrial end user representative/s. Lead by the software house, supported by the academic partners. Essentially selecting, customising, and integrating the commercial tool.

6

18.4

78K

12

19.8

174K

15 (due to overlappin g of tasks)

46.5

341K

Table 3 Project Team, Times Scales, Manpower, and Costs (in Euro)

The table shows that the pre-implementation tasks, i.e., requirements and case structure definition and collection, account for roughly half of the total costs.

3.4

Installation and End User Training

A well-defined training procedure was required in order to ensure that the expected results were obtained when using the application, at the two test sites. Firstly, the system developers installed the software with the assistance of a computer administrator from the company. A Computer Based Training package was developed using Macromedia (a multimedia authoring package) and individual sets of usage guidelines for GDA and TSI were written. The developers carried out the training sessions, and the user group was made of a Project Manager, a Technical Manager and a design team member. A general presentation of the software was made at the beginning, followed by the CBT and the specific guidelines. At the end, a user was asked to identify a current or recent technical or managerial problem and use the system to find a solution. In the second stage, the user participants in the first training session were asked to perform training for other possible users in the company, using the CBT.

3.5

Current Status of Development

At the present a prototype has been implemented, i.e. a program able to retrieve cases to match with the ‘Problem Situation/Description’. The results consist of a list of similar cases, in a decreasing order of similarity. The program is now currently being tested at the industrial end user sites, on different queries and with different input parameters for the similarity function. The implementation will be finalised after the prototype is thoroughly tested in the companies. Several issues will be considered during the testing phase: • the performance of the system in terms of speed and accuracy of the results; this entails looking also at the performance of the CBR tool, in terms of textual retrieval capabilities and any possible limitations of the database capacity • the user feedback, regarding: - the relevance of the information retrieved (“how suitable is the solution retrieved for the current problem ?”) - the relevance of the fields in the case structure - the ease-of use of the system, in terms of the entry forms layout and level of detail. • ‘how to use the software’, in terms of relationship between the detail of data entered in the query and the accuracy of the results. Additionally during refinement, customisability of the interface and the way of defining the case structure will be approached.

3.6

Issues Emerging

The development has raised a number of important issues for people interested in using case based reasoning for supporting decision making in NPD. These are discussed below. 3.6.1

Case Collection and Maintenance

Case collection and the development of an appropriate structure has been an important issue for the success of CBR implementation and hence the decision support module. Training the end users on how to collect and enter cases is important. Additionally, the type of cases collected is very important. Cases that have a short shelf life in terms of validity of knowledge (i.e., obsolete knowledge within a year or earlier) will not be included in the case base to guarantee a high relevance for industrial use of the cases. The case base has to be maintained and updated regularly. Once the user makes a decision, based on the information provided by the CBR system, the new solution has to be implemented (tested) in order to find the outcome. This solution and its outcome will be “retained” together with the problem (query) as a new case in the database.

3.6.2

Case Terminology

To enforce consistency of data across a case base, the terminology used in describing the cases has to be specific for that company. Users must use the same terminology for the same concepts (e.g. the NPD phases are usually formalised). A lack of consistent terminology could lead to problems with case matching for the case similarity function- the most relevant cases could be missed due to text based similarity calculations. In order to help this, a taxonomy (list of values for a field) is suggested by the system, where possible. For instance the ‘Program Main Phase’ field in TSI cases can take the following values: Design, Production or Maintenance. However, these values are a “guideline” for the user and not restriction, the user being allowed to enter a new value. 3.6.3

Case Similarity

The similarity calculation algorithm implemented by The Easy Reasoner does not consider vocabulary; this raises terminology problems, which we have tried to avoid by restricting the terminology used in some fields (see section 3.6.2), and offering a considerable level of detail in the case structure as a whole. However, terminology problems might still occur, due to the difficulty to further structure some of the fields, like Problem Description, Constraints, considering the scope of decision support required by the end user and the general nature of the application. In these fields, the risk is that the same information described with different words, (i.e., different people might describe the same problem in the different ways), is disregarded by the retrieval algorithm. This affects the results in the sense that the relevance of the case might not be the same as perceived by the user, i.e. the case can be retrieved with a lower similarity percentage. To increase the probability of retrieving the most similar case in the results of a query, our system uses the more textually restrictive fields, within the case structure, containing information about the context of the problem (e.g. Program main phase, Client operating environment, etc). However, due to the technical complexity of the algorithm used in The Easy Reasoner implementation, the similarity percentage offered for each case in the results might not reflect the ‘real’ similarity of the cases, as understood by the user. The software does not show enough transparency and can affect user’s trust. 3.6.4

Case Adaptation

The case evaluation and adaptation functions were not developed in our application. There were a number of reasons for this, as discussed below. In order to perform an evaluation and adaptation of the case(s) retrieved, either a rule-based production system should be used, or the need for adaptation should be decreased. In CE, decision-making problems are ill defined and ill structured, which makes it difficult to formalise the knowledge and build the rule-based model. A rulebased model has to be context specific, hence this would make the product less generic and thus considerably increase the cost of customisability. Currently,

customisability is solved through case representation, designed to fit or adapt to any company context. Case adaptation would require implementing a separate rule-based model for every company context, with additional support from knowledge engineers, which means very high development and maintenance cost for the CBR system. Decreasing the need for adaptation by improving the retrieval capabilities would require the developers (us) to build their own retrieval mechanism, designed for their problem. But since they are using a commercial CBR tool, which has its own retrieval mechanism, this approach for adaptation was not appropriate. So, without the automated adaptation function our system requires additional human reasoning, i.e. increased participation of the user in evaluating the solution and deciding if it can be reused. The problem of reduced system performance or functionality can be overcome through extensive case collection (the more cases in the database, the better results) and via more effort in refinement of the system. Like any decision support tool, a CBR system is limited to ‘suggest’ a solution and not to assert that ‘this is the solution to the problem’. Hence, even an adapted solution would have to be filtered by the human reasoning before being applied. As practice has proved, in some cases the results of an adapted solution would not be justified to a sceptical user. These results of adaptation must be, therefore, validated, like in any other decision support system. The best validation is through a practical implementation of the solution provided by the adapted case. The risk of failure of this implementation is not decreased by adaptation [11] and the cost of failure does not justify the practical validation, especially in costly commercial projects. Thus, with adaptation, the degree of user involvement in the actual reasoning decreases. The human adaptation implies more participation and greater effort from the user, whereas the automatic adaptation improves the relevance of the results and minimises the user effort. The final decision belongs, however, to the user. Recognising that practical retrieval technologies are available, but the general adaptation problem remains extremely difficult for CBR systems, experts in both CBR research [12] and applications [11, 13, 14] agree that the best use of CBR is as advisory systems that rely on the user to perform evaluation and adaptation. Many CBR systems simply dispense with adaptation, replacing the retrieve-evaluateadapt-learn cycle with retrieve and propose cycle [12], i.e. the best case retrieved is proposed to the user, who will evaluate and adapt the solution.

4

Application Benefits

The reason for applying a CBR decision support system was to improve the CE process or practices by improving the decision-making process. The success of CE lies in the achievement of collaboration between different team members. We have developed a decision support tool that provides factual information in a structured and context relevant format, and which encourages human intervention and

discourse. This would enable the team to move towards a more CE conducive culture. This is achieved by empowering the project managers and team members by removing the reliance on the so called hard to get experts and providing the hard to get corporate knowledge hidden in company archives in an immediate, structured, coherent and reader friendly manner. The other benefit is in improved design capability in terms of enabling engineers to evaluate different design options under different constraints and interrelated criteria, i.e., Design for X (DFx) type of studies. The highly structured nature of the cases combined with the what-if analysis enables engineers and managers to investigate different scenarios. Whilst considerable efforts can be put into modelling and capturing the expertise to carry out the automated DFx evaluation of design, these evaluations can be computationally expensive. In this respect they would fail to provide the benefit of timely feedback to the designer which could be realised through the use of cases [15]. Using CBR could result in considerable cost and time savings in the NPD process. However the success of CBR, which can itself be costly to develop and implement depending on the scale of application and tool used, requires considerable training of the managers and team members towards documenting and sharing their experiences and hence knowledge in a manner which others can use. Additionally there needs to a commitment to maintain and improve the knowledge base. An ‘information sharing and knowledge friendly’ culture hence needs to be instilled for such decision support tools to succeed.

5 [1]

References

J. Riedel and K. S. Pawar. The Strategic Choice of Simultaneous Versus Sequential Engineering for the Introduction of New Products. International Journal of Technology Management, Special Issue on Manufacturing Strategy, 1991. [2] L. Trygg. Simultaneous Engineering: A Movement or an Activity of the Few? International Product Development Management Conference on New Approaches to Development and Engineering, Brussels, 18-19 May, 1992. [3] S. G. Shina. Successful Implementation of Concurrent Engineering Products and Processes. Van Nostrand Reinhold, New York, 1994. [4] R. W. Hanssen. Reducing Delivery Times in Engineer-To-Order Firms by Using the th Concepts of Concurrent Engineering. Proceedings of the 4 International Conference on Concurrent Enterprising (ICE’97), The University of Nottingham, October 8-10, pp. 495- 508, 1997. [5] V. Walsh et al. Winning by Design. Blackwell Publishers, 1992. [6] F. Weber et al. User Requirements Definition. CODESCO Deliverable D11, ESPRIT Project No. 25455, 1998. [7] CODESCO Project Programme. ESPRIT Project No. 25455, 22 Sept., 1997. [8] D. B. Leake. CBR in Context: The Present and Future. In D. Leake’s Case-Based Reasoning– Experiences, Lessons, & Future Directions. AAAI Press / MIT Press, 1996. [9] S. Benett. Guide to Management and Technology. Web: http://www10.geocities/SiliconValley/Pines/1814essay501.htm, 1996. [10] J. Kolodner. Case-Based Reasoning. Morgan Kaufmann Publishers, San Mateo, CA, 1993.

[11] W. Mark, E. Simoudis, and D. Hinkle. Case-Based Reasoning: Expectations and Results. In D. Leake’s Case-Based Reasoning – Experiences, Lessons, & Future Directions. AAAI Press / MIT Press. 1996. [12] J. L. Kolodner. Improving Human Decision-Making through Case-Based Decision Aiding. AI Magazine 12(2), pp. 52-68, 1991. [13] R. Barletta. A Hybrid Indexing and Retrieval strategy for Advisory CBR Systems Built with REMIND. In proceedings of the Second European Workshop on Case-Based Reasoning (EWCBR), pp. 49-58, Chantilly, France, 1994. [14] H. Kitano and H. Shimazu. The Experience Sharing Architecture: A Case study in corporate-Wide Case-Based Software Quality Control. In D. Leake’s “Case-Based Reasoning – Experiences, Lessons, & Future Directions. AAAI Press / MIT Press, 1996. [15] Macdonald R. Transforming heuristics into cases: an evolutionary approach to the construction of multi-criteria decision support systems. Published on the web page of the Multi-media Communications Group, Department of Psychology, University of Glasgow, Glasgow, Scotland, 1998. (http://www.mcg.gla.ac.uk/staff/rory/wec2.html)