evaluating case-based reasoning systems:the inreca ...

EVALUATING CASE-BASED REASONING SYSTEMS: THE INRECA CASE STUDY

KLAUS-DIETER ALTHOFF

HABILITATIONSSCHRIFT (Accepted by the Computer Science Department of the University of Kaiserslautern, Germany)

JULY 9, 1997

EXECUTIVE SUMMARY We compared the INRECA case-based reasoning (CBR) system (Esprit project 6322) with five industrial CBR tools, namely CBR EXPRESS (Inference, USA), ESTEEM (Esteem Software, USA), KATE tools (AcknoSoft, France), REMIND (Cognitive Systems, USA) and S3-CASE (tecInno, Germany), and twenty CBR-related research prototype systems (developed at the University of Trondheim, University of Texas at Austin, Carnegie-Mellon University, Massachusetts Institute of Technology, Yale University, IIIA Barcelona, University of Würzburg, University of Technology at Aachen, University of Technology at Berlin, GMD Sankt Augustin et al., and the University of Kaiserslautern), according to a set of systematically chosen evaluation criteria called decision support criteria. These criteria include technical criteria dealing with the limitations and abilities of the systems, ergonomic criteria concerning the consultation of the executable system and application development, application domain criteria dealing with concept structure, knowledge sources, and knowledge base characteristics as well as application task criteria like integration of reasoning strategies, decomposition methods, and task properties. Our evaluation builds upon the evaluation of the above mentioned commercial CBR tools (which has been carried out in 1994 and published in 1995 at AI Intelligence, Oxford, UK) by using the same data, applying the same criteria and carrying out the same experiments with the INRECA system. The original goal of the evaluation task in INRECA was to achieve deeper „insights“ for the final integrated INRECA system (e.g., the identification of four different levels of integration between induction and CBR, and here especially the seamless integration where INRECA starts application development with the full flexibility of CBR and then learns from ongoing experience to improve its performance that evolves smoothly towards the performance needed for consulting a decision tree), to come up with a reasonable and realistic combination of application issues (i.e., implementing a tool that is usable for reallife applications) and research issues (i.e., integrating state-of-the-art methods into the tool), to have a more concrete estimation of what can be expected from CBR technology in the near future (application of CBR technology to analytic tasks like classification, diagnosis, and decision support that involve large sets of case data – or case data that are often to be maintained – and fuzzy matching) and what in the far future (application of CBR technology to synthetic kinds of problems like planning, configuration, and design that involve much case adaptation, handling of changing domains using CBR-related methods, and – as a „side-effect“ – decreasing costs in application development by including to a high degree CBRrelated methods for learning from experience), and to get a guideline how to evaluate CBR tools (viewing a tool evaluation as a systematic organisation and presentation of information on the tools – not just a ranking –, using a mixture of qualitative criteria and formally defined experiments if running systems are available, combining technical, ergonomic, domain, and application task criteria, carrying out case studies, explaining and publishing the criteria: see above). One important result of the evaluation task in the INRECA project is the development of the basis where to build a framework for the evaluation of CBR technology upon (e.g., describing CBR systems and domains and application tasks using a set of decision support criteria, analysing the underlying methods and domain/task characteristics by relating domain/task criteria with technical/ergonomic criteria with the additional use of a task-method decomposition hierarchy for CBR (Aamodt & Plaza, 1994), and developing abstracted algorithms based on the extracted methods and knowledge modelling techniques underlying the analysed CBR-related systems). The central idea is to use a unifying set of decision support criteria that applies for the four main tasks in the INRECA project: the evaluation of CBR technology, the development of a decision support tool, the integration of technologies as well as the development of applications.

i

Another important result of INRECA is the seamless integration of CBR and induction that has been directly motivated by our evaluation task, namely the evaluation of commercial CBR tools (the evaluated tools either did not use integrated methods or allowed only very restricted combinations of different methods) as well as the comparison of knowledge-poor but efficient approaches to CBR (PATDEX system) with knowledge-intensive ones (CREEK system). The seamless integration of INRECA combines the advantages of both: integrating general knowledge to increase the system's performance (and also its transparency). Further advantages of INRECA, which have been brought to light, are the identification of the four different integration levels for integrating different methods (the toolbox, the co-operative, the workbench, and the seamless level), the inclusion of various kinds of general knowledge to support case retrieval and adaptation (a-kind-of relations, is-part-of relations, hard and soft constraints, completion rules, adaptation rules), and the object-oriented case and knowledge representation (CASUEL). INRECA is well-focused for analytic kinds of tasks, a wide variety of applications – ranging from troubleshooting technical devices and help-desks to forestry management and credit assessment – is covered by the approach, applications are easy to implement, and INRECA is especially well-suited for large, externally stored case-bases. Thus, INRECA and its commercial successors – KATE tools and CBR-Works – contribute to the current state of CBR technology for analytic problems. CBR as a field – and thus INRECA – has been influenced by fields like cognitive science (cognitive model of human problem solving behaviour), knowledge-based systems (integration of case-specific and general knowledge), machine learning (learning of generalised knowledge from experience), databases (storing/retrieving case data), information retrieval (knowledge-based retrieval of different kinds of information), neural networks (learning from case data) as well as pattern recognition, uncertainty and statistics (global/local similarity measures, nearest neighbour search). We contrasted INRECA with six different approaches, namely an integrated learning system for technical diagnosis that includes a CBR sub-system as well as a model-based and an inductive reasoning subsystem, an associative rule-based expert system shell for diagnostic reasoning that had been combined with a CBR system, a service support system based on associative rule-based diagnosis and hypermedia techniques, a deep integration of a service support system with the INRECA system itself, a model-based diagnosis system for case adaptation in the area of technical diagnosis, and a neural network based system for diagnostic reasoning using Kohonen nets. A number of applications carried out in the scope of INRECA have been used to evaluate INRECA with respect to practical results from application development: help desk for robot diagnosis (Sepro Robotique), troubleshooting aircraft engines (Cfm International), assessing wind risk factors for Irish forests (Coillte), and reuse of object-oriented software (Daimler-Benz). We also evaluated the INRECA system with respect to related work.

ii

PREFACE Document History The work described in this thesis has been carried out within the scope of the INRECA Esprit project, a cooperation between AcknoSoft (Paris), IMS (Dublin), tecInno (Kaiserslautern), and the University of Kaiserslautern. The evaluation of the technologies underlying INRECA was one major effort within the project. There have been a number of INRECA documents which dealt with evaluation and which influenced this thesis: Traphöner, Manago et al. (1992) defined an initial set of so-called „industrial criteria“ for evaluating the technologies underlying INRECA. This set has been extended, changed, refined, and corrected for the comparison of industrial CBR tools, as first described in Althoff, Auriol et al. (1994) and later published in Althoff, Auriol et al. (1995a). In parallel, a set of criteria for the evaluation of domains and application tasks as well as CBR-related research prototype systems has been developed (Aamodt & Althoff, 1993; Althoff, 1994a; Althoff, 1995a; Weis, 1995a; Weis, 1995b; Althoff & Bartsch-Spörl, 1995; Althoff & Bartsch-Spörl, 1996; Althoff & Aamodt, 1996a; Althoff & Aamodt, 1996b; Althoff & Weis, 1996). In Althoff, Wess et al. (1995) all these criteria have been combined and a complete evaluation of the INRECA system has been presented. Thus, it can be seen as a pre-version of this thesis that has been available to a restricted number of people to achieve some feedback.

Acknowledgement I would like to thank a number of persons who supported me in various ways and, by this, contributed to the development of this document: I would like to thank Michael M. Richter, the leader of our research group and the main supervisor of my thesis, for his fruitful support, his huge number of interesting ideas, and his competent comments over more than 10 years. He „initialised“ the co-operative atmosphere in our group and by this created the basis for real teamwork. This helped all of us very much and was the reason behind many results achieved in our group so far. He defined the right research goals at a very early stage, which allowed all group members to focus on the relevant topics within their respective research tasks. Prof. Richter already stated the idea of evaluating CBR systems according to a set of evaluation criteria which he had defined in early 1993. These criteria have been an input for Althoff, Auriol et al. (1994). He also developed many views on CBR as such, but especially in relation to other fields, which was very helpful for me to keep my work on the right track. However, for me the most valuable thing was that – during our cooperation – I learned what science in general and computer science in particular is about. iii

Agnar Aamodt supported my work in many ways. Our cooperation started in the early 90's and was intensified by Agnar's visit in Kaiserslautern during summer 1993 and the preparatory work of EWCBR-93. We had several intensive discussions on the topic of domain and application task characteristics and their role in CBR systems analysis and development. The work described in this document has been very much influenced by such discussions and several used methods or applied views have their roots in Agnar's scientific work. In addition, Agnar's constructive criticism during all the INRECA meetings we had together has always blown a fresh wind into the work of the INRECA team in general and into my work in particular. Therefore, I very much appreciate that Agnar accepted to be a reviewer of this thesis. I enjoyed the time in Kaiserslautern where I had the opportunity to work together with Stefan Wess. He has always been a good friend of mine and his helpful openness was often the decisive hint to correct things etc. I benefited much from Stefan's work on CBR and also of his practical talents of organising things and being in communication with other people and groups. Stefan gave me the opportunity to already study his doctoral dissertation at a very early stage in summer 1994 which was of great help for me while preparing the evaluation of the final INRECA system. Stefan contributed to my work with his competent and practical view on CBR both during the evaluation of the commercial CBR tools and – still more important – during our daily work by keeping things running. The cooperation with Michel Manago cannot be adequately described with words, and even if new technical possibilities like world-wide-web and others are considered, it cannot be documented in an appropriate way: Michel means (and is) real life. I very much enjoyed the cooperation with him. It was never monotonous. Michel not only had the starting idea for the INRECA project, he was INRECA's „main engine“ and, by this, contributed to a very high extent to its success. To do the evaluation of the technologies underlying INRECA on a very broad, but still detailed and reproducible basis was Michel's idea. He also had the idea to publish the evaluation of commercial CBR tools and managed all the necessary organisational things. He also had the idea to give tutorials on CBR which very much helped to broaden CBR technology and especially helped me to rethink and document much of the work done in our group in Kaiserslautern, which turned out to be of high value in many aspects. Karl-Heinz Weis jun. contributed very much to this document. He focused on the evaluation topic in both his project thesis as well as in his diploma thesis. The results he achieved are reflected in the chapters 5 and 6. I benefited much from the intensive discussions we had together. Karl-Heinz also contributed to section 1.3 (alternative diagnostic approaches) and very much to chapter 8 on related work. Wolfgang Wilke invested much effort into the task of putting the chosen evaluation criteria into practical use. He accepted to evaluate two commercial CBR tools (KATE tools and S3-CASE) and freed me from many organisational tasks both during the evaluation of the tools and during the development of this document. Harald Holz, Alexandre Meissonnier, and Carsten Priebisch evaluated the other commercial CBR tools. Harald's analysis of REMIND, the tool that contained the most capabilities, was done iv

very carefully and his description offered many helpful examples. CBR Express (CBR 2) was evaluated by Carsten Priebisch. He developed the S3+/INRECA system within his diploma thesis which was a helpful project that allowed to study different integration options and validation possibilities. Carsten also contributed to the chapter on related work, especially to the part on induction. Alexandre evaluated ESTEEM and contributed to section 1.3 and to the chapter on related work, here especially to the parts on model-based diagnosis and integration. Ralph Traphöner initialised the set of „industrial criteria“ that has been a valuable starting point for our evaluation task. He contributed many creative ideas (and realisations) to the INRECA project and helped me very much by offering the opportunity to work for tecInno for a certain time. I benefited also much from his competence in administrative tasks as well as his knowledge in practical computer science issues. Eric Auriol contributed to the revision and extension of the industrial criteria set and brought the evaluation strategy of Gaschnig, Klahr et al. (1983) to light. He initialised the structuring of chapter 4 and contributed the first two sections, which reflect Eric's and Michel's experiences on these applications. Eric also contributed to parts of chapter 7 (integration issue in INRECA). Ralph Bergmann contributed the results of the study on reuse of object-oriented software for Daimler-Benz, for which he was the responsible person within our group. Ralph also contributed to parts of chapter 7. I benefited much from his various activities within the INRECA project, his always helpful support, as well as his scientific competence. Brigitte Bartsch-Spörl commented on various aspects of our evaluation work and helped me to define the right focus from a practical point of view. This has especially influenced parts of chapters 2 and 9. Her well-organised and competent activities within the CBR community helped us very much to put things forward and were one important cornerstone for the successful development of CBR research and application in Germany as well as in Europe. Roy Johnston and Sean Breen contributed the results of the Coillte application in chapter 4. Michel Baudin helped me in improving the structuring of this document, especially in developing the executive summary. I would like to thank Martin Bräuer, Cesar Carranza, Stefan Dittrich, Christoph Globig, Wolfgang Lenski, Frank Maurer, Hector Muños-A., Jürgen Paulokat, Gerd Pews, Jürgen Rahmel and all other colleagues of our research group for the various helpful discussions and the patience and insight during the time when the main parts of this document had to be developed. Thanks go especially to Edith Hüttel and Willi Klein for their helpful support over a long period of time. I also appreciated very much the contributions made by David Aha, Hans-Dieter Burkhard, Padraig Cunningham, Peter Jaenecke, Brice Lepape, Rick Magaldi, Robin Muir, Mario Lenz, Enric Plaza, Gerhard Strube, and Ansgar Woltering during a number of discussions at a number meetings and workshops. I very much want to thank Monika Althoff: We share not only our love, our lives, and our three children Tim, Robin, and Maren, but also our belief in Jesus Christ. This helped us very much,

v

especially when time was short and work was much and/or the concrete future of our family was unclear. Funding for this work has been provided within the INRECA project („Induction and Reasoning from Cases“) by the Commission of the European Union (Esprit Contract 6322; May 27, 1992, until September 27, 1995) to which the author is greatly indebted. The partners of INRECA are AcknoSoft S.A. (Paris, co-ordinating partner), tecInno GmbH (Kaiserslautern), Interactive Multimedia Systems Ltd. (Dublin), and the University of Kaiserslautern.

vi

TABLE OF CONTENTS Executive Summary .........................................................................................................................i Preface........................................................................................................................................... iii Document History ............................................................................................................. iii Acknowledgement............................................................................................................. iii Table of Contents ..........................................................................................................................vii Chapter 1: Introduction ...................................................................................................................1 1.1 Overview ....................................................................................................................1 1.2 Positioning CBR ..........................................................................................................1 1.2.1. CBR and Cognitive Science.........................................................................2 1.2.2. CBR and Knowledge-Based Systems..........................................................3 1.2.3. CBR and Machine Learning ........................................................................4 1.2.4. CBR and Information Retrieval...................................................................7 1.2.5. CBR and Statistics/Pattern Recognition ......................................................8 1.2.6. CBR and Neural Networks ..........................................................................9 1.2.7. CBR and Uncertainty.................................................................................10 1.3 Contrasting CBR with other Approaches to Diagnostic Problem Solving ...............10 1.3.1. MOLTKE Workbench for Fault Diagnosis in CNC Machine Tools...........12 1.3.1.1. Workbench: Modelling Knowledge for Diagnostic Tasks .........13 1.3.1.2. MOLTKE Shell: The Basic Diagnostic Problem Solver ..............14 1.3.1.3. iMAKE: Building the Basic Knowledge Base.............................17 1.3.1.4. GenRule: Learning of Strategy Knowledge................................18 1.3.1.5. PATDEX: Case-Based Diagnosis .................................................19 1.3.1.6. Conclusion ..................................................................................21 1.3.2. MOLTKE/PATDEX: Associative Rule-Based Diagnosis and CBR .............22 1.3.3. Characterising the Task of Fault Diagnosis of CNC Machine Tools 23 vii

1.3.4. S3: Service Support System......................................................................24 1.3.5. S3+/INRECA: Integrating S3 with INRECA................................................26 1.3.6. MoCAS/2: Integrating CBR with Model-Based Diagnosis .......................27 1.3.6.1. Model Representation .................................................................28 1.3.6.2. Abstraction and Similarity ..........................................................28 1.3.6.3. Case Retrieval and Adaptation ...................................................29 1.3.6.3.1. Justification of a Hypothesis .......................................30 1.3.6.4. Discussion ...................................................................................31 1.3.6.4.1. Comparison with Model-Based Diagnosis ..................31 1.3.6.4.2. Comparison with CBR .................................................31 1.3.6.4.3. Conclusion ...................................................................32 1.3.7. KoDIAG: Diagnosis with Spatial Neural Maps.........................................33 1.3.7.1. The Structure of Explicit Knowledge .........................................33 1.3.7.2. The Implicit Knowledge Modelling ...........................................34 n ×n 1.3.7.2.1. The Euclidean Distance Measure in ℜ ...................34

1.3.7.2.2. The Neighbourhood Relation Function .......................34 1.3.7.2.3. The Kohonen Training Learning Function..................34 1.3.7.2.4. Relevance.....................................................................35 1.3.7.2.5. Local Similarity ...........................................................35 1.3.7.2.5.1.

Local Similarity for Continuous Values ............35

1.3.7.2.5.2.

Local Similarity for Symbolic Values ...............36

1.3.7.3. The Problem Solving and Learning Process...............................36 1.3.7.4. Conclusions.................................................................................37 1.4 Summary...................................................................................................................37 Chapter 2: Scientific Grounding of the Evaluation Task ..............................................................38 2.1 Evaluation of CBR Systems ......................................................................................38 2.2 Decision Support Criteria for CBR System Development ........................................39 2.3 How Evaluation Guided the INRECA Project ...........................................................41

viii

2.3.1. Purpose......................................................................................................41 2.3.2. Influence on the INRECA Project ..............................................................42 2.3.3. Development of the Evaluation Criteria Set .............................................42 2.4 Evaluation Criteria for CBR Systems........................................................................44 2.4.1. The Ergonomic Criteria ............................................................................45 2.4.2. The Technical Criteria ..............................................................................46 2.4.3. The Integration of the User Domain .........................................................49 2.4.4. The Knowledge base Characteristics ........................................................50 2.4.4.1. The Explicit Knowledge Sources ...............................................50 2.4.4.2. The Concept Structure ................................................................51 2.5 Summary...................................................................................................................52 Chapter 3: Comparing Inreca with Industrial Cbr Tools ..............................................................53 3.1 The Evaluation Framework ......................................................................................53 3.1.1. Presentation of the Evaluation Criteria.....................................................54 3.1.2. Presentation of the Test Domains .............................................................58 3.1.2.1. CAR Domain ...............................................................................58 3.1.2.2. CNC Machine Tool Diagnosis.....................................................59 3.1.2.3. Marine Sponges ..........................................................................59 3.1.2.4. Travel Agency.............................................................................59 3.1.2.5. Summary of Test Domains .........................................................60 3.2 Evaluation Based on Technical Criteria ...................................................................60 3.2.1. Case and Knowledge Representation .......................................................60 3.2.1.1. Describing Cases.........................................................................61 3.2.1.1.1. Representation of Structured Domains........................61 3.2.1.1.2. General Expressiveness of the Tools...........................61 3.2.1.1.3. Other Specific Features ...............................................62 3.2.1.2. Organisation of the Case Library................................................63 3.2.1.3. Assessing Similarity ...................................................................65 3.2.1.4. Reusing, Revising, and Retaining Cases ....................................67 ix

3.2.1.5. Extracting, Representing, and Using General Knowledge 68 3.2.2. Executable System ....................................................................................69 3.2.2.1. Ability to Handle Noisy Data During Consultation....................69 3.2.2.2. Ability to Handle Incomplete Data During Consultation ...........70 3.2.2.3. Flexibility of the Executable System ..........................................71 3.2.2.4. Performance of the Executable System ......................................72 3.2.2.5. Correctness of the Executable System........................................76 3.2.2.6. Completeness of the Executable System ....................................78 3.2.2.7. Consistency of the Executable System .......................................79 3.2.2.8. Effectiveness of the Executable System .....................................80 3.2.3. Development System ................................................................................80 3.2.3.1. Ability to Handle Noisy Data During Application Development..............................................................................................80 3.2.3.2. Ability to Handle Incomplete Data During Application Development..............................................................................................81 3.2.3.3. Performance of the Development System...................................82 3.2.3.4. Consistency of the Development System ...................................84 3.2.3.5. Effectiveness of the Development System .................................84 3.2.3.6. Adaptability of the Development System ...................................85 3.3 Evaluation Based on Ergonomic Criteria .................................................................86 3.3.1. Application Development .........................................................................86 3.3.1.1. Control of Application Development..........................................86 3.3.1.2. Validating and Testing the Executable System ..........................87 3.3.1.3. Acquiring and Maintaining Knowledge and Data ......................87 3.3.2. Explainability and Modelling Support......................................................88 3.3.3. User Acceptance .......................................................................................88 3.3.3.1. User Interface..............................................................................89 3.3.4. Organisational Impact of the Technology ................................................89 3.4 Synthesis of the Evaluation ......................................................................................90 x

3.5 Summary...................................................................................................................91 Chapter 4: Feedback from Industrial Applications in the Scope of Inreca ...................................93 4.1 SEPRO .......................................................................................................................93 4.1.1. Context of the Application........................................................................93 4.1.2. Evaluation .................................................................................................94 4.1.2.1. Consultation System ...................................................................94 4.1.2.2. Application Development ...........................................................96 4.1.2.3. Ergonomic Criteria......................................................................96 4.1.3. Benefits .....................................................................................................98 4.2 CFM ...........................................................................................................................99 4.2.1. Context of the Application........................................................................99 4.2.2. Evaluation ...............................................................................................100 4.2.2.1. Consultation System .................................................................100 4.2.2.2. Application Development .........................................................101 4.2.2.3. Ergonomic Criteria....................................................................101 4.3 COILLTE ..................................................................................................................104 4.3.1. Context of the Application ........................................................................104 4.3.2. Availability of Existing Data.....................................................................104 4.3.3. Case Representation ..................................................................................105 4.3.4. Case Library ..............................................................................................106 4.3.5. Similarity Assessment ...............................................................................106 4.3.6. Revision and Retention..............................................................................107 4.3.7. User Acceptance and Ergonomics .............................................................107 4.3.8. Customisation of the Interfaces; Data Maintenance..................................107 4.4 DAIMLER-BENZ ......................................................................................................108 4.4.1. Context of the Application ........................................................................108 4.4.2. Evaluation..................................................................................................109 4.4.2.1. Consultation System .................................................................109 4.4.2.2. Ergonomic Criteria....................................................................110 xi

4.4.2.3. Benefits .....................................................................................110 4.5 Summary.................................................................................................................110 Chapter 5: Comparing Inreca with Cbr-Related Research Prototype Systems...........................111 5.1 The Explicit Memory Base – Examples for Knowledge Characteristics ...............111 5.1.1. Episodic Memory Packets (CYRUS, CASEY)..........................................111 5.1.1.1. Episodes by k-d Trees (INRECA, CcC+) ...................................112 5.1.2. The Category Model for Knowledge Representation .............................113 5.1.2.1. Categories by Similarity (PATDEX/2, CcC+)............................114 5.1.2.2. Symptom Categories (MOLTKE, CREEK) .................................115 5.1.3. Sequential Retrieve with Cluster Trees (CcC+) .....................................116 5.1.4. The Context-Graph (MOLTKE, S3+).......................................................117 5.1.5. Integrating Symbolic Models (MoCAS) .................................................118 5.1.5.1. The q-Model (REMIND)............................................................121 5.1.6. Semantic Networks or Integrated Knowledge Representation (PROTOS, CREEK) ................................................................................................123 5.1.7. Neural Filters ..........................................................................................125 5.1.7.1. Neural Test Selection (MOLTKE)..............................................125 5.1.8. Modification Knowledge ........................................................................126 5.1.9. Conclusions to Explicit Knowledge Representations.............................127 5.2 The Implicit Knowledge Modelling .......................................................................131 5.2.1. Similarity and Distances .........................................................................131 5.2.1.1. Knowledge-Poor and Knowledge-Intensive Distances ............132 5.2.1.2. Local Similarity ........................................................................132 5.2.1.3. Standard Global Similarity Measure.........................................132 5.2.1.4. Measure for Detecting Splitting Attributes in k-d Trees (INRECA) ..................................................................................................133 5.2.1.5. Distances for Spatial Filtering (WheelNet) ..............................134 5.2.1.6. Feature-Set Similarity (PATDEX, CcC+) ..................................135 5.2.1.7. Similarity Derived from Causal Models (CASEY)....................136

xii

5.2.1.8. Similarity of Explanations (CREEK, PROTOS) .........................137 5.2.1.9. Similarity of Methods ...............................................................139 5.2.2. Relevance Modelling ..............................................................................139 5.2.2.1. Determination Factors (PATDEX/1, MOLTKE, CREEK) ............141 5.2.2.2. Neural Relevance Modelling (PATDEX/2, CcC+).....................141 5.2.2.3. Relevance Modelling with Combined Local Intra-Inter Similarity (INRECA) .................................................................................142 5.2.2.4. Modification Costs of Case Adaptation....................................142 5.2.2.5. Conclusions to Relevance .........................................................143 5.2.3. Conclusions to Implicit Knowledge Representations.............................143 5.3 Summary.................................................................................................................144 Chapter 6: Analysing Inreca Using the Task-Method Decomposition Model for Cbr ...............145 6.1 The Task-Method Decomposition Model...............................................................145 6.1.1.

Definitions for Task-Method Decomposition...........................145

6.1.1.1. The Scientific Background of the Task-Method Decomposition Model..............................................................................147 6.1.1.2. The Main Task of CBR in the Task-Method Decomposition Model..............................................................................147 6.2 Retrieve...................................................................................................................149 6.2.1. Identify Features .....................................................................................151 6.2.1.1. The Integrated Algorithm for Test Selection and Descriptor Identification ..........................................................................152 6.2.2. Search......................................................................................................154 6.2.3. Initially Match.........................................................................................157 6.2.4. Select.......................................................................................................159 6.2.5. Knowledge-Intensive Retrieve with Integrated Symbolic Models (MoCAS, KALES).................................................................................................160 6.2.5.1. An Inquiry for Knowledge-Intensive Retrieve .........................162 6.2.6. Integrated Retrieval with k-d Trees (INRECA) .......................................165 6.2.7. Sequential Retrieve (CcC+) ....................................................................166

xiii

6.2.8. Integrated Retrieval with k-d Tree and Context-Graph (INRECA/S3+) ......................................................................................................167 6.2.9. Explanation-Based Retrieve in Integrated Knowledge Representations (PROTOS, CREEK) .....................................................................169 6.2.10. Conclusions to the Retrieval Task ..........................................................171 6.3 Reuse ......................................................................................................................172 6.3.1. Copy (PATDEX/1,/2) ...............................................................................173 6.3.2. Case Adaptation Methods .......................................................................173 6.3.2.1. Substitutional Reuse (CHEF, INRECA) ......................................173 6.3.2.2. Transformational Reuse (CASEY).............................................174 6.3.2.3. Derivational Reuse Planning the Modification (PRODIGY/ANALOGY, CAPLAN/CBC) .....................................................175 6.3.2.4. Case-Based Derivational Reuse (INRECA) ...............................176 6.3.3. Conclusions.............................................................................................177 6.4 Revise (CHEF, PATDEX/2) ......................................................................................177 6.4.1. Evaluate Solution ....................................................................................179 6.4.2. Repair Fault.............................................................................................179 6.4.3. Validity Measures to Detect Faults in the Knowledge Base ..................180 6.4.3.1. Similarity as an Effectiveness Measure for Case Cluster Revision (INRECA)...................................................................................180 6.4.3.2. Consistency Check in Symbolic Models (MoCAS) ..................182 6.4.3.3. Detecting Deficiencies in Context-Graph and k-d Tree (S3+/INRECA) ..........................................................................................183 6.4.3.3.1. Structural Deficiencies in the Strategy Knowledge ...................................................................................183 6.4.3.3.2. Revision of Knowledge Structure Deficiencies with Functional Impact on the Selected Solution ........................185 6.4.4. Conclusions.............................................................................................186 6.5 Retain......................................................................................................................187 6.5.1. Extraction (CASEY, CREEK) ...................................................................189 6.5.1.1. The Inference Method and Explanation....................................189 xiv

6.5.1.2. Relevant Descriptors.................................................................190 6.5.1.3. The Justification........................................................................191 6.5.1.4. Extracting Empirical Knowledge by Analogy Using Symbolic Models (MOLTKE, KALES)......................................................192 6.5.2. Integration of Knowledge and Indexes ...................................................194 6.5.2.1. Changing the Concept Structure of the System (CcC+) ...........194 6.5.2.2. Integrating Nearest Neighbour Links (CcC+, CREEK) .............195 6.5.2.3. Integrating Noise-Resistant Prototypical Cases with Gauss-Filters (WheelNet) ........................................................................196 6.5.2.4. Automatic Integration into Experience Graphs (PATDEX/1) ..............................................................................................198 6.5.2.5. Update of Relevance Matrices (PATDEX/2)..............................201 6.5.2.6. Generating Shortcut Rules (MOLTKE)......................................202 6.5.2.7. Integration of Symbolic Models (MOLTKE, MoCAS)...............204 6.5.2.8. Sustained Learning in Integrated Knowledge Structures (PROTOS, CREEK) ....................................................................................205 6.5.3. Conclusions to the Retain Task ..............................................................208 6.6 Summary.................................................................................................................209 Chapter 7: One Result of the Evaluation of Cbr Technology: The Seamless Integration of Induction and Cbr in Inreca ....................................................................................................210 7.1 Introduction ............................................................................................................210 7.2 The Inreca-Tree ......................................................................................................215 7.2.1. Extending the k-d Tree Building and Retrieval Methods .......................216 7.2.1.1. Extending the BOB-Test............................................................216 7.2.1.2. Extending the BWB-Test...........................................................217 7.2.2. Generality of the Inreca-Tree..................................................................217 7.3 Compiling Inductive Knowledge into the Similarity Assessment .........................218 7.3.1. Determination of Optimal Weight Vectors.............................................219 7.3.2. Experiments ............................................................................................220 7.3.3. Learning Preferences ..............................................................................221

xv

7.3.4. What Did We Really Gain? ....................................................................224 7.4 Conclusion ..............................................................................................................225 7.5 Summary.................................................................................................................226 Chapter 8: Comparing Inreca with Related Work.......................................................................227 8.1 Introduction ............................................................................................................229 8.2 Case-Based Reasoning ...........................................................................................229 8.2.1

A Selected CBR Application: Legal Reasoning......................................230 8.2.1.1. Semantic Nets for Legal Reasoning..........................................230 8.2.1.2. An Integrated Case-Based and Rule-Based Approach for Legal Reasoning ......................................................................................231

8.2.2

Selected Integrated CBR Applications ....................................................231

8.2.3. CBR Systems as Tophead on Databases..................................................232 8.2.3.1. Similarity Based Queries with SQL..........................................233 8.2.3.2. Statistic Analysis with CLIPS ....................................................233 8.2.4. Goal-Directed Queries as Tophead on Databases...................................234 8.3. CBR Systems with Cluster Trees ............................................................................234 8.3.1. Integrated Conceptual Clustering and Categorisation ............................235 8.4 Induction.................................................................................................................236 8.4.1

Non-TDIDT Approaches..........................................................................236

8.4.2

Improvements on TDIDT Algorithms......................................................237

8.4.3

Continuous-Valued Attributes ................................................................237

8.4.4

Random Splitting ....................................................................................238

8.4.5

Induction as Knowledge Acquisition Technique....................................238

8.4.6. Decision Tree with Hyper-Planes ...........................................................239 8.4.7. Decision Tree with a Contrast Measure for Cut-Points' Selection .........240 8.4.8. Binary Decision Trees with Bayesian Diagnosis....................................240 8.4.9. Binary Decision Trees with ID3 and Entropy ........................................241 8.4.10. Decision Tree with Failure/Success Statistic for Information Retrieval...............................................................................................................242

xvi

8.5 Diagnosis ................................................................................................................243 8.5.1

Ambient Conditions of Commercial Diagnosis......................................245

8.6 Model-Based Diagnosis..........................................................................................249 8.6.1

Some Example Systems ..........................................................................251

8.6.2

Monitoring and Diagnosis ......................................................................252

8.6.3. Model-Based Information Retrieval for Design Tasks...........................253 8.6.4. Symbolic Models for Diagnosis..............................................................253 8.6.5

Model-Based Diagnosis and Repair .......................................................253

8.7. Real-Time AI Approaches......................................................................................254 8.7.1. Monitoring and Diagnosis with AND/OR-Nets.......................................254 8.7.2. Real-Time Decision Support Modules with Constraint Satisfaction ..........................................................................................................255 8.7.3. Symbolic Models for Real-Time Applications .......................................257 8.8.

Rule-Based Approaches .......................................................................................258 8.8.1. Integrating Time Dependent Symptoms in Rule-Based Systems ...........258 8.8.2 Tracing Rule Chains in Qualitative Constraint-Linked Domain Models 258 8.8.3 Dempster-Shafer Guided Rule-Chain-Tracing in Cause-Effect Hierarchies...........................................................................................................259 8.8.4. Semantic Nets with Rules .......................................................................261 8.8.5. Meta-Level Reasoning by Application-Specific Meta-Level Rules 262 8.8.6

Meta-Level Reasoning within a Meta-Level Architecture .....................263

8.9 Working with Vague Relations ...............................................................................265 8.9.1

Vague Relations for Production Rules....................................................265

8.9.2

Scale-Driven Object Matching for Model-Based Reasoning .................266

8.9.3

Real-Time Fuzzy Control .......................................................................267

8.10 Integration...............................................................................................................269 8.10.1 Integration for Software Development Project Support .........................269 8.10.2 Integrated Model-Based Diagnosis.........................................................270 xvii

8.10.3 Hybrid Integration of Rules and Neural Nets .........................................271 8.10.4 Integrated Help Desks.............................................................................271 8.10.5 Integration of Planning Tasks in a Multiprocessor Model .....................272 8.11 Summary.................................................................................................................273 Chapter 9: Towards an Analytic Framework for Evaluating and Developing Cbr Systems........................................................................................................................................275 9.1 References and Roots .............................................................................................276 9.2 The Framework and the Methodology ...................................................................277 9.2.1. Introduction.............................................................................................277 9.2.2. Basic Approach.......................................................................................278 9.2.2.1. Knowledge-Level Analysis.........................................................278 9.2.2.2. Similarity as a Focusing Mechanism ........................................279 9.2.2.3. Tasks and Domain Characterisations..........................................280 9.2.3. Framework and Methodological Issues ..................................................280 9.2.3.1. Basic Framework Components ...................................................280 9.2.3.2. Methodology...............................................................................282 9.2.4. Examples.................................................................................................284 9.2.5. Conclusion and Further Research ...........................................................286 9.3 Case-Based Reasoning for Analytic Tasks.............................................................286 9.3.1.1. Classification ..............................................................................287 9.3.1.2. Diagnosis ....................................................................................288 9.3.1.3. Decision Support.........................................................................289 9.4 Summary.................................................................................................................291 Chapter 10: Conclusions on the Evaluation of the Inreca System ..............................................292 References ...................................................................................................................................298

xviii

CHAPTER 1: INTRODUCTION 1.1

Overview

The goal of INRECA1 was developing and implementing a case-based reasoning (CBR) system for real applications with a special focus on diagnostic kinds of problems. Comparing INRECA to other approaches for diagnostic problem solving must be done along different dimensions. At the beginning we will give a short overview on other fields influencing CBR and position INRECA on a very abstract level. For some selected systems this will be deepened later in this chapter. We will introduce different kinds of evaluation criteria that will be used to compare INRECA with industrial CBR tools as well as CBR-related research prototype systems. A number of selected applications will be presented that give feedback on the appropriateness of INRECA for the respective kinds of tasks. We will use the task-method decomposition model for CBR (Aamodt & Plaza, 1994) for classifying the methods used in INRECA with respect to methods used in other CBR-related systems. We will also show INRECA's role within newly developed abstracted2 algorithms that have been recently introduced for the task-method decomposition model, based on the selected CBR-related systems (Weis, 1995a; Weis, 1995b; Althoff & Aamodt, 1996a). We will present the seamless integration of CBR and induction in INRECA as one main result which has been motivated by our evaluation task. We will compare the INRECA system with related work according to a list of systematically chosen aspects. Further, a first step into the direction of an analytic framework for CBR systems will be sketched. Finally, we will draw a number of conclusions on the evaluation of INRECA.

1.2

Positioning CBR

CBR is a recent approach to problem solving and learning that has got a lot of attention over the last few years. Originating in the US, the basic idea and underlying theories have spread over the continents, and we are now within a period of highly active research in CBR in Europe as well. CBR is a problem solving paradigm that in many respects is fundamentally different from other major approaches in artificial intelligence (AI). Instead of relying solely on general knowledge of a problem domain, or making associations along generalised relationships between problem descriptors and conclusions, CBR is able to utilise the specific knowledge of previously

1

2

Detailed and compact descriptions of the final integrated INRECA system, to which we refer as INRECA system or shortly INRECA, can be found in Traphöner, Bräuer et al. (1995) and Wess (1995). An overview is given in Althoff, Auriol e al. (1995b). Throughout this document the description of INRECA is distributed on several chapters and sections according to the purpose of the respective sub-parts. from the analysed systems

1

experienced, concrete problem situations (cases). At the highest level of generality, a general CBR cycle may be described by the following four processes: Retrieve the most similar (probable, expressive, evident) cases, reuse the information and knowledge indexed by this case to propose a solution to the real world problem, revise the proposed solution (and knowledge structures), and retain the parts of experience likely to be useful for future problem solving (Aamodt & Plaza, 1994). CBR as a field has been influenced by a number of other fields, e.g. cognitive science, knowledge-based systems, machine learning, databases, information retrieval, and neural networks. There are also a lot of commonalties with fields like uncertainty, pattern recognition, and statistics (fig. 1.1). Knowledge Based Systems

Cognitive Science Machine Learning

Databases

Case-Based Reasoning

Uncertainty

Pattern Recognition Neural Networks Information Retrieval

Statistics

Fig. 1.1 – Main fields influencing CBR

1.2.1.

CBR and Cognitive Science

Cognitive science had large influence on the development of the field of CBR. From the cognitive science point of view, CBR represents a cognitive model of human problem solving behaviour (Kolodner, 1993; Strube & Janetzko, 1990; Weber, 1993). Here, cases correspond to concrete empirical knowledge. They abstract from underlying events or processes that have a concrete correspondence in time and space (Strube, 1989). A central aspect is the notion of an episodic memory (Tulving, 1972). Further important aspects of the cognitive modelling are the representation (and formation) of concepts (Wrobel, 1994), the representation of the case memory, the reminding of made experiences (Bartlett, 1932), and the transfer of known solutions to the problem at hand. Cognitive science also contributes various models, based on psychological knowledge, for determining and evaluating the similarity between cases (e.g., Tversky, 1977; Gentner, 1989; Gentner & Forbus, 1991: fig. 1.2) as well as procedures for applying empirical knowledge to new situations (Burkhard, 1995).

2

Problem Description

simple matching

simple matching

simple matching

complex matching

S e l e c t o r

complex matching

S e l e c t o r

Best Match

Case Memory MAC Phase

FAC Phase

Fig. 1.2 – The MAC/FAC Model1 (Gentner & Forbus, 1991)

1.2.2.

CBR and Knowledge-Based Systems

In the past knowledge-based systems mainly focused on the use of general(ised) knowledge, especially rules. Problems like exploding search spaces (Newell & Simon, 1972), the knowledge acquisition bottleneck (Feigenbaum & McCorduck, 1983), the maintenance of the knowledge base (Hayes-Roth, Waterman & Lenat, 1983), the brittleness problem (Lenat et al., 1986), and the problem of missing remindings (Riesbeck & Schank, 1989) led to a number of approaches sometimes called „Second Generation Expert Systems“ (David, Krivine & Simmons, 1993). These approaches include the integration of deep causal models, the combination of different reasoning techniques, the use of knowledge level analysis as well as different knowledge acquisition support tools (David, Krivine & Simmons, 1993). Especially much effort has been spent on enhancing the knowledge engineering process (Steels & Lepape, 1992). The – especially in Europe – rapidly growing field of CBR offers an additional contribution to the development of knowledge-based systems. It utilises case-specific knowledge in a way that is more flexible than its use within other knowledge acquisition approaches, including those basing on inductive learning techniques. CBR offers a direct approach to the integration of problem solving and learning which helps to overcome the problems associated with knowledge base maintenance (e.g., sustained learning from experience, Aamodt, 1995). CBR also helps to develop an integrative view on the processing of data, information, and knowledge (Aamodt & Nygård, 1995). CBR represents a natural approach to knowledge-based systems that learn from experience by retaining and reusing case-specific knowledge. Until now knowledge-based systems have mainly focused on utilising general domain knowledge for solving problems. The difficulty of modelling this type of knowledge, and its insufficiency for a range of problem solving and learning tasks, has led to the increased interest in case-based problem solving and learning methods. Current approaches – including INRECA – try to benefit from the synergetic combination of case-specific and general domain knowledge

1

Many are called, few are chosen

3

with implications for knowledge modelling, problem-solving methods, learning methods as well as system development in general (Bartsch-Spörl & Wess, 1996b).

1.2.3.

CBR and Machine Learning

From the retain process in the above mentioned CBR cycle it is clear that there is a strong relationship between CBR and (other approaches to) machine learning. Learning of specific knowledge in terms of past cases represents the major approach to sustained learning in today's machine learning research (Aamodt, 1995). The learning process becomes a process of extracting relevant information from a problem-solving experience, and indexing this case in the system's knowledge structure in a way that facilitates retrieval when a similar problem is encountered later. The case-based approach to reasoning and machine learning has had a considerable growth during the last few years (Kolodner, 1993; Aamodt & Plaza, 1994; Wess, Althoff & Richter, 1994). Earlier research of fundamental importance to this field includes Schank's and Kolodner's work on memory structures for learning and reasoning (Schank, 1982; Kolodner, 1983) and the work on transformational and derivational analogy (Carbonell, 1983; 1986). From an abstract point of view, e.g. using the characteristics of inductive learning of Jantke and Lange (1989), case-based learning can be described as shown in the lower part of figure 1.3: From a given sequence of cases, learning hypotheses are incrementally generated. Such a hypothesis consists of a pair of a set of cases and an associated similarity measure. Questions to be answered are: Which cases will be entered into the case-base, which will be removed from it, and how to realise the similarity measure. Since the criteria of Jantke and Lange (1989) can be analogously applied to case-based learning, it can be viewed as a special instance of inductive learning. A common theoretical framework is necessary to achieve results on this topic. First results have been described by Jantke (1992), Globig and Lange (1994), and Wess (1995).

4

Input:

C1, C2, C3, ..., Ck

sequence of cases

Output: sequence of hypotheses

IND

Goal: Convergence ∃ m ∀n≥m Hn=Hn+1

H1, H2, H3, ..., Hk C1, C2, C3, ..., Ck Input: sequence of cases Output: sequence of case sets and similarity measures

CBL

Goal: Convergence (CS1,SIM1), (CS2,SIM2), ..., (CSk,SIMk)

∃m ∀n≥m (CSn,SIMn)=(CSn+1,SIMn+1)

CSi ⊂ {C1, C2, C3, ..., Ci}

Fig. 1.3 – Inductive versus case-based learning

Both inductive learning and case-based learning have in common that they derive „global“ knowledge, which in both cases is uncertain, from „local“ observations. However, they use different techniques to achieve this: inductive learning bases mainly on logical concept descriptions („logical reasoning“), whereas case-based reasoners often use analytic descriptions („geometric reasoning“) (cf., e.g., Richter, 1992). One consequence from this is that inductive learners often start with the „dropping of complete dimensions“ in contrast to case-based reasoners which „decompose complete dimensions into intervals“. Which particular technique is the more successful one depends on how the learning result will be used (see also Morik, 1996, for a contrast of CBR and (other) machine learning methods, especially induction). On the other hand, if we take a more concrete view (Richter, 1995) there appears to be an important difference between CBR and (other approaches to) machine learning, especially those dealing with inductive inference (fig. 1.4). While inductive learning techniques generate a knowledge base mostly in a compilative style (the cases disappear), CBR includes the flexibility of deciding how much knowledge is compiled into the similarity measure, and how much knowledge is kept in the case base and can be interpreted at run time (i.e., the user/system developer can decide to which degree knowledge compilation can take place). This offers more flexibility and it depends on the application domain and task at hand which approach is the best suited. The seamless integration of induction and CBR in INRECA (chapter 7) allows to combine the flexibility of CBR in the first phases of systems development with the performance of decision tree consultation in the later phases. Here both learning strategies are integrated within a broader architecture for identification and diagnostic reasoning.

5

Database Approach

Case-Based Reasoning

Machine Learning

Concept = Case Base + Similarity Measure • Knowledge coded into the “cases” (all situations) • Simple “similarity measure”

• Knowledge coded into the “similarity measure” (predicates) • No cases

Storing

Generalising Storing and Generalising

Fig. 1.4 – General knowledge in case-based systems

The above contrast can also be applied to databases as well as to systems that are based on general knowledge only. While in databases no knowledge compilation takes place and all „knowledge“ is kept in the cases, in systems based on general knowledge the whole knowledge has been compiled into the system „by hand“. In both cases no flexibility concerning the compilation of knowledge is available to support system development. One improvement here is the integration of case-specific and general knowledge – as INRECA does – and the combined use of databases together with a CBR server as, e.g., in the INRECA client-server-architecture (fig. 1.5). Application Environment

Case Retrieval and Adaptation

Database Server

Application

Application

Case Storage

INRECA-Server

Database Server Database Server

Application Development Environment Domain Model

Fig. 1.5 – Client-server model of INRECA (Wess, 1995)

6

1.2.4.

CBR and Information Retrieval

Although CBR and information retrieval have a great deal in common and are often used for similar tasks (table 1.1), the two techniques have been compared very seldom (for a more recent comparison see Globig, Kamp & Lange, 1998). This separation may result from the fact that CBR and information retrieval originated and were developed in two different communities. More recent work on "textual CBR" tries to overcome this problem (e.g., Lenz, 1999). Primary goal Kind of information Kind of search Interpretation of data Kind of queries Result of a query Indexing Feedback of the user Access via

Databases efficiency data records/objects exact match no automatic set syntactic no attributes

Information Retrieval quota of matches text vague match yes interactive list semantic possible contents

CBR problem solution objects vague match yes interactive list semantic yes dependencies

Table 1.1 – Comparing databases, information retrieval and CBR

CBR as well as information retrieval focus on retrieving relevant information from a database of collected data. Both allow flexible database querying and result in a collection of relevant but inexact matches (that is the main difference between the two and relational database systems). The following differences between CBR and information retrieval can be set out: • Data type. Whereas information retrieval methods mainly operate on textual data, traditional CBR methods operate on vectors of several basic data types: real, integer, symbol, Boolean, string etc. • Amount of data. Information retrieval methods can handle huge amount of data. Information retrieval can search thousands of documents, consuming up to some gigabytes of memory. CBR systems are comparatively limited. • Use of knowledge. Information retrieval systems operate without knowledge about the users' problem solving task. They provide a generic indexing and retrieval engine that can be used for wide a range of tasks. The consequence is a limitation in accuracy with respect to a query. CBR systems make much use of knowledge about the problem solving process to build effective indexes and to improve retrieval accuracy. These differences are correct in a surface evaluation of the CBR and information retrieval systems. However, in a deeper comparison, the differences become blurred. Current information retrieval tools often operate on mixed data types and use a thesaurus or concept hierarchies during retrieval (Fuhr, 1996). On the other hand, some of the commercial CBR tools do not represent and use general knowledge (chapter 3). They often only use a similarity function on flat attribute-value vectors. Therefore, it is better to say that CBR tools are primarily concerned with mixed representations whereas information retrieval systems are primarily concerned with textual databases. Furthermore, CBR tools often explicitly represent the knowledge they use, whereas information retrieval systems do not. Hence, it is possible to consider that for complex-structured application 7

tasks that require an integration of different, knowledge-intensive problem solving and learning methods – e.g. in synthetic applications domains – the difference between CBR and information retrieval becomes very obvious. However, for application tasks like decision support, help-desk and diagnosis where often knowledge-poor approaches to similarity assessment and simple reuse strategies are sufficient, the differences between a information retrieval system and a low-level CBR tool are minor, especially when compared to knowledge-based approaches to information retrieval.

1.2.5.

CBR and Statistics/Pattern Recognition

CBR and statistics are complementary techniques in many problem solving processes. Statistics works well on large amounts of standardised data to test known hypotheses. However, most statistical methods are not suited for exploratory analysis (i.e., when all hypotheses are not yet known) because they require strong underlying critical assumptions that are often overlooked by the end-user (for example, the independence of attributes). In addition, when using statistical methods, it is hard to take into account general knowledge. CBR, on the other hand, can make use of general knowledge when available: It integrates numeric as well as symbolic techniques. A CBR system must solve problems with given incomplete information, with varying interpretations of the information depending on the respective context, and without clearly defined goals and preferences of the user. Here the correspondences to fields like uncertainty, fuzzy logic, and also decision theory are very obvious. The use of explicit similarity functions and nearest neighbour classification is a characteristic of CBR that it has in common with pattern recognition and statistics (Dasarathy, 1991; Nakhaeizadeh, 1996: fig. 1.6). alternative forecasting approaches

statistical approaches

regression technique

case-based reasoning

Box-Jenkins method

neural networks

K-nearest neighbours

symbolic machine learning

decision trees

regression trees

Fig. 1.6 – Alternative forecasting approaches (adapted from Nakhaeizadeh, 1994)

Puppe (1994) contrasts approaches to learning from cases for classification problem solving and differentiates between three communities, namely knowledge-based systems, statistics, and neural networks. The methods used for classification problem solving are listed according to the type of abstraction in the knowledge representation. It ranges from "no abstractions", where cases remain unchanged, via "simple abstractions" with only simple symptom-diagnosis 8

correlations, to "complex abstractions". Complex abstractions are divided into complex abstractions with complex rules or an intermediate diagnostic structure and complex abstractions, where the learning technique relies on additional general knowledge which does not stem from the cases. In the category no abstractions general knowledge may be also used for supporting the similarity measure, which can be viewed as a substitute for the generalisations of the other types (table 1.2). No abstractions Simple abstractions Complex abstractions without general knowledge Complex abstractions with general knowledge

Knowledge-based systems Case-based reasoning

Statistics Nearest Neighbour Bayes theorem; linear decision functions

Inductive learning of decision trees or rules Explanation-based learning, adaptation of probabilities

Neural Networks Perceptron learning Backpropagation learning

Bayesian networks

(pre-trained neural networks)

Table 1.2 – Case-based learning methods for diagnostic reasoning (adapted from Puppe, 1994)

1.2.6.

CBR and Neural Networks

Neural nets perform better than CBR in a knowledge-poor environment, when the data cannot be put in symbolic form - for example, recognising radar signals. Their field of application spreads towards pattern recognition, when a lot of points are provided as raw data (vision, speech, image processing). Neural nets are very resilient to noise in the consultation phase: for instance, even with only a fraction of the original information in the fields of features and feature values, the retrieval performance is very high. However, current neural networks present technical deficiencies that are not yet solved. Firstly, they are not well suited when general domain knowledge has to be taken into account. Secondly, they cannot cope with complex structured data. Up to now, even in simple domains, where data are represented through a flat attribute-value table, it is an uneasy task to match a case in a binary bit vector such that it can be used in the network. Furthermore, to perform well the coverage of the domain has to be exhaustive during the "learning" phase. From an ergonomic point of view, neural networks suffer from the "black box syndrome". Their decisions are difficult to evaluate by the field specialists because of the network's black box effect: The output of the network results from a combination of inner weighted vectors which depends on the network's architecture and learning mode. No justification or explanation for these results can be easily provided. On the one hand, the classical bottleneck problem of questioning the field specialists is avoided (as it is done with other machine learning paradigms). On the other hand, the experts can neither validate nor modify the resulting system in a goal-oriented way. Neural networks and CBR can be integrated and co-operate together in some sub-parts of a system. For instance, in the PATDEX system (Althoff & Wess, 1991; Richter, 1992), the relevance of the features with respect to a given diagnosis is updated through a competitive learning mechanism known from unsupervised neural networks. They also complement each other for various application fields: Depending on the amount of available knowledge and on the goal of the target 9

application, the technique used will vary between a low-level knowledge approach where little explanation is required (neural nets) and a rich and complex learning scheme that involves highlevel learning techniques.

1.2.7.

CBR and Uncertainty

The treatment of uncertainty and imprecision is of high importance in CBR (Wess, 1995). This depends on both the kind of applications considered and the technology used within CBR (Bartsch-Spörl, 1994a+b). The most important sources of uncertainty and imprecision in CBR are (Wei & Tsatsoulis, 1993): • Not all the information necessary for a decision is available. • The meaning of the respective information varies in different contexts. • The goals and preferences of the user are not clearly defined. There are several frameworks where authors tried to reuse results other research fields within CBR, namely fuzzy logic (Plaza & Lopez de Mantaras, 1990; Torasso, Portinale et al., 1992), possibility theory (Bonissone & Dutta, 1990), rough sets (Richter & Wess, 1991), utility theory (Richter & Wendel, 1993), theory of evidence of Dempster/Shafer (Richter, 1994) as well as the use of local similarity measures known from pattern recognition and mathematics (Althoff, Auriol et al., 1995).

1.3

Contrasting the INRECA CBR Approach with a Family of Approaches to Diagnostic Problem Solving

In the last section we positioned CBR technology, and hereby also INRECA, by contrasting CBR with neighbouring fields. In this section we will be more concrete by referring to a set of systems (fig. 1.7) that have been developed in our research group in Kaiserslautern, in the INRECA consortium, or in groups that are somehow related to our university group in Kaiserslautern.

10

GenRule

Patdex/1

Make

Awacs

KoDIAG

Moltke

iMake

Kales

Patdex/2

MoCAS/1

Patdex-KD

MoCAS/2

Inreca

Kate-CBR

S3

Paris

S3-CASE

S3+Inreca

Kate

Fig. 1.7 – MOLTKE and INRECA system families

All these systems have been developed within the scope of the MOLTKE project (Pfeifer & Richter, 1993; Althoff, 1992a) or the INRECA project which can be viewed as a natural supplementation (Wess, 1995). By referring to such systems we can classify INRECA, still on some abstract level but more concrete than in section 1.2. The explicit relationship of INRECA to these systems opens up the door for evaluating INRECA with respect to other approaches to diagnostic problem solving because all the systems described below have been published and discussed with related work in their respective fields or sub-fields. We will also refer to systems included in figure 1.7 throughout this document for the purpose of exemplifying the application of the respective evaluation criteria. We contrast INRECA with six different approaches (fig. 1.8): •

•

• • • •

an integrated learning system for technical diagnosis (MOLTKE workbench) that includes a CBR sub-system as well as a model-based and an inductive reasoning sub-system (Althoff, 1992a), an associative rule-based expert system shell for diagnostic reasoning (MOLTKE diagnosis shell) that had been combined with the PATDEX CBR system (Althoff & Wess, 1991) – both part of the MOLTKE workbench –, a service support system (S3) based on associative rule-based diagnosis and hypermedia techniques (Traphöner, 1993), a deep integration (S3+/Inreca) of a service support system with the INRECA system itself (Priebisch, 1995), a model-based diagnosis system for case adaptation (MoCAS) in the area of technical diagnosis (Pews, 1994), and a neural network based system for diagnostic reasoning (KoDIAG) using Kohonen nets (Rahmel, 1995).

11

We hope that these contrasts give some concrete insights in the differences and commonalties of the underlying technologies of associative and model-based diagnosis as well as learning approaches to diagnostic problem solving. Associative Diagnosis & CBR

Model-Based Diagnosis & CBR

MoCAS

Moltke/Patdex INRECA

Integrated Diagnostic Systems

Moltke workbench KoDIAG

S3

Service Support

S3+/ Inreca Service Support & CBR

Diagnosis with Neural Networks Fig. 1.8 – Contrasting INRECA with other approaches

1.3.1.

MOLTKE Workbench for Fault Diagnosis in CNC Machine Tools

Fault diagnosis in engineering systems is a domain of extraordinary economical importance. Currently, many approaches and applications exist which try to handle diagnostic tasks, but there are still some unsolved problems. Due to task customisation and rapid further developments, only few fully identical CNC machines of a given type exist. This makes it difficult to find somebody who is really an expert for the maintenance of a special machine, especially if the type is relatively new. Service technicians troubleshooting such a new machine cannot rely on machine-specific heuristics, but have to use general technical knowledge, a detailed description of structure and behaviour of the machine, and a general understanding of the functionality of the device to find any faults. In the course of time, technicians learn more about a special machine type through the faults they are confronted with during their work (cases). Such empirical knowledge enables the refinement and application of machine-specific heuristics. Within this section, we present an expert system architecture (technical diagnosis shell) which has successfully handled the problem of fault diagnosis of CNC (computerised numerical control) machining centres (as well as similar problems) on the level of fully implemented research prototypes which have been developed in cooperation with a mechanical engineering research institute (Althoff, Faupel et al., 1989). The expert system architecture is embedded in a

12

complete knowledge acquisition workbench (the MOLTKE1 workbench; Althoff, Maurer & Rehbold, 1990; Althoff, 1992a+b; Pfeifer & Richter, 1993) which makes use of the three main knowledge sources of the CNC domain (fig. 1.9). Case-specific knowledge

General knowledge of the experts

General knowledge about the machine

Integrated workbench for technical diagnosis (Moltke)

Knowledge base Fig. 1.9 – Main knowledge sources of technical diagnosis

After the introduction of the basic workbench characteristics, we present its main sub-systems (which deal with these knowledge sources) during the following sections. Finally, we summarise the key features of the presented approach.

1.3.1.1.

Workbench: Modelling Knowledge for Diagnostic Tasks

The MOLTKE workbench combines different techniques of second generation expert systems (qualitative, case-based, inductive reasoning). It can process knowledge from the main sources of technical diagnosis tasks: General knowledge about the function and behaviour of the machine (technical model), general knowledge from the expert as well as case-specific knowledge of service technicians. The workbench consists of four main components: A diagnostic object-oriented shell (MOLTKE shell), a knowledge compiler which generates a (partial) knowledge base from construction plans of the machine (iMAKE), an inductive learning system for the extraction of strategy knowledge from cases (GenRule), and a CBR system which co-operates with the shell problem solver (PATDEX). Building a knowledge base with MOLTKE (fig. 1.10) starts with the graphical modelling of sub-parts of the machine. Then MOLTKE generates a component-oriented, hierarchical model of the technical device and a qualitative, static description of the device's behaviour which is (incrementally) compiled into the knowledge base. This knowledge can be improved by the (incremental) generation of strategy knowledge from cases as well as manual knowledge acquisition techniques. Additionally, the CBR system offers support for the handling of specific cases, especially exceptional ones.

1

MOdels, Learning, and Temporal Knowledge in Expert systems for engineering domains

13

Patdex

Cases

Component library

Cooperation

GenRule Component hierarchy

iMake

Initial knowledge base

Optimised knowledge base Manual refinement

Fig. 1.10 – Developing an application using the MOLTKE workbench

One basic assumption of the workbench is that the overall (technical) diagnosis task can be decomposed into the sub-tasks of classification (selecting the most appropriate diagnosis) and test selection (selecting the best next test) (fig. 1.11).

Diagnosis Classification

Test selection

Fig. 1.11 – Decomposing the diagnosis task

Within the workbench, there exist additional components for the processing of temporally distributed symptoms (Nökel & Lamberti, 1991) as well as the adaptation of available knowledgebases to changes in the underlying technical system (the KALES sub-system: De la Ossa, 1992; section 6.5.1.4).

1.3.1.2.

MOLTKE Shell: The Basic Diagnostic Problem Solver

To represent the given knowledge sources, the MOLTKE workbench uses three different representation languages for describing cases, the functional model of the machine, and the general knowledge manually entered by the expert, respectively. In principle, these languages have to be compiled into a representation formalism which is „understandable“ by the underlying diagnostic problem solver. For reasons of simplicity, this diagnostic representation language is identified with that of the expert-entered general knowledge. Thus, this description language functions as the terminological basis for the whole workbench (fig. 1.12).

14

KA component

Knowledge acquisition component

Models

MOLTKE Shell

Model description language

MOLTKE Basic description language for technical diagnosis • Contexts • Symptoms • Tests • Corrections

Compiler Internal representation for technical models

Context graph

KA component Cases Case description language

Compiler

Context interpreter

String Interface

Static knowledge base

Case structure

Integrated Problem Solver Model Simulator

Graphical Knowledge Acquisition Interface

Case-based diagnosis

User Interface dynamic KB

Fig. 1.12 – Knowledge representation and processing in the MOLTKE workbench

The MOLTKE shell itself is a rule-based object-oriented expert system shell within which the knowledge about the diagnostic process can decomposed in a „horizontal“ as well as in a „vertical“ manner. For the latter one, the knowledge is split into contexts which correspond to rough, intermediate, and final diagnoses and build up a directed acyclic graph, the context-graph (fig. 1.13). Within a context, three kinds of knowledge can be differentiated: ordering rules (for choosing a reasonable order of tests), shortcut rules (for deriving unknown symptom values from known ones), and diagnostic rules (for representing the relations between symptoms and all kinds of diagnoses, i.e. a precondition based on symptom values associated with a context).

15

CNC machine failure tool arm not right

tool arm not left

leaky machine tool i47

back spacer valve 21Y1k4 defect

valve 21Y2 switched

relais 21k4 switched

relais 21k4 defect

valve 21Y1 switched

leaky machine tool i46

back spacer valve 21Y1 defect

cable defect relais 21k2

valve 21Y2 not switched

relais contact surface 21k2 defect

I/O-card failure Fig. 1.13 – An exemplary context-graph

The shell problem solver processes a knowledge base using an establish-and-refine strategy (fig. 1.14). The diagnostic process „walks“ through the context-graph by testing symptoms according to the ordering rules of the actual context and switching to a refinement context when its precondition becomes true. If a leaf context is reached the problem solver stops with the current as output. Initialisation

Shortcut rules Classification Auswahl descurrent aktuellen Kontextes Selecting context

Output

yes

Final diagnosis?

Ascertaining symptoms

no

Ordering rules Test selection Fig. 1.14 – Basic algorithm of the MOLTKE shell

16

1.3.1.3.

iMAKE: Building the Basic Knowledge Base

iMAKE (Rehbold, 1991; Schuch, 1992) gets construction plans of the machine and uses its builtin knowledge of electrical and mechanical engineering – e.g. the behaviour of typical components – to form a (deep) causal model of the machine and generate an initial MOLTKE knowledge base (fig. 1.15). We call such a knowledge base without any machine-specific heuristic knowledge basic knowledge base. General components description

Component library

Concrete components descriptions

iMAKE Incremental Model-based Automatic Knowledge Extractor Connectivity description Basic knowledge base for the Moltke shell

Functionality description

Fig. 1.15 – Overview of iMAKE

An important point is that the construction plans can be entered by anyone who is able to read them. There is no need for the special knowledge of an expert to do that. The expert is needed only to provide information about the typical atomic parts in the descriptions, e.g. valves, switches, contactors, hydraulic pistons. The generation of a basic knowledge base then consists of four major steps (fig. 1.16): • An expert builds up a library of the primitive (and maybe common complex) components that are used. This task needs an expert since it should be done as precisely and completely as possible for the sake of the resulting knowledge base. Of course, the library can be used for many different machines. Thus, an expert is not needed for every single machine type or series. • A person with technical skill – at least he/she shall be able to read, understand and reproduce technical structure descriptions – is needed to enter the connectivity description of a given machine, i.e. to build up the structural model. He/she will select parts from the library and build them together to more complex ones finally describing the whole machine. A graphical editor that allows to mimic the construction plans helps to accomplish this step. Note that this task is independent from the availability of a diagnosis expert if the library (step 1) completely covers all components used. With a fully computerised design, development, and production system (computer integrated manufacturing: CIM) it should be possible to get the connectivity information immediately from the design data. 17

• The designer of the machine can specify the desired functionality in form of input/output behaviour. • Finally, the iMAKE system transforms the model of the machine (put together from the library and the connectivity description) with help of the desired input/output behaviour into a knowledge base for the MOLTKE shell. Components

Expert, general

Construction

catalogue, literature

technical knowledge

documents, plans

iMAKE

iMAKE

iMAKE

Phase 1

Phase 2

Phase 3

Model of the technical device

Moltke knowledge base

Components library

User

Fig. 1.16 – Generating a basic knowledge base with iMAKE

1.3.1.4.

GenRule: Learning of Strategy Knowledge

In principle, the basic (iMAKE-generated) knowledge base can diagnose given fault situations (using the shell problem solver). What's still missing is strategy knowledge: How to find the underlying fault(s) as „economical“ as possible. Up to now, the shell problem solver – using the generated basic knowledge base – can only apply uninformed (blind) search because strategy knowledge is not part of the respective knowledge source (construction plans). In spite of this, strategy knowledge can be directly entered by the expert using the shell or represented by the use of case-specific knowledge which reflects observed expert behaviour. The GenRule system (Althoff, 1992a) can improve the generated basic knowledge base by the use of diagnosis cases, i.e. cases that include a diagnosis as a solution. GenRule implements an incremental inductive learning strategy which generates its learning hypotheses (shortcut rules, ordering rules) based on a memory of diagnosis cases and the generated knowledge base (section 6.5.2.6; alg. 6.33). Within the memory, symptom values and diagnoses are used for an efficient indexing. The basic idea of GenRule is to compare observed cases with existing diagnostic „paths“ of the knowledge base. If a case includes a better strategy than the strategy known by the knowledge base (which results in a certain „path“), additional shortcut and ordering rules are generated (left part of fig. 1.17). Both kinds of rules are used to represent a better diagnostic strategy, i.e. the classification result of the basic knowledge base will not be influenced.

18

Associative diagnosis

Case-based diagnosis

Activation

Rules

Cases

Extraction of diagnostic paths

Extraction of exceptional cases

Case memory Fig. 1.17 – Architecture of the case-based sub-system of the workbench

All diagnostic paths are automatically generated from the given knowledge base. The rules are indirectly generated via the integration of cases and/or paths into the case memory (fig. 1.17). The rules are included into the knowledge base if their evaluation based on a special statistic measure exceeds a given threshold. For these rules, the case memory functions as both a dependency network for efficiently updating the statistic rule justification as well as a means for automatically retracting and/or establishing rules within the knowledge base.

1.3.1.5.

PATDEX: Case-Based Diagnosis

While GenRule extracts general strategy knowledge from frequently occurring cases, PATDEX concentrates on the processing of the more exceptional cases (right part of fig. 1.17). Since exceptional situations are „very informative“, they must play an important role within the knowledge acquisition process. PATDEX interprets such cases directly using CBR and implements an extension of both the workbench's classification abilities as well as its strategic abilities. Actually, it is an alternative problem solver to the basic shell problem solver (fig. 1.18). Thus, the underlying case representation language is both an intermediate knowledge representation for further processing via GenRule or by hand as well as a final representation for case-based diagnosis. The PATDEX system (Wess, 1993; Althoff & Wess, 1991; Richter & Wess, 1991; Althoff, 1993) consists of two sub-components for classification and test selection, respectively. Both components rely on a CBR mechanism which bases on the similarity of the underlying cases. The similarity is interpreted by the use of evaluation functions (similarity measures). A procedure which dynamically partitions the case base enables an efficient computation and updating of the similarity (section 5.1.2.1).

19

Initial situation Technical knowledge

Feedback

Similarity Test selection Synthesis

Diagnosis possible?

Feedback

no

yes Diagnosis

correct? (yes/no)

Fig. 1.18 – Basic algorithm of PATDEX

In case of the classification sub-component, the applied similarity measures are completely dynamic. The underlying evaluation function can be adapted to observed expert behaviour using a connectionist learning technique (competitive learning; section 6.5.2.5). For the test selection component, the adaptation of similarity measures is based on an estimation of the average diagnostic costs using an A*-like procedure. PATDEX can deal with redundant, incomplete, and incorrect cases and includes the processing of uncertain knowledge via default values for symptoms. All the general knowledge being represented within the workbench can be used as background theory (e.g. for the identification of abnormal and/or relevant symptoms as well as the derivation of further symptom values). PATDEX offers a learning-apprentice-like support for the development and the maintenance of the case memory. It can also work as a stand-alone case-based diagnosis system.

20

diagnostic cases

model inquiry

qualitative engineering

deep model

case memory PATDEX

GenRule

iMAKE

case-based diagnosis

case-compiler

model-compiler

certainty statements

heuristic rules

contexts + causal rules

Diagnosis Shell Fig. 1.19 – The organisation of the MOLTKE workbench

1.3.1.6.

Conclusion

The MOLTKE workbench has been successfully applied to several real world domains. It integrates different knowledge representation formalisms, learning strategies, and reasoning schemes. Thus, MOLTKE enables a flexible knowledge acquisition support which automates as much of the knowledge base development as reasonable. Of course, the automated tools underlie certain restrictions, e.g.: iMAKE can mainly deal with hydraulic and electric/electronic components of technical devices. Since iMAKE uses a static description of the devices' behaviour, the machine should have a central control unit. The (space and time) complexity of GenRule's case memory requires a strongly decomposable domain. PATDEX's adaptive learning mechanism is restricted to the given (built-in) schema of the evaluation function. Normally, fault diagnosis in engineering systems describes a class of application domains which meets these requirements. As an illustration of the dependencies of the above described components, the overall organisation of the workbench is shown in figure 1.19. Using the MOLTKE workbench we developed (a.o.) knowledge-bases for fault diagnosis of CNC machining centres, 3D-CNC measuring machines, driving machines in mining, and for the diagnosis of failures in heterogeneous computer networks.

21

1.3.2.

MOLTKE/PATDEX: Associative Rule-Based Diagnosis and CBR

Based on the two available workbench problem solvers there can be a number of different strategies implemented for both the classification and the test selection sub-task (fig. 1.20).

Initialisation Shortcut rules

GenRule

Auswahl des aktuellenPatdex Kontextes Moltke-Shell

Case base

Classification

Result

yes

final diagnosis

test symptoms

no

Test selection Ordering rules

Shortcut rules

Diagnosis and strategy cases

Strategy cases

Fig. 1.20 – Cooperation of PATDEX and the MOLTKE shell

The default co-operation between PATDEX and the MOLTKE shell is as follows. The MOLTKE shell controls the test selection while PATDEX does its case-based classification as a background job. The test selection component of PATDEX is used for clarification purposes only, i.e. if contradictory classifications of the systems are present and cannot be solved immediately. In view of the co-operation of the two problem solvers the following situations are of special interest: • The shell classifies correctly and PATDEX finds no a sufficiently similar case in its case-base: The diagnosis can be made, because no contradiction has been observed. • Both the shell and PATDEX classify correctly: The diagnosis can be made. There is the possibility to exclude the cases from the case-base, because the knowledge base already covers them. • PATDEX classifies correctly and the shell does not find any diagnosis: There is a gap in the knowledge base. If the most similar case within the PATDEX case base is a temporary exception, then the knowledge base can be extended and the case excluded from the case base. • The shell and PATDEX classify differently: Different conclusions have to be taken depending on the actual situation. It is possible that PATDEX is over generalising and the shell is classifying correctly, or that PATDEX is classifying in a more special way than the shell and an exception can be identified. It is necessary to prove the case chosen by PATDEX . If the case is proven to be sound, then the knowledge base does not fit to the current situation. It must be checked if the knowledge base still makes sense taking a 22

more general point of view. If it is not possible to prove the selected case, the respective diagnosis threshold must be increased. • Other situations, e.g. wrong classifications by PATDEX or the shell: The correct case can be directly given as input. Afterwards PATDEX can correctly classify the identical situation. If necessary wrong parts of the knowledge base have to be corrected. In addition to the above co-operation possibilities, PATDEX 's test selection mechanism can be used as the default test selection component of the shell. GenRule determines, and regularly updates, the sub-set of the overall case memory which is available in the PATDEX case base (fig. 1.17 and fig. 1.20). The MOLTKE shell can reset a symptom value to unknown and to reject all inferences by shortcut rules based on its old value. This facility is used when the user or the system retracts uncertain symptom values. The mechanism is supported by means of a formula dependency network (fig. 1.21). In the co-operating version of PATDEX and the MOLTKE shell (Koks, 1992) the same network is used to support the relational indexing in PATDEX. Here instead of complex formulas cases are represented. Thus, one identical dependency network is used for accessing cases, rules, or contexts.

Symptoms

Atomic Formulas

Complex Formulas

F1 R22K1

F7

R22K1=switched F2

Relay22K1

(and F2 F3)

R22K1=not switched F3

R22K2

F8

R22K2=switched F4

Relay22K2

(and F2 F4)

R22K2=not switched F5 SPP460

F9

SPP460=24V

Voltage Pot.pkt.460

F6

(and F8 F6)

SPP460=others

Fig. 1.21 – Exemplary formula network of the MOLTKE shell

1.3.3.

Characterising the Task of Fault Diagnosis of CNC Machine Tools

We can differentiate between three main views on the MOLTKE workbench resulting in three different groups of heuristics for its design: the simulation view, the technical view, and the pragmatic view. These heuristics arise from corresponding domain and application task

23

characteristics for fault diagnosis of CNC machine tools (Pfeifer & Faupel, 1993; Althoff, 1992a). The simulation view: Modelling the learning behaviour of experienced experts: • Learning for a special purpose, namely to improve the diagnostic capabilities of the system. Thus, we have a strong interlocking of the diagnostic and the learning process; • Representation and efficient processing of general and case-specific knowledge; • Generating general knowledge from case-specific knowledge; • Representation and efficient processing of general knowledge about the CNC machine; • Modelling of the „shortcut-oriented diagnostic problem solving behaviour“ (ascertaining as few symptoms as possible) of the respective experts; • Modelling of forgetting strategies. The technical view: Knowledge acquisition support as far as possible: • Learning of elements of the MOLTKE representation language(s); • Simple, understandable modelling and processing of uncertain knowledge; • Direct interpretation of case-specific knowledge; • If a correct diagnosis case is known, then it must be used for classification purposes if the same situation occurs again. The pragmatic view: Cost-benefit analysis concerning usable resources: • Effective reduction of the needed knowledge acquisition effort; • Improved maintainability of the knowledge-base; • Ease of adaptation of the knowledge base to simple modifications of the machine; • Ease of transfer of available knowledge-bases for diagnosing similar machines; • Efficient learning procedures to supplement interactive knowledge acquisition mechanisms; • General classification knowledge acquired manually or generated by iMAKE is considered to be more certain than case-specific knowledge; • General strategy knowledge acquired manually or extracted from case-specific knowledge is considered to be more understandable than general strategy knowledge which has been extracted from the general knowledge about the CNC machine (using iMAKE).

1.3.4.

S3: Service Support System

The S3 system (Service Support System; Traphöner, 1993) is a system for technical diagnosis like the systems mentioned above. Its focal point is the administration of technical documentation. Its basic paradigm is associative rule-based reasoning using contexts and rules. The idea was to represent failures of the target machine by the explicit definition of contexts. These are ordered hierarchically due to the target machine's conceptual structure (fig. 1.13). This way the so-called failure graph evolves. The failure contexts are linked by the edges to the more specialised failure concepts. Unlike the MOLTKE shell a decision tree is used instead of (ordering) rules to determine specialisations of a current context, which is directly related to the

24

current situation. The current situation is derived from the initial situation and tests, which were explicitly provided by the decision trees of the former more general contexts. The diagnostic process is directly determined by incremental specialisations of the initial rough diagnosis. In addition to this basic functionality, S3 supports the following hypermedia structures: Graphics, slides, films, and audio. Further annotations may be integrated into symptoms, contexts, and decision nodes. S3 – as a MOLTKE shell derivative – was especially developed for the purpose of service support (fig. 1.22). Therefore, the target group were technicians, which use or repair, e.g., CNC machine tools. The system should be typically in use on a laptop. During the diagnostic process the system provides the technician, which uses the system as an expert, with information or the documentation of the machine. Form this point of view the diagnostic process can be seen also as a process for information retrieval, which in general is a primary database action. Afterwards, the technician is supported by the development of the protocol of the diagnostic process. The only process not integrated yet is a database for the administration of client-specific data. Customers' and machines' data bases

Protocols as a case base

Service Support Systems Knowledge for diagnosis

Technical documentation

Fig. 1.22 – Components of a service support system

Requirements for service support systems fulfilled by S3: •

•

• •

• • •

Different kinds of knowledge: The S3 system supports directly general knowledge of the experts and general technical knowledge about the machine (section 1.3.5 for an improvement here). Variants: Though S3 does not directly support different variants of the same problem, it can easily handle several because of its highly modularised structure of the contextgraph. Transparency: S3 has a graphical interface, an explanation or justification module, and hypermedia properties. Maintenance: S3 can be easily maintained because of its highly modularised structure of the context-graph. However, there is no automatic support up to now (section 1.3.5 for an improvement here). Free selection of input data: The service technician can freely select his strategy and can provide any input data on any time. Non monotony in the inference process: S3 applies pure top-down search and provides no backtracking algorithm (section 1.3.5 for an improvement here). Efficiency: S3's efficiency is directly related to the complexity of the knowledge base and the amount of used hypermedia components, which are memory-intensive structures. 25

• • • • • • •

Repair messages: These can be integrated into the diagnosis context using the hypermedia components. Easy access to information: S3 has an advanced graphical user interface which can provide any information request at any time. Polyglot: S3 supports different languages. Plausibility test on input data: There is no plausibility test. Explanation: The explanation of the reasoning process is extracted directly from the semantics of the rules (section 1.3.5 for an improvement here). Incremental knowledge acquisition: Contexts and their decision trees may be integrated incrementally. Inconsistency check: There is no validity measure implemented (section 1.3.5 for an improvement here).

1.3.5.

S3+/INRECA: Integrating S3 with INRECA

There are three main co-operation modes possible between a CBR system like INRECA and a rule-based system like S3 (fig. 1.23).

CBR systems

CBR component

Rule- or modelbased systems

Rule- or modelbased component

Rule- or modelbased component

CBR component

Fig. 1.23 – Examples for the integration of CBR components into rule- or model-based expert systems, and vice versa

Here it was decided to take the second category, where the CBR component of INRECA is used in an equivalent way as the rule-based component of S3+, which is a – for the purpose of deep integration – improved reimplementation of S3 using the representation language underlying INRECA (Priebisch, 1995). It was decided to implement a control mechanism which can use the two inference methods simultaneously or in concurrence. Thus, one can select another strategy if one of the inference methods fails to provide a user with a satisfying diagnosis (section 6.2.8) or one can control the completeness of the different knowledge structures by comparing the results of each other (section 6.4.3.3). The resulting level of integration is, thus, a deep one. The conceptual models in use are the Inreca-tree (chapter 7) and the context-graph of S3+ with integrated decision trees (section 6.2.8). This combination now offers different possibilities of interactive combinations of the inference processes which were not possible with the earlier integration of the MOLTKE shell and PATDEX (section 1.3.2). The integration of the PATDEX and the MOLTKE shell was considered to provide the possibility to select up to eight different inference modes, with their own test selection, similarity measures and classification component (Koks, 1992; Althoff & Wess, 1991). In the integrated S3+/INRECA system it is possible to select 26

individually during the inference process any combination of any available test selection strategy and classification component. How such an interleaving of different problem solving methods can be realised becomes more obvious when looking at the alternative diagnosis strategies implemented in INRECA and S3+ (fig. 1.24).

INRECA

S3+

initial situation

initial situation

look for most similar case

found evident case

no

test symptom

test symptom

test selection

test selection

propagate until found most probable context new context

install new decision-tree

no

yes no

adaptation as needed

yes

is the context an evident diagnosis yes

diagnosis

diagnosis Fig. 1.24 – Structural diagrams for INRECA and S3+

Both system increase evidence by incrementally testing symptoms and trying to find an appropriate classification to the current situation. The starting point for the integrated control structure are, therefore, the test selection and the classification steps (section 6.2.8). Further different strategies were implemented to validate the completeness and consistency of their knowledge-bases. This was done by the possibility to directly compare results of the different inference processes, or by consistency checks for the context-graph and its integrated decision trees (section 6.4.3.3).

1.3.6.

MoCAS/2: Integrating CBR with Model-Based Diagnosis

Although CBR is a very powerful technology, many CBR approaches still suffer from a major disadvantage: They mainly operate on numbers and symbols without any notion of their meaning. The meaning of the case attributes is implicitly included in the similarity measure in the form of weights, local similarity measures, and other features. A similarity measure that

27

performs well on a complex domain is difficult to build. As it only consists of a set of numerical and symbolic computations, it is hard to understand and, thus, difficult to maintain. A contribution to solve such problems is the approach undertaken in MoCAS/2, the Model-based Case Adaptation System (Pews, 1994) which is described in this section. MoCAS/2 has been derived from MoCAS/1 which is a component of the MOLTKE workbench for technical diagnosis (Pews & Weiler, 1992). Instead of a similarity measure, MoCAS/2 uses a functional model of the domain to identify similar attributes of two cases. This approach has the following advantages: If the domain contains several components of the same type, like e.g. the cylinders of a car, a model-based approach can identify them as similar objects or objects of the same type without problems, whereas a similarity function normally views them as different objects. Achieving the same result with a similarity function requires either a set of specialised cases for each of these components (cylinders) or an adaptation mechanism, which processes the retrieved case. In a system based on similarity function(s), the only possibility to justify a solution consists of the retrieved case together with its similarity score, which is usually a number. A model-based system can justify its result in a more intuitive way by showing effect-paths in the model.

1.3.6.1.

Model Representation

The items of a model in MoCAS/2 are components. Each component is characterised by its ports, its internal structure, and its function. The ports of a component are its interface to the outside world, i.e. the other components. They are represented as slots of a frame structure, which represents the component. Some components may also have internal states. These are represented as a functional feedback of input and output ports. A component of a MoCAS/2 model can be an elementary part or consist of several sub-components which are represented in the same way. The ports of such a compound component are logically connected to the ports of the sub-components. The behaviour of a component is determined by functions which describe the relationships between the ports of a component. These functions are actually implemented as rules. Connections between components are also implemented as rules.

1.3.6.2.

Abstraction and Similarity

MoCAS/2 defines similarity as identity on an abstract level. This definition requires the representation of abstractions. Such abstractions exist for all parts of a model: components, port values, and functions. All components of the model are arranged in a hierarchy of functional abstraction. This hierarchy ranges from the actual parts of the device up to the most abstract concept called „thing“ which is the root of the hierarchy. For example the abstraction levels for a transistor are the following: 28

• • • • •

Transistor of type BC107 Generic transistor Electric switch Generic switch Thing

In this hierarchy, a transistor is more similar to a relay which is also an electric switch than to a valve which is only a generic switch. Each abstract component needs ports whose values are also abstractions of the underlying actual values. These are usually qualitative values rather than quantitative ones. If, e.g., the port values of a transistor are electric voltages and those of a valve are pressures, the values of their common abstraction „generic switch“ can only be described with the values „abstract_on“ and „abstract_off“. Here, the abstraction consists of a mapping from the special values to the abstract ones. Note that it is possible that a component may have more ports than its abstraction. One can map, e.g., a data bus with 32 ports of type „voltage“ on each side to a generic signal conveyor with one port of type „abstract_on“ or „abstract_off“ on each side. The abstraction of functional behaviour is defined similarly. Each function is mapped onto another function which describes the corresponding behaviour of the abstract component. Each of these abstraction steps means a loss of information. On the final level, an object characterised as a „thing“ bears no information at all. On this most abstract level all components are similar to each other. It is important for the model developer to provide as many intermediate levels of abstraction as possible in order to obtain a fine-grained similarity assessment. A completely different kind of abstraction consists of collecting components into groups. The result is a complex object whose behaviour can also be described as an abstract function. Among the many possible ways of selecting groups of components, it is most useful to choose components which are also physically mounted into a complex unit. If a diagnosis can be located only in such a complex component, but not in an elementary one, it is then still possible to replace the whole complex unit.

1.3.6.3.

Case Retrieval and Adaptation

In MoCAS/2, case retrieval and adaptation cannot be separated from each other. In fact, the adaptation is a vital part of the retrieval algorithm. A case is retrieved if it is similar enough that it can be adapted to the query. The algorithm implemented in MoCAS/2 achieves this by carrying out the following five steps:

Step 1: A case is given as two sets of symptoms, i.e. a set of ports together with their values. These sets describe the intended and the observed behaviour of the device. As a case description usually does not contain the values of all possible ports of the machine, the first step consists of a simulation using the model together with the observed symptoms as input in order to compute as many port values as possible. Step 2: Sometimes the diagnosis can be found immediately from the result of the simulation. If the (given or computed) values of the ports of a component are not consistent with the functional 29

description, this means that this component is defective. In this case, MoCAS/2 has found the diagnosis based on the model only without consulting the case base.

Step 3: Before the actual query of the case base MoCAS/2 determines the relevant symptoms. A symptom is relevant if it indicates a failure, i.e. if it shows an unintended behaviour. In order to find these unintended values, the model is run a second time, now with the given intended symptoms as input. The unintended symptoms are those which show different values in the two runs of the model. These unintended symptoms are used for indexing in the following way: In each case in the case-base, every unintended symptom is marked as relevant. In the query case, however, only those unintended symptoms are considered relevant, which are not caused by other unintended symptoms. Thus, the relevant symptoms in the query are those which are closest to the cause of the failure, i.e. the diagnosis. Now the case base is searched for all cases which have relevant symptoms that are similar to those in the query. The following steps use these retrieved cases to narrow down the diagnosis.

Step 4: The retrieved cases are now partially adapted to the current situation. The parts used for the adaptation are the contexts of the relevant symptoms. These contexts are defined as the components along the effect-paths from the diagnosis to the relevant symptoms, along with their behaviour (i.e. port values). For each of these context components, MoCAS/2 tries to find a similar component with similar behaviour which has an effect on the corresponding relevant symptom in the current situation. This adaptation is done backwards from the symptom to the diagnosis. If MoCAS/2 succeeds in finding such a complete chain of components up to the diagnosis of the retrieved case, the component at the beginning of this new chain is considered a hypothesis for the current situation. Step 5: The last step consists of a verification of the hypotheses. This is done by another simulation of the model. This time, the (abnormal) values of the hypothesis are used as start values. Based on this error simulation, there are two possibilities to reject a hypothesis: • The current situation shows an unintended behaviour which cannot be caused by the hypothesis. • The hypothesis would cause an unintended behaviour of some components, but the observed behaviour of these components is correct. The hypotheses that have passed this verification are finally presented to the user as possible solutions. The remaining task consists of verifying these hypotheses against the real-world situation with a series of tests. Currently, MoCAS/2 does not yet supply any test selection strategy. This final verification still must be done manually.

1.3.6.3.1. Justification of a Hypothesis In most case-based system the only possibility to justify a solution consists of the case itself and its similarity score. As the similarity score is only a number, it does not always provide a satisfying justification. In MoCAS/2, a hypothesis can be justified in two different ways:

30

• The result of the error simulation: During the verification of a hypothesis (step 5), MoCAS/2 creates a causal explanation for each unintended symptom, consisting of the chain of components affected by the hypothesis. This explanation justifies why the system has chosen this hypothesis as a possible diagnosis. • The case: The case on which the hypothesis is based refers to a diagnosis which has occurred in the past. The similarity between this case and the current situation is justified by the result of the adaptation (step 4). The components along the adapted effect-path are similar, they have similar effects on one another and their behaviour is similar.

1.3.6.4.

Discussion

MoCAS/2 integrates elements of both case-based and model-based reasoning. Therefore, it also shares advantages and disadvantages of both techniques.

1.3.6.4.1. Comparison with Model-Based Diagnosis The main difference between MoCAS/2 and pure model-based diagnosis is the search strategy. Most model-based systems track the effect-paths in the model backwards in order to find out which component caused the unintended symptoms. If the model reaches a certain complexity, the search space gets unreasonably large. Most systems use heuristics to restrict the search space to a size that can be handled. In contrast, MoCAS/2 does not search the model directly. It chooses its hypotheses from the case-base, using the model to verify them. The complexity of this task is much lower than a search of the model. First, a case base is usually much smaller than the number of possible paths in the model (otherwise the cases would be useless). Second, MoCAS/2 does not generate hypotheses by itself. It rather tests whether the hypotheses given in the cases can be adapted in a way to match the current situation. This improvement in complexity also bears a major weakness of the MoCAS/2 approach. As it cannot generate new hypotheses, it must rely on the completeness of the case-base, i.e. for each possible diagnosis the case base has to contain a case with a similar diagnosis.

1.3.6.4.2. Comparison with CBR From the perspective of CBR, the MoCAS/2 model provides some additional general knowledge which improves the retrieval and adaptation of cases in several ways: • A case retrieved as the nearest neighbour of a query may still be inconsistent with the query in a sense that its diagnosis cannot be the cause of the observed unintended symptoms. As in MoCAS/2 all candidates for hypotheses are verified through a simulation of the model, these cases are eliminated from the set of hypotheses. This step guarantees that MoCAS/2 only generates reasonable hypotheses.

31

• The similarity assessment method used by MoCAS/2 is much stronger than usual numeric similarity functions. This results from the ability to identify physically different parts as long as they are sufficiently similar. This is difficult to realise with similarity functions which normally only compare corresponding attributes. With this strong similarity assessment, MoCAS/2 can work with a significantly smaller case base than a CBR system without general knowledge about the technical device that shall be diagnosed. • MoCAS/2 can give an explanation for each proposed hypothesis. It does not only name a possibly defective component but it also explains how this part could have caused the observed symptoms. This makes it easy for the user to verify a hypothesis by testing symptoms on the effect-path from the hypothetical diagnosis to an unintended symptom. As a system integrating two technologies, MoCAS/2 does not only combine the advantages of these technologies but also some of their disadvantages. As the causal model plays an important part, MoCAS/2 shares some general problems of model-based systems: • A simulation of a model is always a very complex task. The more complex a model is, the more computer performance is needed for the simulation. On the other hand, the model needs a certain complexity in order to describe the domain precisely enough. • Building a model of a domain always requires a lot of knowledge and understanding of the details of the domain. Yet, even if this knowledge is available, it is still a timeconsuming work to enter these data into a model and to design appropriate abstraction hierarchies. These points show clearly that the MoCAS/2 approach is only applicable to domains in which the data for a model are available. If this requirement is satisfied, one will still have to consider the trade-off between the improved quality of the results and the increasing resource consumption (development and run-time) which distinguish MoCAS/2 from CBR systems using less general knowledge. It depends highly on the complexity of the domain and on the expected quality of the classification whether MoCAS/2 is more appropriate than such approaches.

1.3.6.4.3. Conclusion MoCAS/2 combines the experience contained in a case base with the semantic power of a model. Both its performance as well as its drawbacks and restrictions are situated between the properties of pure case-based and model-based reasoning. In comparison with a system like INRECA, the MoCAS/2 approach provides a more powerful retrieval algorithm. However, the field of application of MoCAS/2 is restricted to technical domains for which the knowledge needed to build a model is available, whereas INRECA can be used in all (analytic) domains in which case data have been collected. On the one side, one may classify MoCAS/2 as a tool optimised for a special class of application domains and INRECA as a generic but still very powerful system suitable for many different domains. On the other side, MoCAS/2 could supplement INRECA in technical domains (this is the same idea that was underlying MoCAS/1 for the PATDEX/2 system: Pews & Weiler, 1992)

32

1.3.7.

KoDIAG: Diagnosis with Spatial Neural Maps

Spatial maps are represented as self organising feature maps (Kohonen, 1989) for data clustering and information and knowledge representation. The mapping is topology preserving (Szepesvári, Balázs & Lörincz, 1993). Here they are used to build a system for the purpose of CNC machines' diagnosis as described in the following sections.

1.3.7.1.

The Structure of Explicit Knowledge

The idea is to construct a neural network that is based on the neighbourhood preserving weight adaptation of the Kohonen algorithm (Kohonen, 1989; 1990), but that uses local and global similarity relations instead of the minimal distance criterion to determine the winning cluster (Rahmel, 1995a). The clusters are realised by spatial maps, which directly transform a highly dimensional topologic data space into two dimensional maps. A map for diagnosis is like the following structure (fig. 1.25): Attribute field 1 1 0 0 .. ..

Value field

Diagnosis field

0 1 0 0 1 .. ..

0 1 0 ..

List of attributes Attribute 1

List of diagnoses Value A

Value B

Diagnosis 1

Attribute 2

Diagnosis 2 Value C

Value D

Value E

Attribute 3

Diagnosis 3

. .

. .

Fig. 1.25 – The coding of cases in KoDIAG (adapted from Rahmel, 1995)

For every feature in the case base one bit in the feature vector is reserved. For every attribute a bit in the attribute vector and for every diagnosis a bit in the diagnosis vector is reserved. Like proposed in (Ritter & Kohonen, 1989) the feature and the attribute set where divided up in two different vectors of the form:

⎛x ⎞ ⎛x ⎞ ⎛ 0 ⎞ x = ⎜ F⎟ = ⎜ F ⎟ + ⎜ ⎟ ⎝ xA ⎠ ⎝ 0 ⎠ ⎝ xA ⎠ This was done to be able to differently weigh influences of features and their values on the metric of the distance or similarity measure. Thus, the locations of the symbols of the input data are ordered on the map in a way that should reflect their contextual appearances. This must be done manually when encoding the knowledge-base, like it is supported in CASUEL (Manago, Bergmann et al., 1994). 33

1.3.7.2.

The Implicit Knowledge Modelling

As usual in neural systems, a high amount of implicit knowledge is integrated. This is not necessary for the content, but for the quality of the underlying reasoning process. The implicit knowledge is only used for internal control structures that guide the problem solving and the learning process. n ×n 1.3.7.2.1. The Euclidean Distance Measure in ℜ

An Euclidean distance measure is used to determine best matches of maps to the input. Further it can be used to provide a distance-dependent learning rate for the following Kohonen training. This formula is not valid for map initialisations with low average values (Weis, 1994; 1995b), because it may cause always the same cluster to win independently what kind of input is provided. It is more useful for conventional clustering, because of its direct metric properties. d2 ( X ,Y ) =

1 n2

∑ (x n

i, j =1

ij

− y ij ) 2

1.3.7.2.2. The Neighbourhood Relation Function Different forms of neighbourhood are possible. One possibility is a distance-dependent learning rate like proposed in (Weis, 1994; 1995b) or like in the following KoDIAG-specific example (Rahmel, 1995a+b; Rahmel & von Wangenheim, 1994). A distance-dependent function is necessary: ⎛ || r − c||2 ⎞ h = a exp ⎜ − ⎟ ⎝ σ2 ⎠ h: Learning rate for the Kohonen neighbourhood training. a: A constant which determines the maximum learning rate. r: Radius around the neurone, in which the neighbour neurones should be considered to be replaced. c: Distance of the winning neurone to the neurone to be replaced. σ: A decreasing time function like: t

⎛ σ ⎞ t max σ (t ) = σ init ⎜ end ⎟ ⎝ σ init ⎠ σinit, σend: The initial and final radius of the neighbourhood relation.

1.3.7.2.3. The Kohonen Training Learning Function After presenting the input vector and the determination of the winning neurone the weights may be updated by the following learning function: wr (t+1) = wr (t) + α(t) h ( x(t) - wr (t) ) x(t): Input vector component. 34

w(t): A single (map)-neurone's weight. α(t): A time-dependent learning rate.

This way the strength of the connection between the neurones is always related to their distance.

1.3.7.2.4. Relevance Relevance can be directly interpreted as the weights in the feature and in the attribute map. This is possible because of the binary representation. Thus, one has a local relevance (Weis, 1995b) on two different levels of abstraction. The first is how important is a feature for a case, the second is how important is the attribute for the case. This way there is no need to scale between features with many and features with few attributes.

1.3.7.2.5. Local Similarity In general domains usually also continuous, ordered, and unordered symbolic values need to be determined. This cannot be done by the Euclidean distance function because of the binary representation, which was selected for the KoDIAG approach. For this purpose local similarities were developed that are able to handle these data types. These local similarities can then be integrated into a global similarity like also explained in (Rahmel, 1995a) or by normalisation-summingnormalisation like explained in (Weis, 1995a; 1995b). The main disadvantage (or advantage) is that all the following similarity measures need to be defined manually. Nevertheless, we will call them implicit knowledge (section 5.2). 1.3.7.2.5.1. Local Similarity for Continuous Values

To avoid side effects of scaling the interval it was decided two construct similarity measures with the help of Triangular Fuzzy Numbers and Trapezoidal Fuzzy Numbers. Therefore, one first defines the intervals where the different „Fuzzy-similarities“ are valid (fig. 1.26):

μ

sim1

sim2

simn-1

simn

1

xi Fig. 1.26 – Definition of similarity ranges (Rahmel, 1995a+b)

Now, the similarities simk can be separately defined by hand using Triangular Fuzzy Numbers (fig. 1.27).

35

simk 1

xi Fig. 1.27 – Range dependent similarity

The local similarity Simij for the continuous attributes Ai compared to the corresponding weight of neurone Nj can now be calculated by summing up the contributions of the different ranges: Simij ( x i , w j ) = ∑ μ k ( x i ) simk (x i , w j ) k

1.3.7.2.5.2. Local Similarity for Symbolic Values

Similarity of unordered attributes is realised by a similarity matrix fkl, which contains the lateral similarities of the attributes Aik and Ail of the feature Fi. For ordered values, there is the additional restriction that for |k-l| >1 simi=0 holds.

1.3.7.3.

The Problem Solving and Learning Process

The problem solving algorithm consists of the usual diagnosis cycle as can be seen in the MOLTKE-shell (fig. 1.14) or the PATDEX system (fig. 1.18). The classification component is a specialisation of the usual Kohonen pattern match. This had to be done to build an appropriate interface to case-bases' knowledge descriptions. Further the test selection component provides a simple but effective algorithm to determine relevant unknown information to the system. The classification and learning algorithm works as follows (alg. 1.1): provide(user_data) Competition; WHILE (no_evident_solution) IF(resting_clusters_too_near) THEN Test_selection(discriminating_features) ELSE Test_selection(hypothesis_determining_features) Competition; Kohonen_training(winning_neurone, final_situation) Alg. 1.1 – Classification and learning in KoDIAG

First the user provides relevant input data like error messages and obvious pathologic symptoms, e.g. smoke, non moving parts etc. While there is no evident solution, a test will be selected to 36

provide further information about the proposed hypothesis, which result from the first competition. This is done by counting the weights of the not yet tested features. The feature with the highest weights will be tested. This makes the current solution converge to the hypothesis. If there are several very similar hypotheses one can select discriminating tests. When an appropriate solution is found, Kohonen training is carried out with the winning neurone and its neighbours guiding to the finally resting situation. This learning process may be divided up into two phases, where the first one restricts training to the feature part and the second covers also the attribute weights, where the learning rates can be selected individually.

1.3.7.4.

Conclusions

KoDIAG is an expert system, which is based on spatial maps realised with Kohonen nets on the first level. It has integrated structures to store other data types, namely intervals as well as ordered and unordered symbolic values. The connection strength is always related to the topological representation of the map weights. Maps are not used to filter the data, but directly as the pattern which should be found. Their is an integrated relevance based test selection, which guarantees convergence of the inference process (Rahmel, 1995a). Up to now, if the case base has changed, then all map values need to be recalculated to guarantee a balanced scaling, because their is no strategic learning algorithm yet. This approach has a high complexity of O(#cases*#features). Therefore, it is only convenient for small case-bases.

1.4

Summary

The goal of INRECA is developing and implementing a CBR tool for diagnostic real-life applications. With respect to the CBR methods implemented in INRECA, it can be contrasted to (other approaches to) machine learning, knowledge-based systems, neural networks, information retrieval, pattern recognition etc. As INRECA can be used for diagnostic problem solving, its technologies can be positioned by concrete references to other fully implemented systems in the MOLTKE and INRECA system family. This includes systems for associative rule-based diagnosis, model-based diagnosis, and diagnosis based on Kohonen networks. It also covers integrated approaches for technical diagnosis and service support including inductive, model-based, and case-based reasoning and learning for classification and test selection as well as case retrieval, adaptation, and learning. For the purpose of experimentation INRECA has also been integrated with a reimplementation of tecInno's S3 service support system.

37

CHAPTER 2: SCIENTIFIC GROUNDING OF THE EVALUATION TASK In this chapter we will describe the scientific grounding of the evaluation work presented in this document. We will discuss current views on evaluation and position our approach. We will motivate our set of evaluation criteria and show how evaluation guided the INRECA project. We will also present the basic set of evaluation criteria, which will be detailed and completed within the chapters 3, 5, and 6. The step towards an analytic framework for evaluating and developing CBR systems will be discussed in chapter 9.

2.1

Evaluation of CBR Systems

For an introductory view on evaluation we would like to cite Paul Cohen and Adele Howe (1988a), who wrote an article on how evaluation guides AI research: „Evaluation means making observations of all aspects of one's research. Often, we think of evaluation only in terms of how well AI (in our case: CBR) systems perform, yet it is vital to all stages of research, from early conceptualisation to retrospective analyses of series of programmes. Indeed, some of the most informative observations are not performance measures but, rather, describe why we are doing the research, why our tasks are particularly illustrative, why our views and methods are a step forward, how completely they are implemented by our programmes, how these programmes work, whether their performance is likely to increase or has reached a limit (and why), and what problems we encounter at each stage of our research. To the research community, these observations are more important than performance measures because they tell us how research should proceed.“ By this it seems to be clear that evaluation is necessary. However, how to do evaluation? How could we come up with a methodology for evaluating CBR systems? In Cohen and Howe (1988b) the authors argue that methods known from the, as they say, behavioural sciences are inappropriate for AI in general and, thus, CBR in particular. Cohen and Howe (1988a) argue that we must develop our own evaluation criteria, experiment designs, and analytic tools according to the specific area in which we want to achieve results. Explorations along these lines have been reported for areas like machine learning (Langley, 1987), distributed systems (Decker, Durfee & Lesser, 1988), and knowledge-based systems (Geissman & Schultz, 1988; Gaschnig, Klahr et al., 1983; Rothenberg, Paul et al., 1987). Although these papers address different facets of evaluation, a common theme is that we must develop our own evaluation methods appropriate to the practise in our field (in our case: CBR). This is exactly the approach we took. Cohen and Howe (1988a) wrote their article primarily from the point of view of evaluation of research as such. Nevertheless, we can apply their anal38

ysis and suggestions to our goal of evaluating CBR systems and their development. If we take a deeper look into fields like AI in general or CBR in particular, it soon becomes clear that the development of experimental systems for theory formation and testing is an important issue here: „Systems are good science“ as James Hendler stated in his editorial for the JETAI journal in April 1995 (Hendler, 1995). Experiments and their analysis sometimes answer the questions of the developer, sometimes direct the researcher to other questions and serendipitous results, and point out the scope and limitations of the method (Cohen, 1989). So what we intend to do is to provide specific answers to the following questions about case and knowledge representation, indexing and retrieval, similarity assessment, learning, and other methods in the context of the INRECA project: • In what respect is the method to be evaluated an improvement, an alternative, or a complement to existing technologies? Does it account for more situations, or produce a wider variety of desired behaviour, or is it more efficient in time or space, or does it model human behaviour in a more useful way/detail? • What are the underlying architectural assumptions? Are all design decisions justified? Does the method rely on other integrated methods? Does it subsume other methods? • What is the scope of the method? How extensible is it? Will it scale up? • Why does the method work? Under which conditions would it not work? • What is the relationship to the underlying class of task, of which the current task is an example for? Can this relationship be stated clearly enough to support a claim that the method is general to the class of task?

2.2

Decision Support Criteria for CBR System Development

The issues of system development and evaluation are tightly connected. An implemented software system needs to be evaluated with respect to some (evaluation) criteria to validate whether it meets all the application task requirements at hand (Grogono, Batareh et al., 1991; Hollnagel, 1989; Kirchhoff, 1993). If the application that needs to be developed shall be based on a commercial tool, all candidate tools must be evaluated to select the most appropriate one for developing the application (Rothenberg, 1989; Drenth & Morris, 1992). If a tool is available the question that arises is: How should this tool be used to develop a – maybe – complex application as reasonable as possible. Therefore, all candidate methodologies need to be evaluated to – again – make the right decision (Hilal & Soltan, 1991; Aamodt, 1995).

39

Evaluating one software system Bottom-up criteria

Evaluating different software systems and tools Top-down criteria

Evaluating different software system development methodologies Meta criteria

Domain criteria

Technical criteria

Task criteria

Ergonomic criteria

Knowledge engineering criteria

Fig. 2.1 – Different views on evaluating software systems Evaluation is an important issue for every scientific field and a necessity for an emerging software technology like CBR (Cohen, 1989; Aha, 1994; Althoff, 1995a). However, in different contexts the notion of evaluation is used differently. We would like to suggest an integrative view on CBR system evaluation that integrates several important aspects from currently known views (fig. 2.1). Our experience is based on several successfully finished or still ongoing case studies (Althoff, 1992a; Althoff, Auriol et al., 1995a; Aamodt & Althoff, 1993; Aamodt, 1995, Althoff & Aamodt, 1996a; Althoff & Bartsch-Spörl, 1996). The main idea is to combine different kinds of evaluation criteria: • Domain and application task oriented criteria (e.g., size, theory strength, openness). These criteria are used to describe the requirements that arise from a certain domain and application task and, thus, need to be satisfied if an application problem is to be solved (e.g., Althoff, 1992a; Althoff, 1994a; Aamodt & Althoff, 1993; Slagle & Wick, 1988; Benjamins, 1993; Bartsch-Spörl, 1995a). • Technically and ergonomically oriented criteria (e.g., case and knowledge representation, similarity assessment, user acceptance). These criteria are used to describe the capabilities of CBR systems and, thus, allow the comparison of different systems (e.g., Althoff, Auriol et al., 1995a, Rothenberg, 1989; Drenth & Morris, 1992). • Knowledge engineering criteria (e.g., ease of use of methodology, development phases, tools). These criteria are used to describe the basic characteristics of methodologies and shall help to select an appropriate one (e.g., Hilal & Soltan, 1991; Aamodt, 1995). Methodologies are needed in order to increase the efficiency, reliability, maintainability, and modifiability of systems. Examples are Components of Expertise (Steels, 1990), CommonKads (Wielinga, Van de Velde et al., 1993), Generic Tasks (Chandrasekaran & Johnson, 1993), or 40

Kemras (Hilal & Soltan, 1991). A methodology depends to a high degree on the above mentioned domain/task and technical/ergonomic criteria (e.g., Aamodt & Althoff, 1993; Althoff & Aamodt, 1996a). This led us to our goal of developing CBR systems based on the relationship of domain/task criteria and technical/ergonomic criteria. In practise such knowledge is implicitly used for a long time. More recent work tries to use it more explicitly within CBR system development which results in a system development approach that heavily combines top-down and bottom-up approaches using both high-level structures of generic descriptive components and example instances of existing CBR systems. On the one side existing systems could be described by relating them to high-level characterisations, on the other side high-level characterisations could be made more concrete by relating them to existing systems (Aamodt & Althoff, 1993; Althoff & Aamodt, 1996a). We call the combination of criteria, as presented in this document, a set of systematically chosen decision support criteria for application development based on CBR technology (fig. 2.2). The definition of such a criteria set is an important step towards building a methodology for evaluating CBR systems.

Decision Support Criteria Top-down criteria Ergonomic criteria handling consultation system

Technical criteria

Bottom-up criteria Application domain criteria

limitations

concept possibilities structure

application development

knowledge sources

Application task criteria

knowledge base characteristics general criteria

task properties

decomposition integration of methods reasoning strategies

Fig. 2.2 – Decision support criteria for CBR system development

2.3

How Evaluation Guided the INRECA Project

Evaluation was an important task during the whole INRECA project. We will shortly summarise the purpose of evaluation in INRECA, we will describe how evaluation influenced the project, and how the set of evaluation criteria were developed.

2.3.1.

Purpose

The original aim of the evaluation task in INRECA was: • To achieve deeper „insights“ for the final integrated INRECA system; 41

• To find a reasonable and realistic combination of research and application issues; • To have a more concrete estimation of what can be expected from CBR technology in the near and in the far future. While the first aspect is, of course, INRECA-specific the two other ones are also of more general interest. In addition, there is a fourth aspect that was not originally intended but soon turned out to be of high value and is a major focus of this book: • To get a guideline how to evaluate CBR tools.

2.3.2.

Influence on the INRECA Project

There are four main tasks in the INRECA project that had to be achieved, namely the development of a decision support tool, the development of different applications, the integration of induction and CBR, and the evaluation of current CBR technology (fig. 2.3). We will show that one unifying set of decision support criteria (sections 2.3.3 and 2.4) can be used to evaluate CBR tools (chapter 3) and CBR-related research prototype systems (section 2.4 and chapter 6), to evaluate methods underlying CBR-related systems (sections 5.1 and 5.2), to organise information about domains and applications tasks at hand (chapter 4), to set up the right goals for integration of different methods (seamless integration of induction and CBR, chapter 7), and to guide the development of a CBR tool as such by relating domain and application task criteria with technical and ergonomic criteria (towards an analytic framework for CBR systems and their development, chapter 9).

Domain and Application Task Criteria

Application Development

Decision Support for CBR System Development

Integrated Systems Development

Support Tool Development

Technical and Ergonomic Criteria

Evaluation of Technology

Fig. 2.3 – Decision support criteria for CBR system development

2.3.3.

Development of the Evaluation Criteria Set

In Traphöner, Manago et al. (1992) an initial criteria set („industrial criteria“) was defined. This criteria set was adapted for the use in Althoff, Auriol et al. (1994) to evaluate five different commercial CBR tools that represented the state of CBR technology (fig. 2.4). 42

CBR Express ReMind

INRECA KATE tools

ESTEEM

S3CASE

Fig. 2.4 – Commercial CBR tools influencing INRECA

Based on the feedback of the tool providers and many external reviewers the results of the tool evaluation have been made publicly available (Althoff, Auriol et al., 1995a). The same set of criteria has also been used to evaluate the final integrated INRECA system (chapter 3).

Protos

Creek

Patdex/1

Weiß 93

Awacs

Patdex/2

Moltke

S3

Inreca

Mocas

Casey

D3

CcC+

Evaluation S3+Inreca

Fig. 2.5 – Relations between the analysed CBR-related research prototype systems

In addition, the whole evaluation was extended by also including about a dozen CBR-related research prototype systems (fig. 2.5). The evaluation of the research systems was in many aspects not at the same level of detail (e.g., no running systems were available). But by including them more systems than in the CBR tool evaluation alone could be investigated that – altogether – cover more alternative methods (section 5.1) and representation techniques (section 5.2). The set of evaluation criteria was extended by including a number of domain and application task criteria described in Althoff (1994a). These have been used to characterise explicit methods (section 5.1) as well as implicit knowledge modelling techniques (section 5.2). For a subset of systems from Fig. 2.5, Fig. 2.6 gives an example on the use of such criteria in relation to complete systems. The relationships between methods and domain/task criteria can be found in sections 2.4.3 and 2.4.4.

43

Domain Knowledge

Weak

Strong

Creek Inreca Protos

Patdex-1 Patdex-2 Moltke

Environmental Dependency

Size

Theory Strength

Large

Creek Inreca Moltke

Small

Open

Closed

Patdex-1 Patdex-2 Protos

Creek Inreca Protos

Casey Patdex-1 Patdex-2 Moltke

Fig. 2.6 – Comparing INRECA with research prototypes (domain criteria)

By applying knowledge level analysis onto CBR methods the set of evaluation criteria could be extended again (fig. 6.3). This was done by the introduction of structural criteria which directly correspond to the task-method decomposition hierarchy for CBR developed by Aamodt and Plaza (1994) and represent detailed technical criteria. These structural aspects could be completed by more dynamic control criteria which have been newly developed in Althoff, Wess et al. (1995). Here for every node in the task-method decomposition hierarchy (integrated) algorithms have been introduced which describe a control structure for those methods (from the known CBR-related systems) that can be used at the respective node in an integrative way. The relationship of these algorithms to INRECA is explicitly described.

2.4

Evaluation Criteria for CBR Systems

The criteria are explained along with the following sections. In chapters 5 and 6 we will have a completion of these criteria. In addition, the criteria set, as introduced in this section, will be detailed in chapter 3 for the comparison of INRECA with commercial CBR tools, for which copies of the running system (including documentation) have been available. Please note that tabulated parts will be evaluations,

i.e. the application of the evaluation criteria,

and descriptions written in small font will be the explanations of these criteria.

44

2.4.1.

The Ergonomic Criteria

Ergonomic criteria include those criteria dealing with application development, explainability, modelling support, user acceptance, and the organisational impact on the underlying technology (fig. 2.7). As in Fig. 2.6 for domain criteria, Table 2.1 relates ergonomic criteria to existing systems.

Ergonomic criteria

Application development control management validation automatic

Consultation system

data explainability maintenance data acquisition

accustoming

user acceptance

manual

technological influences organisational influences near future

far future

Fig. 2.7 – Analysing ergonomic issues The control management: Can the user control the complexity of the problem solving process or can he manually integrate the elaboration of additional knowledge representations and how does this affect the quality of the executable system? Validation: Does the development system provide any automatic or manual facilities to test and validate the produced system with respect to input data and additional test data? Data acquisition: How does the system cope with frequent updates of the explicit knowledge? It does not mean that the user must cope with frequent updates, but how. Do not confuse it with the following. Data maintenance: Is there any automatic optimisation task for the knowledge-base? It does not mean prompting for knowledge to be inserted to improve the system, but automatically adapting implicit knowledge. Explainability: Can the system provide justifications or explanations for its questions, decisions and conclusions? How well does the system support domain modelling? This does not mean to provide always the same text to the user. Accustoming: Is the system well documented? Is there also an on-line-help or a demonstration? What simplifies or complicates system usages? User acceptance: Let us summarise some basic features from the user's interface point of view:

•

Is it possible to configure the system interface according to the user's current role, qualification, and needs?

•

Does the consultation follow a logical flow, expected by the domain expert from this field of expertise, i.e. is the system understandable for him?

•

Does the system improve the user's motivation?

•

Does the user interface support the use of additional multimedia (graphics, sound, hypertext, video)?

Organisational and technological impact: Is training of the user or even of the domain expert necessary and what happens if one of them is ill, on vacancy, or not available anymore?

45

Existing architecture1 : PATDEX/1 PATDEX/2 INRECA CASEY MOLTKE PROTOS CREEK CcC+

Time criticality critical less critical

• • • •

• • • •

Interactiveness Additional tasks interactive less interac- training documentative tion

• • • • •

• • •

Rapid prototyping low-cost high-cost

• • •

•

• • • 2

•

• • •

Table 2.1 – Relating ergonomic issues to existing systems Time criticality: Whether the execution time becomes a bottle-neck with ongoing knowledge acquisition. This is certainly more relevant for the retrieve task than for the other ones. The time factor may become more or less critical. Interactiveness: Whether the user can select his own executable methods. It may not be compared with the task internal guidance of the problem solving process. A system may be more or less interactive. Rapid prototyping: Whether it is possible to try out the system in a short time. Certainly, this is not possible if a large amount of input data is necessary to make the system work. Therefore, it may be a low- or a high-cost prototyping. Additional tasks: Like an introduction to the knowledge-bases, an on-line documentation of the programme, a functional demonstration, or a learning program. They may be for the purpose of training and documentation.

So far a description of the ergonomic criteria that reflect up to which degree the user will like working with his/her system. What comes next, even more important for the purpose of this document, is characterising what a CBR system can achieve technically.

2.4.2.

The Technical Criteria

The technical criteria are the ones characterising the possibilities of a system in terms of its limitations. Like describing a car by its max speed, acceleration, and gas consumption there are also technical criteria for a accurate description of CBR systems, namely (fig. 2.8):

1 2

For detailed information about system properties see for PATDEX/1,/2 (Stadler & Wess 1989; section 1.3.1.5, Wess, 1990), for CASEY (Koton, 1989), for MOLTKE (section 1.3.1; Althoff, 1992a), for PROTOS (Bareiss, 1989), for CcC+ (Goos, 1995). CASEY was not developed for universal use, but for diagnosis of heart diseases.

46

Technical criteria Executable system

Development system

Case representation description organisation retrieval

consistency

noisy data

correctness

incompleteness

similarity reuse revision retention general knowledge

completeness

flexibility performance

effectiveness

adaptivity

Fig. 2.8 – Analysing technical issues

In principle, the criteria for the executable system are the same as the criteria for the development system. However, that does not mean that one gets the same results. The characterisation criteria may be evaluated in the following way: Noisy data: The ability to handle noisy data during the consultation. If there is no need for an exact simulation, one may change a certain amount of attributes. This may be done by randomly selecting certain attributes and by setting their values to „unknown“. Flexibility: How many different consultation modes are possible. The following modes were found in the systems of which we take the examples:

Existing architecture: System-guided User/system-guided User-guided

PATDEX/1,/2

INRECA

•

•

CASEY

•

MOLTKE

PROTOS

•

•

CREEK

•

CcC+

•

Table 2.2 – Different consultation modes

•

System-guided: The system does the inference process totally automatic and requests only additional information. It does not mean that it may work totally unsupervised.

•

User/system-guided: This means that the user provides the system with the information he has, the system may infer on its own or quasi-automatically, but may also request additional information.

•

User-guided: All the system does is determined by the user, i.e., the system inquires user input for almost every step it does.

However, this is not the only important point. There are also other questions to be answered, simply with yes or no1: •

Can the user revise information provided earlier in the session?

•

Is the system able to use different strategies of problem solving?

•

Can the system confirm or deny solutions, if asked for?

•

Does the system handle a refutation provided by the user?

1

These are mainly criteria, which are more related to the consultation shell than to the inference mechanisms. Therefore, they will not be evaluated at this place. Nevertheless, they should be considered important for the flexibility of the system.

47

•

How many attributes can be selected at once (for user driven consultation)?

•

Who is in charge of the flow of control?

Performance: Mainly the retrieval time to select a certain number of best cases and the time to build an application in the development system, but also the amount of main memory and disk space necessary to make the system work efficiently. Scaling validity: Is there a method to improve the retrieval quality? This is, e.g., the case when adding new cases improves the quality of the elaborated problem solutions. Correctness: If one takes a copy of a case as query, the system should find the corresponding case as best match. This may be expressed in a percentage of success for different cases, or in a number of cases to be retrieved to find that special case. There may also a test of the similarity measure be included. Note that it is important, if the tool can select the order of questions for narrowing the right diagnosis. Completeness: Can the system detect that a query cannot or probably cannot be classified? This may be done by using knock-out attributes, thresholds, failure reminders, inductive classification, and/or prototypic cases.

Existing architecture: Knock-out attributes Thresholds Failure reminders Induction Prototypic cases

PATDEX/1,/2

• •

INRECA

• • • •1

CASEY

MOLTKE

• •

•

PROTOS

CREEK

•

• • •

•

CcC+

• •

Table 2.3 – Different modes of abstraction and selection

•

Knock-out attributes: After the evaluation of such an attribute it is known for sure that there is no other case to be examined. This can be implemented by using, e.g., local constraints.

•

Thresholds: If the system cannot reach a certain evidence (or probability) threshold it needs another solution selection method.

•

Failure reminders: A kind of causal reminding: I was here already and I did not solve the problem. It is a link from a failure to a successful solution.

•

Induction: Is there the possibility to create a decision tree?

•

Prototypic cases: Can cases be stored together in prototypic cases if they satisfy certain criteria (see also section 6.5 Retain)?

Consistency: Is the system stable over time? If the case base does not change, this means one gets always the same result for the same input. Is the system sensitive to the order of the input? Effectiveness: Does the development system minimise the amount of internal data? Does the system take use of additional knowledge representations? Is the executable system optimised automatically? Adaptivity: Is there automatic generalisation and purge?

Existing architecture: Generalisation Purge

PATDEX/1,/2

•

INRECA

• •

CASEY

•

MOLTKE

• •

PROTOS

CREEK

•

CcC+

Table 2.4 – Deriving general knowledge and removing/discovering inconsistencies

This completes the enumeration of technical criteria for characterising CBR-related research prototype systems. Domain and application task criteria will be discussed in sections 2.4.3 and 2.4.4. We may excuse ourselves for the incompleteness of the examples presented here, but unfortunately we needed to extract most of the information from papers and theses about the respected systems. Testing out all these systems empirically would have been an amount of work 1

Here a prototype denotes a bucket where clusters or classes can be stored.

48

that lies beyond the scope of this evaluation task (this was carried out for the INRECA final integrated system in relation to commercial CBR tools: chapter 3). However, there will be a lot of additional examples for characterisation criteria for CBR-related research prototype systems in the next sections.

2.4.3.

The Integration of the User Domain

The characterisation of case-bases for structure-dependent data elaboration is a very important field in current CBR research. It is necessary, because differently structured domains require different problem solution methods to keep balance of efficiency and effectiveness. Therefore, we would like to present some main characteristics (fig. 2.9), mainly based on works of (Aamodt & Althoff, 1993) and (Althoff, 1994a). Later on we will refer to these different items to select adapted problem solving methods. The main domain characteristics are listed and their intended meaning is explained. These will be applied in sections 5.1 and 5.2. Openness by means of environmental dependencies. A domain may be an open or a closed domain. An open domain has got a lot of environmental outer domain dependencies, a closed domain has got none or at least a neglectable number.

•

E.g., a closed domain would be fault diagnosis of a CNC machine. This has plugs for energy supply and keyboards as the sole relevant interactional structures. An especially open domain is medical diagnosis, because a desease does not „respect“ any boarders or limitations.

Change over time: Domains may be static or dynamic, depending on inter conceptual changes. It is called static if there are no expected changes and it is called dynamic if it is clear that changes will appear subsequently.

•

E.g., a static domain would be the classification of animals if one neglects evolution and no new species are expected to be found anymore. Paying respect to evolution or that research has not finished yet, one has got a dynamic domain.

Complexity means the number of different taxonomic relations (Weis, 1995b). A domain is called complex if there is at least a two-dimensional taxonomic hierarchy and it is called less-complex if there is at most a one-dimensional one.

•

For instance, domains of which causal relations may be abstracted to a tree with only one hierarchical order are less complex, if one needs complex object links like in databases they are considered complex. Medical analysis in itself is always complex. A technical diagnosis environment, however, may be abstracted to a less complex domain without losing effectiveness.

Theory strength by means of the certainty of relationships. If the theory is strong it means that all relations are certain. If relationships are based on probabilities or heuristics we will call it weak.

•

When relying on heuristics or probabilities one has a weak theory, when using formal predicate logic one has a strong theory.

The Size of the domain is characterised by the number of different items representing the explicit knowledge. There may be the following typical contents (here the first two bullets mention the more dynamic instance-level part of the size notion, while the other ones refer to the more static conceptualization part): •

The number of cases.

•

The percentage of unknown attribute values.

•

The number of classes.

•

The number of symbolic, ordered, numeric, non discriminating, and (non discriminating) target attributes.

•

The average of different possible values per attribute.

A domain would be called large with e.g. more than 1000 cases, more than 100 classes, or if the product of the number of attributes with the average of values per attribute is greater than 1000. If not, it may be called a small domain.

49

General domain criteria

Complexity Theory Openness Changes

complex

open static strong lessweak closed dynamic complex

Size concepts, classes, relations, attributes, values

Fig. 2.9 – Analysing the application domain

There are also other possibilities to characterise outer computer system domains, but they are considered to have no influence on internal data elaboration (at least on the level of abstraction presented in this document). There may be e.g.: Was an expert involved in building the case base? Is it a natural or an artificial domain? Is it a simple domain? These are hints for the responsible person structuring the case base from the domain or about effectiveness of given data.

2.4.4.

The Knowledge base Characteristics

Let us take a look on how we can characterise the stored knowledge, which should be used by the system. In the following sections we explain some regularities of the knowledge sources and the underlying conceptual structures. The remaining aspects must, at least up to now, be described by words. Therefore and for examples, see section 5.1 on explicit knowledge modelling, for the task characteristics see chapter 6 on the task-method decomposition model.

2.4.4.1.

The Explicit Knowledge Sources

As every computer system, a knowledge-based system requires to get its knowledge in an exactly specified format and way. This depends on the input structures and certainly on the kind of information which may be elaborated by the algorithms. If someone needs to use a CBR system, a certain amount of knowledge to be implanted into the knowledge base of the future system will be readily available for integration. One of our aims was to find out, which kind of data, information, and knowledge can be well elaborated by which algorithm or system. We decided to take three levels of integration possibilities: •

easy: structures for interaction are already implemented and easy to use.

•

difficult: structures for interaction may be implemented or are already implemented, but a knowledge elaboration process done by hand is particularly recommended to use them.

•

no, none: we saw no possibility to integrate them efficiently.

50

For this purpose the following criteria have been elaborated by Althoff (1994a) to characterise certain knowledge sources and their possibilities of integration into knowledge-based systems. Expert knowledge: The representations used by experts. There may be books, verbal explanations, drawings, formulas, or rules. Machine plans: The representations used to plan and build the machine like circuit diagrams, wiring, models, process diagrams, or topological diagrams. Service reports: These are reports written by external colleagues or the repair-shop. There the parts are documented that have been examined together with their causal relation to the case and the results. Other knowledge-based systems: If data from other knowledge-based systems can be „loaded and launched“ like in a hybrid shell or using specified file formats like CASUEL (Manago, Bergmann et al., 1994). This does not mean that the same type of knowledge and information is used but that it can be directly transferred. Databases: If data can be taken directly from standard databases. There must be the possibility to access the data by handling the complex object types. It implies that it is implemented in a programming environment, which has got the possibility to handle standard databases.

2.4.4.2.

The Concept Structure

When looking on certain relations of our domain to be implemented in knowledge-based systems, it is important to consider in which form the „crude“ data are available. Is it possible to support structures, like e.g. a nearest-neighbour-link1 (see also section 6.5.2.2 Integrating nearest neighbour links) between cases or, which is even more fundamental, a case notion as it is naturally given? It is also very important to be sure about how to handle updates and how to test, whether the knowledge base is valid after certain changes. For this purpose Althoff (1994a) describes the following criteria for characterising concept structures and knowledge base updates: Neighbourhood relation: Is there a causal relation available, because of which objects stay near to each other (e.g., because they like each other or because they are physically or biologically connected to each other, or because they stay near to each other in a topological relation). This does not mean that there is a geometrical interpretation that links them together like nearest neighbour links, but that there may be a problem-specific correlation between next neighbours, that would favour a link like that for integrated CBR methodologies. Case notion: Is it possible to describe a problem solution in the standard case notion. A likely good formal environment is to be found in (Althoff, 1992a; Wess, 1995). If not, cases need to be constructed to benefit from CBR methods for problem solving (Johnston, 1994). E.g., cases exist for medical or juridical purposes. In other domains one must decide about the pathologic attributes for the items to be classified. E.g.: Which features may be useful to effectively describe a flower2? Updates: One reasonable but simplifying abstraction is to differentiate between two main kinds of updates, namely regular & shallow ones and seldom & deep ones. Validity measure: Is there a possibility to „measure“ validity of the knowledge structure? This includes, e.g., finding contradictions and quality of clustering. For the validity measures mentioned in this document see also section 6.4.3 Validity measures to detect faults in the knowledge-base.

1

2

This are precalculated similarity relationships (see also section 5.2 The implicit knowledge modelling). For the solution refinement a hillclimbing algorithm (Goos, 1995) may be used. This is more related to the system's internal concept structure than to the real world representation. You should notice that the „flower-domain“ is a very wide spread domain, which contains also plastic or silk flowers' replicates. Therefore, your first question about this domain should be: „What kind of flowers?“. This is how to come up with a specialised context, for which it is easier to define the relevant case notion.

51

2.5

Summary

The evaluation approach used in INRECA has been scientifically grounded. It is based on a unifying set of evaluation criteria, called decision support criteria, rather than on performance measures. We applied these criteria to the four main tasks of the INRECA project, namely evaluation of technology, CBR tool development, integration of technologies, and application development. For this purpose technical and ergonomic as well as domain and application task criteria have been combined. The latter criteria are are used for characterizing problem solving methods. The technical criteria are detailed based on a task-method decomposition hierarchy of CBR. Altogether INRECA has been compared to five commercial CBR tools and twenty CBR-related research prototype systems.

52

CHAPTER 3: COMPARING INRECA WITH INDUSTRIAL CBR TOOLS We evaluated the final integrated INRECA system according to the same criteria and identical data sets and distributions as were used for the review of industrial CBR tools (Althoff, Auriol et al., 1995a). Thus, the reported results of INRECA correspond to the actual INRECA implementation while the results of the other five industrial CBR tools correspond to the time period in 1994 where the CBR tool evaluation was carried out (Althoff, Auriol et al., 1994). By this we can see what we gained for the final integrated INRECA system by the CBR tool evaluation. Looking at the test results, it may strike that the results concerning S3-CASE and INRECA are very similar. This is due to the fact that both tools emerge from the same system family and have a common „ancestor“, namely the PATDEX-KD system (see fig. 1.7), a successor of the PATDEX/2 system based on k-d trees. During the separate development of S3-CASE and INRECA, the focus lay on different aspects. As a commercial system, S3-CASE was mainly optimised in terms of user interaction and performance, which had its effect on the results in the performance tests (T3 and T9). On the other hand, the development of INRECA as a research prototype was focused on the integration of additional features to improve the system functionality. This can be observed in most criteria tables. The following sections in this chapter are based on Althoff, Auriol et al. (1995a) (chapter 7-10). The evaluation is extended with the respective information on the INRECA final integrated system. We decided to keep those descriptions and information of Althoff, Auriol et al. (1995a) that we felt to be necessary to understand the evaluation of INRECA. All other information has been removed. This includes the test named „Voluntary exercise on similarity measures“ because it brought no additional information for INRECA if compared to S3-CASE. Due to the capabilities of the INRECA system we introduced several new characteristics in the respective evaluation tables. All values for CBR EXPRESS, ESTEEM, KATE 3.0, REMIND, and S3-CASE have been kept as published in Althoff, Auriol et al. (1995a). To increase the understandability of this CBR tool evaluation we added many detailed explanations of the used evaluation criteria.

3.1

The Evaluation Framework

In this section, we present the way we have evaluated the CBR tools, describe the criteria and the application domains. The evaluation was structured around two types of criteria — technical and ergonomic. Section 3.2 presents the results of the evaluation based on the technical criteria, and section 3.3 the results based on the ergonomic criteria. 53

To help assess the tools against the criteria, we used ten tests based on data from four domains. Of course, it was not possible to define a test for every criterion. As in Althoff, Auriol et al. (1995a), all the experiments were completely conducted at the University of Kaiserslautern and the results prepared by the evaluation team: Klaus-Dieter Althoff, Harald Holz, Alexandre Meissonnier, Carsten Priebisch, Karl-Heinz Weis jun. and Wolfgang Wilke.

3.1.1.

Presentation of the Evaluation Criteria

There were two main reasons for undertaking an evaluation of software tools. The first reason is to identify problem areas and poor functionalities that require improvement: This is of interest to the tool developers and designers. The second is to test for competence and meaningful results in a particular domain: This is of interest to end users. Thus, some basic principles should be defined in order to avoid pitfalls of an inappropriate evaluation. These principles are (Gaschnig, Klahr et al., 1983): 1. Complex objects or processes cannot be evaluated by a single criterion or number; 2. The larger the number of distinct criteria evaluated or measurements taken, the more information is available on which to base an overall estimation; 3. People will disagree about the relative significance of various criteria according to their respective interests; 4. Anything can be measured experimentally as long as it is exactly defined how to take these measurements. On the basis of these principles, we first developed a framework for expressing a wide range of evaluation criteria that cover most of the theoretical as well as practical aspects of CBR tools (principles 1 and 2). Secondly, we defined a number of experiments (tests) to be run on the tools, so that clear-cut measurements could be used to compare the systems (principle 4). We did not provide overall numerical gradings (principle 3). Instead, we preferred to present, for each criterion, a symbolic scale of which the values are limited to seven possibilities, so that the reader can see at a glance which system seems to perform better on a specific criterion. We used the following architectural framework to structure our collection of criteria:

54

New Case

Development System

Execution System

Library of Reference Cases Output of Consultation

C o n s u l t a t i o n

Application Development

Fig 3.1 – The architectural framework

From this framework we came up with the following basic terminology (table 3.1): Development System This term denotes the functional component for the construction and maintenance of the executable system Application Development The process of using the development system Executable System The application system created using the development system Consultation The process of using the executable system Table 3.1 – Important notions used in the evaluation of CBR tools

Note that even if the executable and the development systems are one and the same in a particular implementation, we use this distinction at the logical level. Two major categories of criteria are presented: The technical criteria that aim at underlining the possibilities and the limitations of a tool, and the ergonomic criteria that deal with how easily these possibilities can be exploited by different kinds of user. Therefore, the technical criteria are mainly related to the executable system, the development system, and the case representation, whereas the ergonomic criteria apply to the control of the consultation system and to the application development. For convenience, we structured the criteria as follows (see tables 3.2 and 3.3).

55

Technical criteria (section 3.2)

Case representation (3.2.1)

Executable system (3.2.2)

Development system (3.2.3)

Describing cases and case Noisy and incomplete data in Noisy and incomplete data in library (3.2.1.1, 3.2.1.2) the database (3.2.2.1, 3.2.2.2) the database (3.2.3.1, 3.2.3.2) Test T0 Tests T1, T2 Tests T7, T8 Assessing similarity Flexibility (3.2.2.3) (3.2.1.3) Retrieval, reuse, revise, and Performance: speed, memory Performance: speed, memory retain (3.2.1.4) (3.2.2.4) (3.2.3.3) Test T3 Test T9 Representation and use of Correctness, completeness, Consistency, effectiveness, general knowledge (3.2.1.5) consistency, effectiveness adaptability (3.2.3.4 -, (3.2.2.5 -, 3.2.2.8) 3.2.3.6) Tests T4, T5, T6 Table 3.2 – Technical criteria

Ergonomic criteria (section 3.3)

Application development (3.3.1)

Consultation system (3.3.2 - 3.3.4)

Control: manual/automatic (3.3.1.1) Validation (3.3.1.2) Data acquisition and maintenance (3.3.1.3)

Explainability (3.3.2) User acceptance (3.3.3) Customisation of the user interface (3.3.3) Organisational and human impacts of the technology (3.3.4)

Table 3.3 – Ergonomic criteria

The above tables stem the sections where the results of the evaluations for each criteria are presented, along with which tests (if any) were used. We follow this criteria categories framework along this chapter, so that for each type of criterion the reader can have an overview of all systems with respect to a particular aspect of utilisation, e.g. noisy data handling, possibilities of data exchange etc. For evaluating the systems, we distinguish three kinds of question: 1. The closed questions (e.g., „What is the quality of the software documentation?“) are answered by a value in the range 0 - 6. A yes/no question is answered by l for „yes“ and a blank for „no“. In general, the higher the value of a closed question, the more positive the result. For instance, giving the value 2 to the question „What is the influence of noisy data on the development system?“ means „noisy data has a rather bad influence on the development system“. We indicate the possible answers each time where it is necessary.

56

If the result of an experiment does not agree with the answer to a closed question (e.g., a bad result on some tests but a positive answer to the corresponding question), a brief footnote provides a justification for the difference. 2. The opened questions (e.g., „What kinds of data types can a system use: symbol, integer etc.?“). 3. The tests provide clear-cut results in terms of percentages or seconds (e.g., the percentage of correct classifications of new cases, or the time needed for case retrieval). However, three tests have qualitative results, namely T0 and T5. This is a result of the contextdependent style of evaluation within these tests. Each criterion is evaluated through a set of questions (table 3.4) and a set of tests. The answers to these questions and tests are based on the four test domains presented below. Therefore, we will have comparison tables similar to table 3.5. CBR EXEMPLARY CRITERIA TABLE EXPRESS m l m 4 scale m yes/no

ESTEEM

KATE 3.0

REMIND

l

l

l

l l

3

5

etc.

l

l

l

S3-CASE

INRECA

l l

l

l

l

l

Table 3.4 – Exemplary criteria table

EXEMPLARY TEST TABLE

10% 20% 40% 60%

CBR

ESTEEM

KATE 3.0

REMIND

S3-CASE

INRECA

100% 93%

98% 94% etc.

98% 95%

100% 91%

100% 96%

EXPRESS

100% 92%

Table 3.5 – Exemplary experiment table

57

3.1.2.

Presentation of the Test Domains

3.1.2.1.

CAR Domain

The CAR domain was created by Jeffrey C. Schlimmer under the original title „1985 Auto Imports Database“ in 19871. Most of the data of the CAR domain was taken from insurance collision reports. We chose this domain for three reasons: • It is freely available; • It can be understood intuitively; • It contains a balanced mixture of symbolic and numerical values. Most of the attributes in the data set characterise cars in various aspects as size, type of engine, and style. Additionally, the set contains an attribute called „symboling“ to hold the assigned insurance risk rating. This rating corresponds to the degree to which the car is more risky than its price indicates. Lastly, every case contains information about normalised losses meaning the relative average loss payment per insured vehicle year. Several of the attributes could serve as the target and we decided to take symboling. To be able to interpret the results of the tests correctly, we need to examine the following figure that shows the distribution of values for the slot „symboling“: 70 60 50 40 30 20 10 0

-3

-2

-1

0

1

2

3

Attribute Value

Fig. 3.2 – Case distribution for values of symboling

The distribution shows that it is easy to make a good guess if concentrating on the values 0 and 1. Besides the target attribute, the two attributes „make“ and „number-of-cylinders“ are not supposed to be discriminant, which means that they should be skipped during similarity assessment.

1

It originates from UCI repository of Machine Learning Databases and Domain Theories: ftp ics.uci.ed, path pub/machinelearning-database/imports-85

58

The CAR domain was used to evaluate the criteria of handling noisy and incomplete data in the database, using tests T1, T2, T7, and T8.

3.1.2.2.

CNC Machine Tool Diagnosis

The CNC domain (Althoff, Faupel et al., 1989) is difficult to handle because of the large number of unknown values. In addition, the overall number of attributes and classes is very high. The experience we gained on the CNC Machine domain with the PATDEX system was the main reason for using it. In fact, due to our deep understanding of the CNC domain, we were able to dynamically change sub-parts of the respective experiments, as they were originally defined. This allowed us to focus these experiments much more on the CBR tools at hand, resulting in more helpful statements on similarity assessment (for these tools). The CNC domain was used to evaluate the criteria of assessing similarity, using test T5.

3.1.2.3.

Marine Sponges

This database was derived from an application developed at the French Museum of National History where the task was to help biologists in classifying marine sponges. There are nine thousands of different species of sponges described in three classes (Calcarea, Hexactinellida, Demospongiae). The problem is: How to represent the expert knowledge? It seems to be necessary to build a database that represents the natural variety and richness of example sponges. That means that all the expert knowledge has to be expressed via the description language. For example, relationships between sub-parts of sponges, uncertainty on measures, taxonomy on some attributes, etc. The MARINE SPONGES domain was used to evaluate the criteria concerned with describing cases, using test T0.

3.1.2.4.

Travel Agency

This domain was first introduced by Mario Lenz who developed the CABATA1 system. The data used in the system was taken from ordinary travel brochures. Mario Lenz added domain knowledge in different forms of representation to make the CABATA system run successfully. We stripped out the additional knowledge to make sure that each tool could handle the domain. This did not constitute a restriction since we used the domain purely for performance testing. The domain has two advantages for these specific tests. Firstly, the number of cases is fairly high and the success of a retrieval step can be judged without any deep expert knowledge.

1

CABATA = CAse BAsed Travel Agency.

59

Each case in the TRAVEL AGENCY domain contains nine attributes: comfort, duration, holiday type, number of persons, region, price, month, transportation, and hotel name. We added a casenumber to make a comparison of different retrieval results possible. This additional attribute and the hotel name are not used for retrieval. The TRAVEL AGENCY domain was used to evaluate the criteria concerned with performance — speed of case retrieval and index construction — using tests T3 and T9. It was also used for the correctness and consistency criteria using tests T4 and T6.

3.1.2.5.

Summary of Test Domains

# of cases # of classes # of attributes # numeric attributes # values per attribute % unknown values can the data be represented as flat tables? background knowledge specific features

SPONGE 121 low 35 10 few low no

TRAVEL 1470 none 9 3 many none yes

CNC 311 high 80 0 few high yes

CAR 205 low 26 15 few low yes

yes

yes

no

no

complex case structures

many cases

many unknown attribute values

public domain

Table 3.6 – Overview of the four test domains

3.2

Evaluation Based on Technical Criteria

The first section of this section covers the criteria that deal with the underlying case and knowledge representation of each of the CBR tools. The second and third sections cover the criteria that relate to the executable system and the development system, respectively. Under some of the criteria, some of the evaluators have chosen to include detailed comments relating to the tool(s) they have evaluated.

3.2.1.

Case and Knowledge Representation

In this section, we cover all representational aspects of the CBR tools. We start with the representation of cases and their organisation within the case-base, then go on to the assessment of similarity between cases, followed by the reuse, revision, and retention of cases. Finally, we cover the representation and use of general knowledge.

60

3.2.1.1.

Describing Cases

In this section we concentrate on the expressiveness of the case representation language. We distinguish between available and definable attribute types, the former referring to predefined data types offered by the tool, the latter referring to user-definable, user-refinable, or composed types. Whereas an attribute of type Case contains a reference to another case in a case library, an attribute of type Object Hierarchy supports modelling of structured domains that require inheritance, such as the MARINE SPONGES domain.

3.2.1.1.1. Representation of Structured Domains

TEST T0: Ability to handle structured domains

As a first test, we determined to which extent the CBR tools were able to model the MARINE SPONGE domain. This task is not trivial because of the complexity of the domain. Firstly, sponges are not represented as flat structures but as nested objects. Secondly, this domain needs attributes that can store one or more values. In particular, most of these multi-valued attributes contain complex objects, so that they cannot be represented as lists of primitive types. For dealing with these problems, the conclusion is that a fully structured, object-oriented language is necessary, allowing hierarchical descriptions involving complex sub-objects and relationships between these sub-objects. The only systems that support such a description language are KATE, S3-CASE, and INRECA, since they support the CASUEL object-oriented language. The other systems do not support object hierarchies and relations (also table 3.8). CBR

ESTEEM

KATE 3.0

REMIND

S3-CASE

INRECA

•

•

6

6

EXPRESS

o

Can the sponge domain be represented? yes/no • o Is it possible to use complex data represented as objects? 21 6 scale

12

Table 3.7 – Results of Test T0

3.2.1.1.2. General Expressiveness of the Tools Table 3.8 concentrates on the general expressiveness of the tools. Pre-defined or definable attribute types are listed, together with the multi-media capabilities. Beyond the „classical“ data types like Boolean, integer, real, symbol, and string, this list includes special symbol types with a linear or hierarchical ordering (ordered symbol and symbol taxonomy) and complex types like

1 2

„Nested Cases“ implemented as pointers to other cases. This nesting mechanism is limited to 5 levels. The only local similarity measure for slots of type „Case“ is a check for identical case ID‘s.

61

date and time or lists. Object-oriented features can be represented either as pointers to other cases or as „has-part“ or „is-a“ relations between objects in an object hierarchy. „Conjunctions“ and „Disjunctions“ are special cases of multi-valued attributes. These attributes are automatically treated as logical conjunctions or disjunctions by the tools supporting this feature. Additional information means data which is not used for the CBR process, but as a documentation of the current case or case base. ESTEEM

CBR

KATE 3.0

REMIND

S3-CASE

INRECA

• • • • • • • • • • •

• • •

• • • •

• • • • • • • • •

• •

• •

•3 • • • • • •

• • • • • • •

• • •

• • •

EXPRESS

o Which attribute types are available? Boolean • • • Integer • Real • • • Date Time String • • • Symbol • • • Ordered Symbol • Symbol Taxonomy • Case1 • List • Relations • o Which attribute types are definable? Sub-range • • • Conjunction Disjunction Enumeration • Interval • Multi-Value • Object Hierarchy • o Which media can be used for additional information? graphics • • sound • • video • •

•2

•4 •

Table 3.8 – Case representation I

3.2.1.1.3. Other Specific Features Table 3.9 covers some specific features that appear to be important. In particular, the handling of unknown values is critical for CBR tools, since generally only a partial description of the new

1 2 3 4

Contained in „object hierarchy“(see next question) Subranges are supported for Date, Time, Real and Integer. It is possible to restrict the allowed values by an enumeration or a range specification for all types. Enumerations of symbols only.

62

case is given to the system. We distinguish between missing values and unknown values. The first are values that are left out in the case-base, since they are not relevant for this case (do not care attributes), while the latter denote missing information in the query. The tools can support these features explicitly with built-in functions or handle them implicitly by using other functions to build these features. The use of uncertain values can be handled in two different ways: Indirectly it is possible to define taxonomies to represent a partial unknown for an attribute. For example, if you do not know the exact colour of an object, you can define dark colour as a value for different dark colours like black, grey, brown etc. KATE, REMIND, S3-CASE, and INRECA share this possibility. The second possibility is to define the colour as a multivalued attribute with a restricted cardinality. This facility is supported in KATE, S3-CASE, and INRECA. Using default values means that a tool can automatically replace missing values by a default. ESTEEM

CBR

KATE 3.0

REMIND

S3-CASE

INRECA

•1

•2

•

•

•

•

•

•

•

•

•

•

EXPRESS

o Is it possible to handle missing values (do not care)?

o

o o o

explicit handling • • 3 implicit handling • Is it possible to handle unknown values (not known)? 4 explicit handling • • implicit handling Is it possible to use uncertain values? yes/no • Is it possible to use negative examples? yes/no Is it possible to use default values? yes/no • Table 3.9 – Case representation II

3.2.1.2.

Organisation of the Case Library

Most CBR tools employ indexing techniques to make the retrieval process more efficient than a simple linear search. We first distinguish between manual and automatic indexing. In a manual mode the user can build or modify an indexing structure manually, e.g. by moving sub-trees of a decision tree, whereas an automatic indexing yields the result of a fixed algorithm. The entry „decision tree“ only refers to entropy-based trees built by ID3-like algorithms. Indexing trees based on statistical approaches as, e.g., the CART algorithm are shown as „cluster trees“. Object1 2 3 4

The user can choose globally between three different methods of handling missing values: 1) perfect match, 2) complete mismatch, 3) ignore the attribute. It is possible to give a zero weight for do-not-care attributes for special cases with rules, such that they have no influence on the computation of the similarity any more. Cases may consist of different numbers of questions. Unknown is a possible slot-value, but a case with unknown values cannot be used for retrieval.

63

oriented case representations can also use the relations between objects as an indexing structure. The indexing schemes used by CBR EXPRESS and ESTEEM are not documented and, thus, cannot be classified. Each tool either uses a fixed default indexing scheme (which in some cases means no indexing, i.e. a linear search) or asks the user which one to use when building a new executable system. CBR EXPRESS uses the first possibility, but also offers the possibility to adjust some parameters which are used as default values. Beyond selecting an indexing technique, some tools give the user the possibility to adjust some indexing parameters or to use additional features which can influence the resulting index. These can, e.g., make use of different kinds of general knowledge such as importance of attributes or relationships among them.

Attributes

Attribute Values

R22K1

Cases

R22K1=switched (Case-1, Case-2)

Relay22K1 R22K1=not switched R22K2

R22K2=switched

Relay22K2

(Case-3, Case-4)

R22K2=not switched SPP460

SPP460=24V

Voltage Pot.pkt.460

(Case-2, Case-4, Case-5, Case-6)

SPP460=others

Fig. 3.3 – Relational indexing in INRECA (and PATDEX)

Beyond the standard nearest neighbour retrieval, some other retrieval techniques are possible. Template retrieval selects only such cases that match the query exactly in all given attributes. Such a database-like feature is useful to maintain larger case-bases. Static inductive retrieval uses fixed inductive knowledge, i.e. a decision tree or a cluster tree. Dynamic inductive retrieval recomputes this structure after each retrieval step and thus can better react to changes in the case base. Relational retrieval1 is based on relational indexing (fig. 3.3), which is a very efficient mechanism for symbolic value ranges.

1

According to Wess (1995), relational retrieval means a case retrieval strategy where the number of controlled cases is reduced by a primary selection, e.g. all cases which contain a certain symptom as relevant are stored together in a certain structure, where they can be found easily. This is similar to the MAC/FAC model (Forbus and Gentner, 1991; fig. 1.2).

64

CBR

ESTEEM

KATE 3.0

REMIND

S3-CASE

INRECA

• •

• •

• • • • •

•

•

•

• • •

•

• • • • •

EXPRESS

o Which indexing schemes are available? manual indexing • automatic indexing • • • decision tree •1 k-d tree cluster tree relational indexing o Is it possible to define a default indexing scheme? yes/no • o It is possible to further influence the indexing? yes/no • o Which retrieval techniques are available? nearest neighbour • • template retrieval static inductive • relational dynamic inductive •

•

Table 3.10 – Organisation of the case library

3.2.1.3.

Assessing Similarity

There are two different approaches to assessing similarity: Exact matching against an abstract description (e.g., automatically extracted knowledge stored as a decision tree), or nearest neighbour matching using a numerical similarity measure. The KATE tool employs the first approach to similarity assessment. CBR EXPRESS, ESTEEM, and S3-CASE allow the adjustment of the similarity assessment methods, but to different degrees. Users of REMIND and INRECA can choose between both approaches. The tools offering nearest neighbour matching allow the assignment of importance for each attribute in the form of weights. Some tools are able to determine a similarity score taking into account only sub-parts of complex cases, called local contexts. Filtering means reducing the whole case library to a conclusion by traversing a decision tree until a conclusion is reached. This can be regarded as exact matching on an abstract level. Static filtering using a fixed decision tree contrasts with dynamic filtering, where reduction takes place incrementally. Preferences allow the explicit ranking of cases using rules.

1

The decision tree in KATE-INDUCTION is static whilst KATE-CBR calculates the tree dynamically.

65

ESTEEM

CBR

KATE 3.0

REMIND

S3-CASE

INRECA

• • •

• • • • • •

• • • • • • •

•

•

• • •

• • •

• • •

• • •

•

•

EXPRESS

o Which similarity assessment techniques are available? nearest neighbour • • local context • assig. of importance • • preferences static filtering • • dynamic filtering • programmable o Can the similarity measure be modified automatically?

•

yes/no o How does the similarity measure handle numeric values? exact match • • linear distance •1 •2 • • programmable o How is the similarity measure determined between symbol values? exact match • • • • user-defined • programmable o How is the similarity measure determined between strings? exact match • • 3 4 built-in matching • • programmable Table 3.11 – Assessing similarity I

In a weighed similarity measure the weights can depend on different aspects of the current situation. Weights can be set for each case separately (case-specific) or for each attribute (attributespecific). A combination of those is the relevance matrix, which stores weights dependent on both the currently examined case (or its class) and each attribute. Tversky-type similarity measures require the ability to give different weights to matching and mismatching values. The handling of unknown values can also be controlled by weights. When an attribute is declared as a knock-out attribute, cases whose value in this attribute does not match are rejected, regardless of the values of other attributes. If this feature is not explicitly present, it can often be simulated by rules or special weights. When a case base does not contain enough cases, it is possible that the most similar case to a query is still not similar enough to provide a good solution. This situation can be avoided by the use of thresholds, which define a minimal similarity score a case needs to be considered similar. In an incremental retrieval process, more thresholds can be used to partition the case base. A fur1 2 3 4

Within a 10%-interval. Within a definable interval. Special string matching (skipping of fill-words, stable against misspelling or changed ordering). Hamming distance can be switched between characters and words, case sensitive or non-sensitive.

66

ther search can then concentrate on the partitions with higher similarity, which increases the retrieval speed, as less cases have to be examined. ESTEEM

CBR

KATE 3.0

REMIND S3-CASE

INRECA

EXPRESS

o What kinds of weights are used? case-specific • attribute-specific • •1 relevance match/mismatch • unknown values global parameters • programmable o Is it possible to use knock-out attributes?

•

•

•2

explicit • • implicit • o Is it possible to use thresholds? yes/no • •3 o Is it possible to incorporate declarative knowledge? 3 scale

•

5

• • • •

• • • • •

•

•

•

•

•

•

4

5

Table 3.12 – Assessing similarity II

3.2.1.4.

Reusing, Revising, and Retaining Cases

As it can be seen in figure 6.2, retaining, revising and reusing cases are valuable features in the CBR cycle. They are necessary in order to make the tool incremental. Note that the incrementallity may be seen as an advantage—for example, taking new entries directly into account for a process control supervisor—or as a disadvantage, for instance when a help-desk system has to be validated before it is delivered on a network. Table 3.13 shows that only a few tools employ techniques for handling these valuable properties. ESTEEM uses rules, REMIND proposes adaptation formulas, and S3-CASE and INRECA an update of their relevance matrix of weights in case of a failure. In addition, INRECA can use adaptation rules and extract rules from the cases that are then to be included as preferences in the indexing tree. S3-CASE and INRECA can temporarily remove all cases that include a rejected diagnosis. INRECA can also temporarily remove all cases that do not meet a set of user-specified constraints.

1 2 3

Can be dynamically computed by the use of rules. Three different matching methods are available: i) perfect match, ii) complete mismatch, iii) ignore the attribute. For matching against prototypes, the handling of missing values can be attribute-specific. ESTEEM always uses a definable (default 50%) threshold to determine, whether two cases are similar enough.

67

CBR

ESTEEM

KATE 3.0

REMIND

S3-CASE

INRECA

EXPRESS

o Does the tool employ techniques to improve the reusing of cases? yes/no • o Does the tool employ techniques to handle case revision?

•

• •

o Does the tool employ techniques to handle case retention (storage of new cases)? yes/no • • • • •

• •

Table 3.13 – Reusing, revising, and retaining cases

3.2.1.5.

Extracting, Representing, and Using General Knowledge

Additional general knowledge is helpful in many situations. For example, it can be used to restrict the search space when looking for the solution of a diagnosis problem. In table 3.14, we give an overview of various types of general knowledge used in the CBR tools. Some of this additional information can be extracted automatically from the case data. These are structures used for indexing, as categories determined by the category utility measure and decision trees, weights for the similarity measure or causal rules. Other further knowledge has to be entered separately by the user. This knowledge can be incorporated into the definition of data types in the form of sub-range definitions, hierarchical or linear orderings on the symbols or object-oriented relations. Another possibility is storing it in separate structures. While most tools can use general knowledge to support the retrieval phase and some to support the reuse phase, INRECA is the only tool that also uses general knowledge to support the revision phase (constraints, relational indexing structure) and the retention phase (decision tree).

68

CBR

ESTEEM

KATE 3.0

REMIND

S3-CASE

INRECA

EXPRESS

o Which extracting techniques are available? categories weights decision trees causal rules o What kind of rules can be used?

•1 • • •

• • • •

causal • o What kind of further knowledge can be used?

•

•

sub-range definitions • • • • symbol taxonomies • • symbol orderings • relations • decision trees • cluster trees • formulas • qualitative models • prototype hierarchies •2 o Where is general knowledge used to improve the CBR process?

• • • •

• • • • • •

•

• • • •

retrieval reuse revision retention

• •

•

• •

Table 3.14 – Extracting, representing, and using general knowledge

3.2.2.

Executable System

Here we focus on the performance of the CBR tools during the consultation phase. We investigate the ability to handle noisy and incomplete data, and consider issues such as flexibility, performance, correctness, completeness, consistency, and effectiveness.

3.2.2.1.

Ability to Handle Noisy Data During Consultation

TEST T1 Systematic introduction of noisy data during consultation

For this test, we used the CAR domain to introduce noise incrementally into the new cases. All 205 cases of the domain were taken as reference 1 2

In the form of clusters indexed by a cluster tree. See figure 5.5 for a further description of q-models and prototypes.

69

cases and used for building an executable system. From the 205 cases, 50 were randomly selected as new cases. Then we introduced noise by randomly selecting a possible value from the respective value range for 10%, 20%, 40%, and 60% of the attributes, respectively. According to the different results of the systems, we computed the percentage of correct classifications for each degree of noise. In this test, as well as in tests T2, T6 and T7, it could happen that the maximum similarity score was reached by several cases. In such a situation, the first case shown may contain an incorrect classification, while the second or third one, still with the same similarity score, is the correct result. So, even if the correct result is among the cases with the highest score, we count this as a bad classification, because we only look at the very first retrieved case. CBR

ESTEEM

KATE 3.0 induction

100% 100% 94% 72%

84% 70% 54% 40%

EXPRESS

10% noise 20% noise 40% noise 60% noise

100% 98% 92% 88%

REMIND nearest neighbour 98% 90% 66% 42%

S3-CASE

INRECA

100% 98% 92% 82%

100% 100% 94% 94%

Table 3.15 – Test T1

3.2.2.2.

Ability to Handle Incomplete Data During Consultation

TEST T2: Systematic reduction of available information within the new case during consultation

One advantage of CBR is the ability to make decisions from incomplete information. The goal of this test was to find out how much information was needed by the different tools to get a correct classification result. The reference case base for this test consisted of all the 205 cases of the CAR domain. From this set, 50 cases were selected randomly to build the new cases. The information in the new cases was then reduced by deleting a given percentage of the attribute values in each case. We did this deletion four times using 10%, 20%, 40%, and 60% as a parameter. The attributes chosen were independent for each run. The resulting new cases were then entered into the CBR tools to retrieve the most similar case for each one. The „symboling“ attribute was used as the classification result and, thus, did not either take part in the similarity assessment. 70

The result of this test is the percentage of correctly classified query cases for each degree of information reduction. All the tools used the same test procedure as in test T1. As the query cases were built from cases that were present in the reference case-base, there was always a case with a maximum similarity score and the correct classification. So, one might expect a classification result of 100% for all tools. However, it was possible that the remaining information in a query case matched several reference cases that could have a different „symboling“ value. As the tools were advised to retrieve only one case, they had to choose one of possibly several cases with a maximum similarity score. In that case, a CBR tool could compute a false classification, which partially explains test results that do not reach 100%. CBR

ESTEEM

KATE 3.0 induction

100% 100% 78% 82%

92% 80% 72% 54%

EXPRESS

10% reduced 20% reduced 40% reduced 60% reduced

100% 100% 100% 100%

REMIND nearest neighbour 100% 100% 100% 98%

S3-CASE

INRECA

100% 100% 100% 98%

100% 100 % 100% 98%

Table 3.16 – Handling incomplete data during consultation (Test T2)

3.2.2.3.

Flexibility of the Executable System

Some tools offer more than one strategy of problem solving. These are shown in the following table. Note that this list does not include the default strategy. As the default strategy is nearest neighbour retrieval in most tools, the additional ones are inductive retrieval with decision trees or cluster trees and dynamic case-filtering, which incrementally reduces the set of considered cases using the entropy measure. In many application cases the user may want to verify the solution proposed by the CBR system. For this task it is desirable that the system is able to justify its result in any way. This makes it possible to confirm or deny the given solution. If the user is not satisfied with the solution, he may ask the system for another one, so that it has to exclude cases with the same classification from the retrieved ones. So far only very few tools support this kind of refutation. In the following different kinds of consultation modes appear: System-driven, system/userdriven, user/system-driven, user-driven. A system-driven consultation completely controls the order of interactions with the user by prompting her/him for answers, in contrast to a user-driven consultation, where the order of attribute value specifications is completely determined by the user. An example of the former is dynamic case-filtering; an example of the latter is nearest neighbour retrieval. System/user-driven is a combination of both consultation modes, while user/system-driven is a restricted user-driven mode. 71

CBR

ESTEEM

KATE 3.0

REMIND

S3-CASE

INRECA

EXPRESS

o Is the user able to correct previous input during interactive consultation? 41 6 6 6 scale o Is the system able to use additional strategies of problem solving?

6

dynamic case-filter• ing decision trees clustering • o Is the system able to confirm or deny a solution when the user asks for that?

6 • • •

yes/no •2 o Does the system handle a refutation of the solution by the user? •3

•4

system-driven • • system/user-driven • • user/system -driven • user-driven • • • o User-driven consultation: How many attributes can be selected at once?

• •

yes/no o Does the system support different modes of consultation?

•

all • • • • some • one • o System-driven consultation: Does the executable system control the flow of the consultation by prompting the user for input? yes/no • • • • Table 3.17 – Flexibility of the executable system

3.2.2.4.

Performance of the Executable System

TEST T3: Retrieval time

The same settings as in tests T1 and T2 have been used.

1 2

3 4

The possibility of correcting previous input is restricted. The questions supplied to the user are collected from the list of currently matching cases while the most promising question is presented first. At the end of an inductive retrieval, REMIND not only returns the set of cases found during cluster tree traversal, but also comes up with an „explanation“ of its result. This explanation consists of an overview about which prototypes applied, which decisions (i.e. tests on attribute values) had been made during tree traversal until it encountered a case cluster, which different outcome values have, thus, been found and the number of their occurrences etc. There is no possibility of denying a solution. The system allows the exclusion of cases to be retrieved with certain classification values. Refutation can be achieved by preventing the same classification being proposed again (in subsequent retrievals during the current consultation). In addition to the ability of S3-CASE, INRECA offers the possibility to specify arbitrary constraints for the cases, i.e. the user can (efficiently) exclude cases that do not fulfill his/her constraints.

72

The TRAVEL AGENCY domain was chosen for this test because it encompasses the highest number of cases. 50 of the 1470 cases were used as new cases. The remaining 1420 cases were partitioned into four sets, two sets of 200 cases, one of 400, and one of 620 cases. By successively merging these partitions, we got reference case-bases of 200, 400, 800, and 1420 cases. The times are measured for retrieving the most similar case and the five and ten most similar cases for a new case. These values were measured on the same hardware with no other programme running on it, except the CBR tool itself and the Windows Program Manager. The values shown in the table below are the average values over the 50 new cases for every combination of the parameters. Since the time was measured manually, the values may include a few tenths of seconds of „reaction time“, which can slightly falsify the results. Our definition of „retrieval time“ used in this context is the time elapsed between triggering the retrieval with a mouse or keyboard action and the displaying of the results on the screen.

73

CBR

ESTEEM1

KATE 3.0

REMIND

S3-CASE

INRECA

7.1

0 ) • sim( Sit , Cs ) = 1 ⇔ ( | E | > 0.∧| W | =| U ∪ N | = 0)

5.2.1.7.

Similarity Derived from Causal Models (CASEY)

The similarity in CASEY is calculated on the basis of the causal states of the hypotheses and the current situation. First the system checks with so-called evidence rules if differences can be corrected. These evidence rules are: • Qualitative similar value: Numerical values are transformed into symbolic ones to be comparable. • Exclusion: Situations of cases may be excluded if there appeared a specified feature value. • Other evidence: If there is an attribute in the current hypothesis, but not in the currently checked case, the system tries to explain this by another attribute. • Additional attribute in the currently checked case: If it is not needed to form explanations it will be ignored. 136

• Explanation by existing situation: If there is an attribute in the current hypothesis and not in the currently checked case then it may be explained by another attribute in the currently checked case. • Additional attribute in the current hypothesis: Is such an attribute value not „normal“ it will be marked „not explainable“ in any way. • Normal attribute: Is in the current hypothesis an additional „normal“ value it will be ignored. • No information: If there is an attribute only in one of the two compared cases and if there is no further explainability it will be considered „not existing“ in the other case. With these rules, the system can filter out not relevant, not explainable, or not fitting information and it can attach corresponding situations. After that the corresponding values are counted without weighing and the number of different attributes is subtracted. That way, the tool finds the cases with the most corresponding attributes that cover the most pathophysiological states. In fact, it finds and infers the number of corresponding attributes and then subtracts the different ones.

5.2.1.8.

Similarity of Explanations (CREEK, PROTOS)

In CREEK (Aamodt, 1991) four different types of explanation tasks are to be found, which try to justify an inferred hypothesis. Explanations are derived by propagating activation from each of the concepts provided, along current explanation relations, to another concept. Explanation relations are provided by a basic set and should be extended and refined application-dependently by an initial knowledge analysis in the model. Here explanation relations should be defined and the user can reweigh them because of coherences. This is the main difference to the PROTOS system, where the default values of the basic set are always used. The following gives an introduction to the different types of explanation extractions, which are to be found there (for integrating extraction section 6.5). • Inferring consequences: What are the consequences of a proposition A? – Spreading activation over relational links (associative or model). – Contradiction detection by constraint violation. – Reading value-justification-field, if set explicitly by definition. • Matching concepts: Is A like B, where are differences and similarities to be found? – Matching of cases and features. – Ιf parts of the explanation chain are too weak, then find additional explanations. Explaining „away“ feature differences, if a feature has a low importance for the hypothesis. Showing that an input feature does not contradict any strongly relevant feature of the case, its solutions and their constraints. Explaining feature matches in the model, here called general domain knowledge network. • Assessing a value: Is A a plausible value for B? Are there constraints on B which may exclude A? 137

– Looking for contradiction by propagating (negative) relations. Derive explanation of the path and check acceptance value to confirm plausibility • Assessing support: Does the existence of A support B? If yes, then how? – Check contradictions and if not found, then search for supportive explanations which are accepted if transgressing the acceptance level. The quality of an explanation is determined by the default strength of each relation in an explanation chain (which may contain multiple paths) and by a set of explanation evaluation rules. The explanation strength of a starting concept is always to be considered 1.0. Unless only relations with default strengths of 1.0 are following, the strength of a single path will be weakened. A concept may be explained by several paths leading to its concept node. These further are to be computed to a total explanation strength for the concept. The strength of an explanation is calculated by taking the strength of the initial concept in the explanation chain and by further propagating explanation strength values to succeeding objects in the chain, in the following way:

Def.: Explanatory strength: • If C1 is a given concept with explanation strength S(C1). If R12 is an explanation relation with explanatory strength S(R12), which points to a concept C2 with an already existing strength S(C2old). It is possible to calculate the new strength of given concept S(C2new) according to the following formula: S(C2new) = S(C2old) + kold • S(C1) • S(R12) • ( 1 - S(C2old) ) • The constant k∈(0,1) determines how the contribution of a new path will be on the old explanatory strength. It is set by the system designer and altered automatically in the following situations: – C2 does not have an old strength. Then k is set to 1.0. – An explanation often determines one or more findings. Then k is set to the ratio of activated findings related to C1, to the total number of findings related to C1. The following example (Aamodt, 1991) will illustrate this: Consider the explanatory strength of the hypothesis low-battery-voltage C1 for explaining the observation starter-motor-not-turning C2. A part of the total explanation structure is a single step path along a causes relation R12. The strength of the assumed hypothesis is by default 1.0. If the existing strength of explanation - which may be expressed as S(starter-motor-not-turningold) - is assumed to be 0.8, the explanatory strength of a causes relation may be 0.9 and k is 0.5, the resulting explanation strength would be: S(starter-motor-not-turningnew) = 0,8+0,5*1,0*0,9*0,2 = 0,89 For evaluating the total quality of an explanation, there are two types of so-called explanation evaluation rules, in dependency on whether they are triggered during or after the search for explanatory paths. The first type uses the basic strength of a path, information about the explanation task and the goal to determine whose relation is the most promising to propagate along. The second type guides the evaluation of complete explanation chains in order to select among competing explanations. These rules should be developed as a part of the application, because generality has not yet been tested out. 138

5.2.1.9.

Similarity of Methods

The degree of similarity should be proportional in some way to the complexity of the reuse of the method. This mainly applies to the similarity of modification traces for reuse of executable methods for case adaptation. For the general reuse of Smalltalk-80 methods see Bergmann, Maurer and Pews (1994). Nevertheless, all relevant information about the different reuse possibilities must be integrated into the similarity measure to provide completeness and consistency of the problem solution related to the method's case base available.

Method index

method name number of parameters: n type, parameter, exchange parameter 1 type, parameter, exchange parameter n reminder to cases reminder to modification cases reminder to methods

Fig. 5.10 – Information to be stored with the methods

In complex domains it can now be inherited for every class the names of methods and the parameter types and values from upper classes in the class taxonomy of the application, which was built in the system in which this method may be integrated. sim =

w type sim type + w instance sim instance w type + w instance

With the weights wi, one can define the balance of the relevance to provide scaling validity system improvement (sections 2.4.4.2 and 6.4.3). The similarity of types must be taken easily from the class-hierarchy of the system where integrated and the instance similarity is a local similarity on the class, where inherited from.

5.2.2.

Relevance Modelling

It is clear, that different symptoms are differently relevant for the finding of different solutions, because of changing pathologic symptoms. Most times this kind of weights is directly prompted by the expert, but during ample knowledge acquisition this may become a badly arranged situation. So integrated implicit relevance modelling became one of the more important parts of CBR research. The following examples also show on which they refer, because they are usually used 139

for every symptom in dependency to a case or diagnosis. This way, one can construct a two-dimensional global matrix, which is easy to use. The following figure 5.11 gives an overview of the already implemented possibilities.

Relevance Conditioned by diagnosis Local heuristic

Global

Global matrices

Initialising Initialising by case data by hand

Weighting all equally

Weighting by model

Heuristic weighting

Initialising by model

Conditioned by heuristic

PATDEX/1 INRECA

CcC+ PATDEX/2

CREEK

CASEY

Fig. 5.11 – Characterisation of relevance

Most of them are explicit approaches, not to be considered here, because they imply a lot of knowledge about how to use a certain system and certainly a lot of work to prompt them. More interesting are the automatic or semiautomatic approaches, working with global matrices, because they are easy to handle and empirically tested to be effective (relevance-based filter nets: Weis, 1994; Derwand, 1994; Wess, 1995; Goos, 1995). They will be explained profoundly throughout this chapter (section 5.2.2.2). The following important properties of relevance must be considered: • Error-messages and pathologic symptoms usually have high relevances. • Relevances of symptoms may be different if conditioned by different diagnoses, current situations, or explicit classes. This means that they can be different also for randomly different situations, but for the same query. E.g., if in a conflict a random generator was used to resolve a dead end. • The more abnormal an attribute is the more relevant it is in general.

140

5.2.2.1.

Determination Factors (PATDEX/1, MOLTKE, CREEK)

The experience graph (Stadler & Wess, 1989; Weis, 1995b; section 6.5) represents all empirical knowledge obtained1. It builds a node for every feature in every new situation. That is why it has got a complexity of O(2n), because different successions of symptoms imply different situations. In the leaf nodes it stores cases. On the edges it stores so-called determination factors. These are estimated probabilities conditioned by a determinated diagnosis. During the problem solving process the algorithm moves from one node to another, by selecting tests conditioned by these determination factors, until it finds the problem solution. The determination factor is a function of the current situation and its more precise situation defined as follows: When Sit1 determines Sit2 with the determination factor δ:

δ ( Sit1 , Sit 2 ) =

|{Sit| Sit 2 ⊆ Sit}| |{ Sit| Sit1 ⊆ Sit}|

In MOLTKE, determination factors are used to determine the validity of a result. In the basis model there are only certain rules, but by using heuristic shortcut rules with determination factors the result becomes uncertain. So, a linear interpolation of all determination factors computes the validity of the inferred result. It may also be used for test selection, like other implemented methods of the MOLTKE workbench.

5.2.2.2.

Neural Relevance Modelling (PATDEX/2, CcC+)

This approach is a pseudo connectionistic approach. It is an abstracted Competition-Hebblearning taken from pattern recognition. It assigns every symptom a specific weight for every possible hypothesis. This weight ωij , respectively the relevance for symptom Si , if statistically proposing the hypothesis Dj , is then stored in a matrix-like structure:

Requirements: ωij∈(0,1) and

D1

D2

…

Dm

S1

ω11

ω12

…

ω1m

S2

ω21

ω22

…

ω2m

Sn

ωn1

ωn2

…

ωnm

∑ω n

i= 1

ij

= 1.

The normalisation of matrix columns is necessary to keep comparability of weights. It has got also the nice property, if used with metric measures, that it always produces results in a certain numeric interval. For initialisation the following formula may be used, which calculates the relative frequency of Symptom Si in all n cases with problem solution Dj .

1

Its original intention was only to deliver the most probable case and to provide determination factors, but it is easy to see that the experience graph is a complete representation of real world expert knowledge (section 6.5.2.4)

141

ω ij =

| S i in D j | n

∑| S j= 1

i

in D j |

It is useful to provide the possibility for the user to explicitly reweigh pathologic symptoms. For updating neural relevance one can use the Hebb-learning algorithms for success and failure (section 6.5.2.5).

5.2.2.3.

Relevance Modelling with Combined Local Intra-Inter Similarity (INRECA)

In an integrated retrieve with k-d trees or retrieval on clusters, one can use similarity measures for the development of class-specific relevance for the symptoms. This local measure works for clusters or classes Mi and attributes or predicates Aj : • Local intra-class similarity: simIntra Mi , A j =

1 | M i |2

∑

c, d ∈ Mi

sim j ( c| Aj , d | A j )

This is a measure how good a certain symptom Aj is clustered in a certain class Mi . • Local inter-class similarity: simInter Mi , A j =

1 ∑ simj ( c| A j , d | Aj ) | M i | | M \ M i | c∈ Mi ,d ∈ M \ Mi

This is a measure how bad this symptom Aj is spread over all M in comparison to Mi . With these similarities, one can develop the class-specific weight used in INRECA: • The combined local intra-inter similarity for classes and attributes: Inter ΔsimMi ,A j = max{0, simIntra Mi , A j − sim Mi , A j } These values may be used equally to the values of relevance matrices in the earlier section or even to build multiple relevance matrices.

5.2.2.4.

Modification Costs of Case Adaptation

These are measured in general, by the number of symptoms to be changed. However, one must consider the number of modification rules to be checked and fired for every symptom or also the complexity of the retrieval-process of modification methods. But this is still the less complicated thing if the consistency of rules must be considered. So it seems to be very necessary, if integrating several rules or symbolic models, to provide also an integrated ad hoc consistency check, here called validity measure (section 6.4). If the rules are consistent that should be valid also for the results. All these relevant costs may now be normalised and multiplied with explicit modification costs. Both can be related to already found solutions, diagnoses, or globally to complete situations. The explicit modification costs, also normalised, are in general oriented on staff and material costs. This multiplier can be used to build an own filter-list, or can be multiplied to al142

ready shown upper relevances and filter-values to be integrated directly into distance or similarity measures.

5.2.2.5.

Conclusions to Relevance

Relevance is considered a high level knowledge approach and that way the kind of relevance used, implies effective system characterisation criteria. We have evaluated these criteria for the systems to be considered in this document in the following way. Certainly, this is only an example and for that purpose it cannot be complete. Existing archi- Feature relevance stored in class-feature matrix tecture PATDEX/1 PATDEX/2 MOLTKE PROTOS CREEK 1 CcC+ INRECA

Feature relevance stored locally with the feature in an index to cases

•

• •

• •

Determination factors conditioned by cases

• • •

Table 5.14 – Different kinds of relevances Feature relevance stored in class-feature matrix: This means that the feature's relevance is stored in a global matrix, where the rows represent features, the columns represent cases or diagnoses. The entries are the relevances of a certain feature for a certain diagnosis (For the structure of the matrix section 5.2.2.2). Feature relevance stored locally with the feature in an index to cases: This means that the feature's relevance is stored totally local for every case. However, it is not stored in the case. It is stored in the feature's pointer structure to that certain case. That is why it is possible to select cases in which a certain feature is important in a semantic network by spreading activation, because one first checks the feature's explanatory strength and if it is above a certain limit, the linked categories are checked. Determination factors for features conditioned by cases: If there is an estimation of a probability calculated, which determines relevant facts (section 5.2.2.1).

5.2.3.

Conclusions to Implicit Knowledge Representations

There is a wide range of implicit knowledge representations available, but their possibilities are limited. It is in the nature of the way that it is not so easy to recognise how implicit knowledge can improve knowledge handling, but this must be clear at least for the programmer. For the user only the work to prompt the explicit knowledge should remain and to select the implicit method with which he thinks that the system works best. Distance and similarity measures can be globalised by summing up and can be normalised by dividing by the number of features. There are two main possibilities to get useful relevances. They may be included into every distance or similarity measure by simply multiplying it to the local similarity or one-dimensional distance.

1

Exceptions may be prompted by the expert to be stored locally in their cases.

143

Existing architecture:

Determination factors Function per diagnosis Function per sub-class Function per attribute range Explanations Dynamic parameters in function Feature relevance stored in class feature matrices3

PATDEX/1

PATDEX/2

•

•1 • • •

INRECA

CASE Y

• • • • •

•

MOLTKE PROTOS CREEK

• •

• •

CcC +

•2 • •

Table 5.15 – Relating types of implicit knowledge to existing systems Determination factors: As described earlier (section 5.2.2.1). Function per diagnosis: There is a similarity measure as described earlier (section 5.2.1). This finds somehow a relevant hypothesis related to the current query which guides the test selection. Function per sub-class: This is similar to „function per diagnosis“. It differs by the property to automatically find sub-classes (disjunctive concepts) that are then treated as the pre-specified classes (diagnoses). Function per attribute range: There is a local similarity measure directly applied to every type a concept slot may have (section 5.2.1.2). Similarities of Explanations: There is a method implemented to validate explanations extracted from a diagnosis path. This directly specifies the evidence of the hypothesis or solution (section 5.2.1.8). Dynamic parameters: This is the case if the weights of feature sets are manually or automatically variable. Note the difference between weights and relevances. Feature relevance stored in class feature matrices: This is a global approach like described earlier (section 5.2.2.2) to store feature relevance conditioned by diagnoses, classes, or other problem solutions.

5.3

Summary

We introduced a number of different explicit and implicit knowledge structures. We used a set of systematically chosen domain and application task criteria to evaluate the explicit knowledge structures on the grainsize of methods. Some additional technical criteria have been introduced to characterise the implicit knowledge modelling techniques. Both kinds of structures form the basis for a task-method decomposition of CBR. They will also provide a new view on task characterisation criteria by abstracted algorithms that base on these introduced knowledge representations. Our analysis is based on a number of CBR-related research prototype systems and commercial tools. Though the number of considered systems is restricted, there are many comparable approaches in the literature. Thus, the achieved results will have some importance beyond the analysed systems.

1 2 3

PATDEX/2 can use shortcut rules of the MOLTKE workbench (see also section 6.5.2.6 Generating implicit shortcut rules). Similarities of different feature values may be promoted explicitly or calculated locally. Namely a relevance matrix.

144

CHAPTER 6: ANALYSING INRECA USING THE TASK-METHOD DECOMPOSITION MODEL FOR CBR We will shortly introduce the task-method decomposition model according to Aamodt and Plaza (1994). The corresponding task-method hierarchy will be used to systematically analyse the CBR-related research prototype systems introduced so far. The task-method decomposition model will be extended by newly developed abstracted algorithms that correspond mainly to the methods and knowledge modelling techniques described in chapter 5. The set of evaluation criteria is extended during the analysis of the underlying CBR systems as appropriate. These criteria are explained and directly applied.

6.1

The Task-Method Decomposition Model

In CBR it was always useful to choose a bottom-up approach for the development of the knowledge representation, because this way it is possible to estimate possible capabilities of the final system. However, for the development of the adapted control-structures, as usual, one takes the top-down approach. For that purpose the task-method decomposition model was developed. It provides the programmer with specified information about tasks and events to elaborate given knowledge for problem solving. The model itself is structured like a tree (fig. 6.3), where the root specifies the main event. It is a dynamic model, because it can be adapted individually. It provides a global view on the sub-processes in a hierarchical task structure (spreading activation: Aamodt, 1994a), about interdependencies and side-effects of each other. It tries to enumerate mechanisms for problem solving methods from the view of the inference module. Its perspectives are oriented on • Tasks; • Methods; • Domain knowledge. It tries to figure out the main task of CBR by its methods, but one has to know that it is not possible to value the quality of explicit information and sequential events. It only consists of implicit information and operations on data structures. For this purpose we also introduce a meta-language and structural diagrams to represent sequential and parallel events. This approach will be continued for the decomposition of the main task, after the following definitions.

6.1.1.

Definitions for Task-Method Decomposition

Tasks arise from global goals and eventually also from several local sub-goals. For the executable of methods global and local knowledge needs to be provided about: 145

• • • •

The application; The domain; The current problem; The current context.

Tasks are described by a set of six descriptors, mainly taken from Aamodt (1995): • The function of a task: The objective or purpose of the task, describing what the task does, without paying attention to how it is done. • The input of a task: All the information that may be needed to perform the task is listed, irrespectively when it is needed. • The output of a task: All the information produced by the task. • The knowledge needed by the task: Here should be specified the kind of knowledge needed to perform the task. This may be done on two abstraction levels: – The real world representation; – The system internal representation. • The problem solving methods of the task: This is a list naming the problem solving methods used by the task. • Special considerations: Here particular things to account for, when performing the task, are collected such as: – Practical problems; – Limitations. Tasks can be further decomposed into sub-tasks by task-decomposition methods or directly solved by task-executable methods: Task-decomposition methods: They are the inner nodes of the evaluation tree. Task-executable methods: They are the leaf nodes of the evaluation tree. Methods are described by a set of three descriptors, mainly taken from Aamodt (1995): • The condition for the method: The conditions which must be valid for the problem solving method that it can be executed. • The task decomposition resulting from the method: A major purpose for a problem solving method is to generate an appropriate task decomposition. This descriptor contains a set of sub-tasks, which the method decomposes the task into. For the task-executable methods this descriptor is empty. • The control-structure of the method: Specifies the structure of traces (steps) which enables the task to be solved. The problem solving strategy here is described in the form of an algorithm, as a structural diagram, or in the terms of a meta-language. From a node there may be edges to other nodes, which are implicitly sub-tasks or they may lead to some leaf nodes, which are methods of this task at this level of abstraction. For solving a certain problem one or more methods may be selected application-specifically.

146

6.1.1.1.

The Scientific Background of the Task-Method Decomposition Model

In this section we roughly describe essentials of the Components of Expertise methodology (CoE) (Steels, 1990). A knowledge level description of a system, according to CoE, is made by splitting the analysis in three perspectives of a knowledge level description (fig. 6.1). DOMAIN KNOWLEDGE

TASKS PROBLEM SOLVING METHODS

Fig. 6.1 – The knowledge perspectives (Aamodt, 1995)

Tasks (or sub-tasks) are identified in order to solve an application problem. Methods are problem solving methods for further decomposing a task into sub-tasks or for solving a task without further decomposition. There are two main types of models: Domain models are general domain knowledge models, e.g. associations between a set of possible findings and a set of possible faults. There may be also a set of decision rules, or a model of functional relationships between devices. A case model is a description of the current problem being solved, e.g. a set of current measurements and observations. Tasks, models, and methods are structured into task decompositions, control structures, and model dependency diagrams.

6.1.1.2.

The Main Task of CBR in the Task-Method Decomposition Model

At the highest level of generality, a general CBR cycle may be described by the following four processes (Aamodt & Plaza, 1994; Aamodt, 1995):

147

Problem

RET RIE VE

New Case

Learned Case Retrieved New Case Case

RETAIN

Previous Cases

REU SE

General Knowledge Tested/ Repaired Case

REV ISE

Solved Case

Suggested Solution

Confirmed Solution

Fig. 6.2 – The CBR cycle

• RETRIEVE the most similar (probable, expressive, evident) cases. • REUSE the information and knowledge indexed by this case to solve the real world problem. • REVISE the proposed solution (and knowledge structures). • RETAIN the parts of experience likely to be useful for future problem solving. A new problem is solved by retrieving one or more previously experienced cases, reusing the case in one way or another, revising the solution based on reusing a previous case and retaining the new experience by incorporating it into the knowledge base. The four processes each involve a number of more specific steps, which are here to be shown in the task-method decomposition model (fig. 6.3; Aamodt & Plaza, 1994) and to be further described in the now following sections.

148

problem solving and learning from experience

case-based reasoning

retrieve

identify features

copy

evaluate solution adapt

search

retain

extract

repair fault

index

interpret problem follow direct indexes search index structure

evaluate by teacher

copy solution

calculate similarity

search general knowledge

use selection criteria

explain similarity

extract relevant descriptors

integrate

initially match select

collect descriptors

infer descriptors

revise

reuse

extract solutions selfrepair

adjust

evaluate copy solution method

elaborate explanations

modify solution method

in real world

modify solution

userevaluate repair in model

indexes

update general knowledge

determine indexes

generalize indexes

extract justifications extract solution method

rerun problem

Fig. 6.3 – The CBR task-method decomposition

6.2

Retrieve

In this section we enumerate and explain different possibilities to find a similar case in a somehow structured case base to a given query or the provided current situation. Most approaches do a knowledge-poor retrieval like, e.g., PATDEX. Others do a knowledge-intensive retrieve like, e.g., MoCAS (section 1.3). Our aim is to provide an integrated structure, where all current effective methods can be used. An integrated retrieval algorithm may look as follows: Sources of the algorithm: Integration of all related aspects mentioned in this document. INRECA related aspects:

•

Initialise data: The case base is downloaded and a query is specified in the concept-guided query editor. This can be done easily by pop-down menus.

•

Plausibility test is done for intervals. For the symbolic values it is not necessary.

•

If plausibility test failed the user repair is carried out by prompting a new value.

•

INRECA does not need to find a hypothesis.

•

Test selection is automatically done by the k-d tree with the related set of feature status (relevance implicit) and shortcut rules (in INRECA they are called completion rules) or by the user.

•

Constraints are integrated by rules.

•

INRECA takes always the most evident cases as solutions. There is no evidence threshold (e.g., a threshold per diagnosis like in PATDEX), but INRECA can retrieve all cases that are more similar than a user specified threshold.

•

Knowledge-intensive retrieve for INRECA is specified by the integration of S3+ (section 1.3.5).

149

initialise_data (input, user, sensors, filters, relevance); IF plausibility_test failed THEN IF system_repair failed THEN user_repair; ad_hoc_diagnosis (set_of_feature_status, relevances, context, filter-list); WHILE (relevant_test_available) test_selection (filter-list, q-model, set_of_feature_status, hypothesis); IF (no_constraint_violation) THEN update_situation; update_hypothesis; IF (Evidence) THEN solution = hypothesis; IF (NO_evidence) THEN IF NO_knowledge-intensive_retrieve THEN user_solution; Alg. 6.1 – Integrated retrieve

In „open“ systems first the relevant contexts in the general domain knowledge are determined. This is to be done by spreading activation in the ad-hoc diagnosis, searching and following links in the semantic network or the main model. Now one can mark non-relevant concepts for no further use. The links mainly in use (Aamodt, 1995) for that purpose are to be enumerated like the following: • • • • •

General taxonomic relations: has sub-class, is sub-class, has instance, instance of. Causal relations: causes, caused by. Associating links: occurs together with, leads to. Application-specific relations: tested by, tested for. Ordering relations: less than, greater than.

By doing spreading activation, one does recursively the following operations on data structures:

retrieve identify features

follow direct indexes

collect descriptors

initially match

search

interpret problem infer descriptors

use selection criteria

calculate similarity

search index structure search general knowledge

explain similarity

Fig. 6.4 – The retrieve task

150

select

elaborate explanations

6.2.1.

Identify Features

The identification of relevant descriptors for the solution of the current problem is one of the most important tasks in the task-method decomposition model, because it is one of the major criteria of the complexity of the whole system. The less symptoms must be evaluated, the more efficient is the system. What is to be done is to select an adapted method for an application to provide best efficiency. In the following part we first list-up the known and implemented task decomposition methods of the task identify features and explain them by examples.

Identify features

KnowledgeMutual Knowledge poor dependency intensive similarity similarity

CYRUS CcC+ PATDEX/1

FABEL INRECA

PROTOS CASEY CREEK MoCAS

Causal paths

Retrieval adaptation

ReMind MOLTKE MoCAS

PATDEX/2 INRECA

Fig. 6.5 – The identify feature sub-task

• Knowledge-poor similarity: Numerically defined measure in the geometrically interpreted application universe (e.g., Althoff, 1992a). Poor knowledge use for systems in which knowledge acquisition is difficult, expensive, not possible, or not wanted (mainly static systems). Main examples are found at: – Neural filters (Althoff, 1992a; Weis, 1994; 1995b) – Experience heuristics (Wess, 1990) – k-d-retrieval (Derwand, 1994) • Knowledge-intensive similarity: Knowledge-intensive use of significance of descriptors in the problem context where general domain knowledge is available. Main examples to be found by: – Semantic networks (Aamodt, 1991) – Knowledge-intensive retrieval (Pews, 1994) • Mutual dependency: Knowledge-poor and knowledge-intensive calculation of similarities and taking benefit of relative importance of symptoms to explain similarities. Main examples by: – Combined reasoning (Voß, 1994; Voß, Bartsch-Spörl et al., 1996) – Integrated inductive retrieval (Wess, 1995; Althoff, Bergmann et al., 1996)

151

• Causal paths: Providing causal and heuristic rules to render an improved retrieval. Examples are: – Shortcut rules (Althoff, 1992a) – q-model (Cognitive Systems, 1994) • Retrieval adaptation: Considering heuristic local symptom relevance and determination factors to be adapted during runtime to select strategy cases or the diagnosis. Advantages and disadvantages result from the representation of the strategy cases during runtime. Therefore, it should be possible to select a representation similar to the case base. This way it is possible to save time during the planning and the implementation phases of the programme development.

6.2.1.1.

The Integrated Algorithm for Test Selection and Descriptor Identification

Test selection is a central part of every CBR system which can handle abstract or non complete inquiries for problem solving. A test is a potential information gain with the aim to increase the evidence of the current situation. There are some different approaches considered effective and efficient: • PROTOS, PATDEX: Importance conditioned by hypothesis. In PROTOS this is a heuristic measure which provides the problem solution to the question: „How important is this attribute for the statistically proposed problem solution“. • CREEK: It is also integrated how important a missing symptom value of the current situation is for the current hypothesis. This approach provides the problem solution to the question: „What influences has this symptom on the evidence of the hypothesis, if we do not test it“. The following shows the integrated version of a test-selection algorithm: Parameters: Current situation Result: Test to be done. Sources of the algorithm: Integration of all related aspects, especially PATDEX, CREEK, PROTOS, MOLTKE, MoCAS INRECA related aspects: Test selection in INRECA is mainly based on the implicit properties of the kd tree. The structure of the Inreca-tree may be, therefore, optimised in many different ways. An important role plays the optimal division of classes or clusters.

propagate(set_of_feature_status, symptom_categories); evaluate_concepts(relevant_links); look_for_relevant_strategy_cases; calculate_similarity(relevant_strategy_cases); select_test(relevant_strategy_cases, q-model, hypothesis, filters, entropy); IF conflict(selected_test) THEN conflict_resolution_process; Alg. 6.2 – Integrated test selection and descriptor identification

152

There are two different types of conflicts that the system must be able to handle: • The inferred descriptors are unknown to the system. The solution would be: IF check_constraint_violation(feature) THEN reject; ELSIF user_request_positive THEN embed_links(feature, evaluated_concepts); Alg. 6.2a – Conflict resolution during feature identification

• No relevant test to the current hypothesis is found. In this case another inference mechanism or empirical knowledge must be used to find another test. There are several possibilities: – Use different selection methods like neural test selection or search in the general domain knowledge like symbolic models, q-models, semantic nets. – Abductive inference: Do a backtracking in the diagnosis path up to a non-elaborated statistically relevant edge. – Entropy: Select the test with the maximal information gain. – Use different selection criteria like costs, relevances etc. The following table 6.1 lists up whether and how the different systems use the test selection process. Existing architecture: PATDEX/1 PATDEX/2 INRECA CASEY MOLTKE PROTOS CREEK CcC+ Test selection used Selecting most informative test Shortcut oriented test selection Case focusing test selection CBR-based test selection Selecting most relevant test Neural test selection Experience based test selection

• • •

• • •

• • •

•

• • • • •

•

•

•

•

•

• •

•

Table 6.1 – Different kinds of test selection used in the analysed systems Test selection used: It is when the access structure is built such that one may follow different paths conditioned by the same current situation, based on the evaluation of different features. This does not apply when all the features are already evaluated and one decides which path to take. Selecting most informative test: This is the case when your test selection bases on entropy-like measures. Note that it has nothing in common with statistics. Shortcut oriented test selection: This applies when shortcut rules or q-models are used for test selection and feature evaluation. It may be also evaluating symptoms by rules of effect-chains without really testing them. Certainly, these features must be marked „evaluated_by_rule“ that the system can remember that in doubt of consistency. It is not propagating certain, but shortcut rules. Case focusing test selection: This means that the test selection bases mainly on empirical knowledge about a primarily selected hypothesis. It does not mean moving in the knowledge structure closer (topological) towards the region where the cases are stored.

153

CBR-based test selection: It is when a tool recognises current situations and selects (retrieves), based on already made experiences, a test to be made, which is also stored in a (strategy-)case. It is not, when any other kind of test selection is used, if not integrated. Selecting most relevant test: This is the case, when the test is selected by heuristics, which determine a test for a hypothesis or a current situation. These relevances must be directly accessible from the current position in the network to select the relevant links to be followed. It is not cancelling not relevant cases in symptom categories until the solution rests (pure relational retrieval). Neural test selection: This is the case, when integrated connectionistic methods are used for test selection. There may be different networks, gauss-filters, or relevance matrices updated by neural learning functions for CBR-based test selection. Experience based test selection: If there is the possibility to directly copy real experts' problem solving methodologies. This is not the case if there is no possibility to explicitly store sequential strategy knowledge.

6.2.2.

Search

In this section standard algorithms and operations for the search in trees should not be further elaborated, because they can be taken from a good book about data structures. It is more important at this point, to take a view on different search strategies conditioned by the knowledge intensity of the general domain knowledge. It is important to select the kind of influence that metaknowledge or model-knowledge can have on the control-structure, respectively about inferred descriptors, to make the system work effectively and efficiently. These so-called secondary descriptors, in contrast to the primary descriptors that were inferred by the geometrical interpretation, result from inference processes in the general domain knowledge. It is the nature of the way how these secondary descriptors are conditioned by the current situation, which respectively represents the current problem and certainly the current context, that should contribute to a solution to the current ambient problem. However, they are according to our experience also declarative, because the models implemented represent a „copy“ of the wanted environmental relations. They can be used to minimise the number of tests to be made or to find constraint violations and inconsistencies. The following illustration (fig. 6.6) shows different search methods, conditioned by the complexity of their search mechanisms and their knowledge intensity.

154

search linear O(n)

relational O(n)

k-d tree O(log n)

semantic net infer relevant descriptors check feature values in context

primary descriptors

primary and secondary descriptors

explicit knowledge

questioning user and model about expectations

Fig. 6.6 – Methods of the search sub-task

These upper facts are noticed to be implemented in the following systems (fig. 6.7).

search following index pointer (diagnostic inference)

following links in the index structure (test selection)

PATDEX/1,/2 INRECA

PROTOS, CREEK

searching general domain knowledge

Fig. 6.7 – The search sub-task

155

Dynamic Memory Model

Existing architecture: indexed by situations indexed by observable symptoms indexed by relevant symptoms indexed by causal states explained by causal relationships indexed by numeric attributes indexed by generalised indexes

PATDEX/1

PATDEX/2

INRECA

•

•

• •

CASEY

MOLTKE

•

• •1

• •

PROTOS

CREEK

•

CcC+

•

•

•

•

•

•

•

•

Table 6.2 – Different kinds of indexing techniques used in the analysed systems Indexed by situations: The cases are directly accessed by lists of symptoms to be evaluated. It is not the case when one evaluates a symptom and checks where its value is relevant. Indexed by observable symptoms: This applies when cases are directly accessible from features, which are pathologic for that cases, but not when this access depends also on other features. Indexed by relevant symptoms: This means that cases are accessible by features which are heuristically selected to be relevant. So the relevance of a feature for a certain case must be stored directly in the pointer structure from the feature to that case. It does not mean just following links of relevant feature values until a case is found. Indexed by causal states: Here a case is indexed by symbolic values (causal states), which must be the same in the query and the case and that are propagated by rules or models. It does not mean a decision tree. Explained by causal relationships: This means that a case is indexed by numerical heuristic relations, which must confirm the case for the query. It is not firing rules, but propagating evidences, which evaluate concepts, if they are above a certain threshold. Indexed by numeric attributes: Here symbolic and numeric attributes can be used to indicate the cases in the knowledge structure. It does not mean simulating numerical processes or building case-bases that should do an abstraction of that. Indexed by generalised indexes: This indicates an integration of an inductive decision tree or prototypical cases, which represent the generalised knowledge. It would be also valid for classes of the decision tree itself, but here we talk about integrating inductive methods into CBR, so without cases, they may not be indexed.

Finally in this section, before elaborating the next sub-task, we show the integrated version of the search algorithm in the task-method decomposition model. Its methods should be interpreted as the so-called task-executable methods, when doing spreading activation. Sources of the algorithm: Integration of all related aspects, especially PATDEX1,/2, CREEK, PROTOS, MOLTKE, MoCAS, INRECA/S3+. INRECA related aspects:

•

Mother context is always available because of the current situation, because of the activated structures in the tree and the explicit context of S3+.

•

Supercontexts are easily definable by rules, or automatically by the construction of the Inreca-tree.

1

Namely the MoCAS system.

156

•

In the context-graph model dependencies may be checked to decide about the further strategy during the search process.

•

One may follow the indexes of the k-d tree, or the indexes of the decision tree in the context-graph.

•

Until initially match means for INRECA until reaching a bucket of the tree. For completeness also the relevant neighbour buckets must be checked. select_mother_context; DO IF supercontext_type THEN exit(solution found); IF check_model_dependency THEN infer_descriptors(model_dependencies); ELSE take_experience_descriptors; follow_indexes; create_new_node-links; UNTIL initially_match Alg. 6.3 – Integrated search

6.2.3.

Initially Match

This task's work is to take best cases of the case base and to store them for further elaboration. The following conditions are minimally necessary to value best cases and to notice the differences to others: • Global similarity measures1. • Relevance conditioned by the expected solution. The task consists of two main steps, where in the terms of saving a direct causal relation to the primary task search is meant. A lot of the data, which were calculated and elaborated earlier, especially the explanations, may be reused and need not to be recalculated. This is valid especially for knowledge-intensive systems. Therefore, one should think about an adapted structure, mainly a graph or a list, where reusable data for the retrieval task can be registered directly. This may have links to the diagnosis path, and vice versa, to increase also the performance of the backtracking algorithms. Thus, the abstracted algorithm for „initially match“ could be as follows (alg. 6.4). Sources of the algorithm: Integration of all related aspects, especially CREEK, PROTOS, PATDEX/2, MOLTKE. INRECA related aspects:

The INRECA algorithm normally works always on all the knowledge structures without preselection or further evaluation of explicit/implicit knowledge. All of this is done during the precalculation phases in this case, i.e. during the construction of the tree. However, the user could explicitly specify constraints that result in the preselection of those cases that fulfill these constraints. Such amechanism is directly supported by the relational indexing in INRECA.

1

Not necessary in CcC+.

157

update(already_saved_structures); save(cases_similar_to_query); add(similarities, distances, context, aims, constraints); build_links(taxonomies, weighted_D-links); Alg. 6.4 – Integrated initially match

E.g., this applies in a bucket of the k-d tree or cluster-tree or by cancelling non-relevant cases of a symptom category. A number of cases is reached when it is efficient to further elaborate them, or in a semantic network certain concepts have already been evaluated, but none of them is evident enough. In CcC+ there is also a nearest-neighbour component for initially match. There must be a „rough“ data elaboration to find a hypothesis to consider large case-bases in the system. Otherwise, the problem is the same as in the experience graph which is a really effective model, namely an exploding search space. To cut this search space one does the initially match before the selection step. The existing approaches can be classified by the following criteria (table 6.3). Existing architecture: Similarity Relevance of missing attribute Relevant attributes Nearest neighbour search k-d tree Cluster tree Decision tree Average similarity tree Constraints

PATDEX/1

•

PATDEX/2

• •

INRECA

• • • • • • •

MOLTKE PROTOS CREEK CcC+

•

•

•

• • • •

• • • •

Table 6.3 – Initially matching techniques used in the analysed systems Similarity supported by:

•

Importance conditioned by hypothesis: This is a heuristic measure, which provides the problem solution to the question: „How important is this attribute for the statistically proposed problem solution?“.

•

Importance of missing symptom value of the current situation for the current hypothesis: This approach provides the problem solution to the question: „What influences has this symptom on the evidence of the hypothesis, if we do not test it?“.

Relevant attributes: Like in relational databases one looks for cases with similar relevant attributes (section 5.1.2.2). This approach may be enlarged by calculating empirical probabilities (section 5.2.2.1). Nearest neighbour search: There are precalculated nearest-neighbour-links or Δ-links, which provide the possibility to do hillclimbing from different starting points to find local maximum similarity cases (section 6.5.2.2). k-d tree: Every step cuts the search space by the factor k (section 5.1.1.1 Episodes by k-d trees). Cluster tree: This is a rough preselection done by a spatial clustering of the case base (section 5.1.3). Decision tree: This is a search tree that reflects a generalisation of the case data based on the entropy criterion. Average similarity tree: This is a search tree that takes benefit of the actually used global and local similarity measures by optimising the average similarity of the respective sub-trees during tree construction. Constraints: There may be rules that exclude cases because of constraint violations.

158

Relevant attributes calculating the access structure insert changing the similarity measure access worst case access best case

Nearest neighbour search

k-d tree

Cluster-tree Similarities

Constraints

O(M*N) O(M log N) recalculation

O(M*N) O(1) yes

O(M*N) O(1) yes

O(M*N) O(M*M)

O(M*N) O(M)

O(M*N) O(M)1 yes

(M*N*logN) O(M*N) O(M*logN) O(M log N) recalculation yes2

O(R*N) O(R)

O(M*N) O(M*N) O(M*N) O(M*F*logN O(M log N) O(M*(L+N/L) ) )

Table 6.4 – Contrasting the complexity of different approaches (Goos, 1995; Weis, 1995b) N: number of cases; M: number of features; R: number of relevant features; L: Number of clusters.

In a runtime test found in Goos (1995) all these methods worked correctly (also table 6.4). The most efficient ones were the cluster-tree and the relevant attributes, which provide only a rough preselection. Those where directly followed by the k-d tree, which is correct and complete and the nearest neighbour search. Preselection by similarity or constraints was not tested out, because it is not implemented in CcC+. It may be noted that the runtime results began to divergate on a Macintosh Quadra 950 on Common Lisp when they consulted case-bases with more than 5000 cases.

6.2.4.

Select

The aim of this task is to find the best match in the whole case base. Certainly, only if the search task was complete. For that purpose, there are different conditions to be considered, mainly: • Feature relevance; • The prototypicality of a case in its class or cluster; • Taxonomies and Δ-links. Therefore, the select task depends directly on the results of the search- and the initially matchtask. The following shows the integrated select algorithm (alg. 6.5). Sources of the algorithm: Integration of all related aspects, especially CREEK, PATDEX/1,/2, MOLTKE, MoCAS, CcC+. INRECA related aspects:

•

In a bucket and the related neighbouring buckets it first must be checked if the number of existing, predestinated problem solutions is sufficient for the user's purposes. This is mainly done by the similarity measure and the BWB-test.

•

If it is not sufficient the existing links to the neighbouring buckets must be followed.

•

If a new symptom has been tested, because some of the solutions were rejected, one must check for constraint violations.

•

This should be done until all predestinated solutions are selected or rejected.

•

INRECA selects always a certain number of best matches.

Input: Evaluated similarities, generated explanation, optimised search structure.

1 2

Without index calculation for databases. The k-d tree of CcC+ needs recalculation.

159

check_ most_frequent_relevant_solution(initially_match); DO IF NOT (follow_links) THEN test_symptom; cancel_cases(constraint_violation, inconsistencies); UNTIL (initially_match = empty) OR (hypothesis_similarity > evidence_limit) IF solution_evident THEN take_best_solution; ELSE check_most_frequent_relevant_solutions; Alg. 6.5 – Integrated select

The result is an evident solution, or an empty set of cases, which means that other possibilities must be considered to get a relevant solution. One can change the evidence limit and optimise the solution, if accepted by the user (section 6.3). The following table 6.5 gives an overview of implemented selection criteria linked to system architectures1. Existing architecture:

PATDEX/1 PATDEX/2

Selecting most similar case Selecting most probable diagnosis Selecting best explanation

• •

INRECA

CASEY

MOLTKE

PROTOS

CREEK

CcC+

•

•

• • •2

•

•

•

•

•

•

Table 6.5 – Select strategies used in the analysed systems Selecting most similar case: Here one selects the most similar case in the case base to the query. The selection is based on some similarity or distance functions, which create a total or partial order on all inspected cases. Selecting most probable diagnosis: This is based on determination factors, which represent conditional probabilities of recent3 heuristics on the determined path to the selected solution. Selecting best explanation: Here the explanation strength of a path is evaluated.

The following sub-sections consist of a rough task decomposition of several existing implemented retrieval methods, considered to be effective and efficient. This is done in a very abstract way, but more details would not be useful to elaborate decomposition and characterisation criteria, as it is the scope of this document.

6.2.5.

Knowledge-Intensive Retrieve with Integrated Symbolic Models (MoCAS, KALES)

This section explains the way to retrieve with symbolic models. It may be used as a stand-alone concept, or in the integrated way as an optimising method for effective retrieval, as there is no case, which is evident enough in the case base. MoCAS originally was designed for the purpose of CNC-machines diagnosis, especially for machines with lots of similar modules, like motors, wheelworks, tube or connection systems, e.g. for hydraulics. The main advantage of this approach is that it can find cases that are similar to the query on different semantic abstraction lev-

1 2 3

Methods are directly related to the ones introduced in the section 5.2 „Implicit knowledge modelling“ Namely the MoCAS system. Especially in dynamic domains.

160

els. It is possible to associate cases that have the same components on an abstracted level, but which are stored in different contexts.

Example: Car body fronts and backs are usually made on their own and are welded on the frame later. So take front and back parts as different contexts. However, changing a front and a back wheel is quite the same. One does not even have to change sides, but if only searching in your initially specified context and there is no case: „How to change front wheel“ the game is lost, also if there occurred once a case: „How to change back wheel“ in another context. To a human, it should be no problem to change front wheels, but an autonomous inspection device would not find any solution without respecting the case of the back wheels. Because of that, we will continue with a description of how to do knowledge-intensive retrieve (Pews, 1994). We will first enumerate some basic conditions to do diagnosis with model simulations. In (Faupel, 1992) some different kinds of simulation are enumerated, mainly: Physical, mathematical, and qualitative simulation. In physical simulation, also-called direct simulation (Heinrich, 1994), it is tried to find out more about the real world on the base of model simulations. In mathematical simulation, also-called indirect simulation, numerical methods are used to solve approximately physical-technical problems. Qualitative simulation is an indirect simulation in symbolic models, where continuous sets of values are projected on intervals which are represented by symbols. This is the kind of simulation we need to do knowledge-intensive retrieve. This is realised like in the following figure 6.8. technical system

symbolic model

model synthesis - function - structure - behaviour

- function - structure - behaviour

observation

observed mode of operation - observing changes - duration and sequential methodology of processes

forecast and prevision by simulation

Check differences between observed and simulated lapses

simulated working methodology - prevision of system states - duration and sequential methodology of processes

diagnosis Fig. 6.8 – Scheme of qualitative simulation for failure detection in symbolic models

The algorithm of purely model-based diagnosis works in the following way (alg. 6.6). Sources of the algorithm: Integration of all related aspects, especially MOLTKE, MoCAS, KALES. INRECA related aspects:

161

INRECA has no such simulation module. It would be realisable with an adapted knowledge base of the integrated S3+ module, but there are no interfaces specified yet. Probably, it is more convenient to directly integrate a modelbased module like an integrated MoCAS module. initialise(variables, processes); DO (simulation_steps) propagate_values(model_dependency_links); FOR all_components_with_new_value check_differences(observed_states, simulated states); find_strategy(effect_chains); cancel_candidates(simulation_rules1, derivation_rules2) 3; UNTIL diagnosis_found Alg. 6.6 – Model-based diagnosis

To do qualitative simulation a language is required, which is capable to describe the physical and functional structure of the system-specific architecture of the technical device to be modelled and simulated. This may be based on component- or process-oriented approaches like described in (Rehbold, 1991; Faupel, 1992). In MoCAS the component-oriented approach was proposed. In the following section we will show how the retrieve task of knowledge-intensive retrieval works.

6.2.5.1.

An Inquiry for Knowledge-Intensive Retrieve

This section refers to section 5.1.5 (Integrating symbolic models; also section 1.3.6). All the definitions given in that section are still valid and to be considered during the elaboration. A basic knowledge construction of the knowledge-intensive retrieval was the definition of abstract knowledge levels. Another important conceptual model to be considered is the effect-path, which has been defined in section 5.1.5.1 on the q-model. • A query for knowledge-intensive retrieve is defined as a triple (S,I,Z) with a situation vector S, an intended behaviour I and a current state of the machine Z, as defined earlier. For that purpose it is easy to integrate queries into the semantic network after the diagnosis is found (section 6.5). The following figure 6.9 will clarify this. The bottom row contains cases or the query with their pointers to model components on the lowest level of abstraction.

1 2 3

A rule or function that determines the output of a component port or a process because of its input values. A rule or function that determines the input of a component port or a process because of its output value. Checking the candidates consistency with simulation- and derivation rules is called constraint suspension (Rehbold 1991; Faupel, 1992).

162

mechanic

electro-mechanic

relaisy3

motorx1 relaisx2

abstract query y

motory1 relais y2 contactor y4 controly5

abstract case x

Fig. 6.9 – An exemplary query with MoCAS

The elaboration of a query is structured in the following way (alg. 6.7). Sources of the algorithm: MoCAS, KALES. INRECA related aspects:

INRECA has no such integrated CBR simulation module, because there is no adapted knowledge base available.

163

• Plausibility control (* identify features *): FOR all symptoms Si,Sj with i≠j in S IF connected(pi,pj) AND vi≠vj THEN inconsistency FOR all symptoms Si in S IF constraint_violation(bi) THEN inconsistency • Finding and filtering fitting cases: c=∅ (* initially match *) FOR all cases Fi FOR all symptoms Sj in S IF status(bj)=probable_inconsistency_cause THEN C=C∪Fi (* search *) FOR all Fi∈C FOR all symptoms Sj in S FOR length_of_effect_path_n follow minimal_effect_paths ( Sij, Sij+n ) IF not_same_topologic_context (Sij+n,Sij+n+1) THEN C=C\Si (* select *) FOR all Fi∈C FOR all symptoms Sj in S IF Sij ≠ Sj THEN C=C\Fi FOR all Fi∈C IF not_same_inputsignals(Sij , Sj) THEN C=C\Fi status_semantic = hypothesis_found_in_C Alg. 6.7 – Knowledge-intensive retrieve

All cases that remained in the set named C are now called semantically similar. This is explained by prompting the user the corresponding components for all components in the query. These are referred by the hypotheses with the following properties: • Similarity between the components in the terms of the level of abstraction. Note that here only abstraction level one is checked out. In other cases, e.g., a diesel may be substituted by an electric or maybe in the nearer future by a hydrogenium motor. • Similarity of the intentional context. This may be enlarged easily by rules to a complete diagnosis assistant (e.g., „Change fuel orders“, or: „Insert optoconnector“). • Similarity of diagnosis and hypothesis and the effect on pathologic symptoms in S. This is a kind of validity measure, but not mentioned as that, because it is neither comparative nor plain. • It is easy to count failures of a certain component and create heuristics. These heuristics may be used, e.g., for enterprise internal statistics, but they can also be used to create relevances for a new filter-list. At this point one would be able to change the slot of spreading activation to be continued with a new test selection. However, also in MoCAS one can continue the inference process. What the system does is generating symptoms to be evaluated for cancelling other cases of the C-set. 164

Integrating knowledge-intensive retrieve should be optional for any integrated system, but it becomes extremely valuable when acting on modularly constructed objects with lots of similar modules or contexts. This approach of finding diagnostic solutions by knowledge-intensive analogy will be further deepened for case adaptation methods in section 6.3 (reuse) and for validity measures in section 6.4 (revise).

6.2.6.

Integrated Retrieval with k-d Trees (INRECA)

Entering the tree one follows a path based on the given symptom values until reaching a bucket (identify features). The feature selection is obviously system-guided. Also building the tree is a totally automatic process. In the bucket first the similarities of the cases stored there must be calculated (select). Normally, searching for the n best matches one would not get a satisfying result, because also the neighbour buckets may contain important cases. Thus, a backtracking is necessary to find best neighbour cases (search, initially match) and to make the search sub-task complete. For that purpose the following two geometrical distance tests have been developed. • Ball-Within-Bounds Test (BWB): It needs to be checked if the n most similar solutions are already found. For that purpose the query is interpreted as the centre of a hypersphere and the greatest distance to the n most similar cases as the radius to this hypersphere. If the sphere is totally contained in the bucket, then the n most similar cases are already found. This is realised by the geometrical function of the sphere and the construction of so-called auxiliary cases on the boundary of the buckets. After that only the boundary must be compared. query

dim 2

F2max

auxiliary case

F1min

F1max ball bucket

F2min dim 1 Fig. 6.10 – The ball-within-bounds test

• Ball-Overlaps-Bounds Test (BOB): If the hypersphere of the BWB-test is not totally contained in the bucket, the neighbour buckets must be checked if the sphere overlaps them. This works in the analogue way by constructing auxiliary cases on the boundary of the buckets and the geometrical representation of the hypersphere.

165

dim2

neighbour bucket

F2min

query auxiliary case

dim1 Fig. 6.11 – The ball-overlap-bounds test

Therefore, the k-d tree retrieve-algorithm is like the following (alg. 6.8). Source of the algorithm: INRECA. INRECA related aspects:

This is the core of INRECA's retrieval mechanism. follow_features_until_bucket; FOR all_cases_in_bucket check_best_matches; construct_test_cases; IF bwb THEN solutions_found; ELSE search_neighbour_buckets; FOR all_neighbour_buckets IF bob THEN check_best_matches; Alg. 6.8 – Integrated retrieve with k-d trees

The above described retrieval structure works well for numeric as well as for symbolic, unordered attributes (5.1.1.1). The bounds tests have been improved to enable dynamic bounds and multiple similarity functions. The optimised tree structure – called Inreca-tree – uses different strategies to build the tree, a.o. it uses the entropy criterion to generate a decision tree. This tree can be used to represent disjunctive concepts and to optimise the weights for the relevance matrix. In addition, such a decision tree supports the learning of rules that are then integrated in the retrieval structure as preferences to avoid backtracking as much as reasonable (chapter 7).

6.2.7.

Sequential Retrieve (CcC+)

The classification process in CcC+ begins when the user prompts the first relevant attributes (fig. 12). These will be transformed into data types which the system can handle. If other features are to be tested can be decided at once. Then the preselection step follows to cut the search space. Here several strategies are provided (section 6.2.3) like: 166

• • • • • •

Cluster-tree and relevant attributes. Cluster-tree and nearest neighbour search. Cluster-tree and k-d tree. Cluster-tree and other inference methods of the existing toolbox D3. Relevant attributes and hill-climbing Cluster-tree and relevant attributes for starting cases for the hill-climbing task.

A further reduction of the search space follows by checking constraints which are related to the problem solutions. After that the similarity check is done for all the resting cases to the query. If there is a case above the evidence threshold it is taken directly as the solution. Otherwise, a „cost/benefit“ analysis is done to decide about further test selection. If there is no more feature to be tested the process ends without an evident solution, but it can prompt the result of the most frequent, most similar cases. cases

test selection

classification stimulation abstract on attributes infer attributes

symptom test neccessary

yes

no

case-base

preselecetion with different strategies set of roughly similar cases reduction with constraints set of similar cases similarity check yes

most similar cases

no

evident solution yes

solution

testselection no

no solution

Fig. 6.12 – Sequential retrieve in CcC+

6.2.8.

Integrated Retrieval with k-d Tree and ContextGraph (INRECA/S3+)

Taking a look at the control-structure of the two approaches, INRECA and S3+, provides the points of integration available. Figure 6.13 shows the structural diagrams of the two reasoning schemes. 167

INRECA

S3+

initial situation

initial situation

look for most similar case

found evident case

no

test symptom

test symptom

test selection

test selection

propagate until found most probable context new context

install new decision-tree

no

yes no

adaptation as needed

yes

is the context an evident diagnosis yes

diagnosis

diagnosis Fig. 6.13 – Structural diagrams for INRECA and S3+

To be able to use results of one classification step in another method the interfaces must be defined where the methods' results correlate. These are obviously the classification steps and the test selection. • Using classification results of INRECA in S3+: – The globally stored (manually changed) current situation of INRECA can be propagated into the context-graph of S3+. – With the cases which are possible solutions of INRECA one can look for a fitting generalised context in S3+ to continue the retrieval process. Note that this is a inductive step, and may be incorrect. • Using classification results of S3+ in INRECA: – The globally stored (manually changed) current situation of S3+ can be propagated into the k-d tree of INRECA. – INRECA can be told to retrieve only cases which are specialisations of the current context of S3+. This is a reduction of the search space and definitely a deductive step. – One can continue, using a local similarity measure, which was registrated earlier in the current context of S3+. • Using the test selection of INRECA in S3+: – One can use the case focusing test strategy, where are selected symptoms of the most similar case to the current situation, which are not already tested. – One can use the Entropy measure which provides the optimal information gain. • Using the test selection of S3+ in INRECA: 168

– The context fitting to the current situation must be found, where the next test can be found in its tree. S3+ stores its strategy knowledge only in spatial decision trees. For a correct and complete retrieval process that needs to be implemented completely. If using all this existing strategy knowledge without finding a merely specialised diagnosis, S3+ can automatically change to the INRECA test selection to continue the retrieval process. Another approach of integration are the so-called completion rules (shortcut rules) of INRECA, which provide inferred symptom values. These are called at once after the test of a symptom, if the tested symptom completed the condition part of this rule. These inferred symptom values can certainly be used also by the S3+ module. Example: There exist a context-graph and a case base for a certain application. A technician who has a problem to be solved first checks the context-graph until he arrives at a context where no further strategy knowledge is available. However, this context is still too general. The system now continues with case focusing test selection and finds another context denied by the technician. The system changes to the INRECA inference module and continues the retrieval with the cases which were valid for the last valid context. If it finds a correct solution there the technician can now change the inference module again, in order to look for further specialisations of the retrieved problem solution.

6.2.9.

Explanation-Based Retrieve in Integrated Knowledge Representations (PROTOS, CREEK)

On the base of the input data the system is first carrying out a frame-check in the model. It checks symptom categories (identify features) and evaluates concepts (initially match), where the system needs to decide if an input feature is a finding or a fault. E.g. „Cable rotten“ would be a fault, where „No electricity“ would be a finding. After that, it applies a case-match of the activated categories. This is done by checking existing links (identify features) and propagating in the most evident direction (search), up to the problem solution (select). For the search task one may use the following links in the semantic network: • Semantic relations (between two features) • Case-pointers (between a feature and cases respectively categories) • Δ-Links (between cases) Cases are indexed by: • Relevant findings with a certain predictive strength. • Differential findings which are remindings of a better solution. • Faults and treatments that indicate problem solutions or failures to be avoided. For the select task the following items must be respected: • The candidate's explanatory strength. • The predictivity of its relevant features, if explanatory support is equal. • The interactively scaled fulfillness of requirements using explicit constraints. 169

Feature elaboration follows (more or less, certainly depending on the given information) the terms of the following figure 6.14, trying to reach target concepts.

relational link feature finding

physical state hypothesis

fault

treatment

feature value

target concept

noise observed finding inferred finding

normal state

abnormal state

normal finding observed evidence

diagnostic evidence

quantitative value qualitative value has subclass has part causes, infers, transforms into

inferred evidence

Fig. 6.14 – Explanation-based retrieve

On the control level there are two different kinds of heuristic rules: • Conditioned rules: They select strategies during the diagnosis process, e.g. by deciding relevant links and by doing that also deciding about necessary test selections (search). This may be done also by determination factors. • Conclusive rules: Those do the decision elaboration for the problem solution (select), e.g. by evidence thresholds of the explanatory strength. All algorithms are implemented functionally in LISP, based on the spreading activation principles. There is a high-level process model to guide spreading activation and the diagnosis task model, which is responsible for the problem solution. Therefore, the algorithm for problem solving in integrated knowledge structures is as follows (alg. 6.9). Source of the algorithm: CREEK. INRECA related aspects:

The same methods as in INRECA are the frame check based on the concept available, the propagation of input findings in the k-d tree, the condition checks of relevant paths by BWB- and BOB-tests, and by the similarity measure. The other retrieval mechanisms are not available, because there is no adapted knowledge representation available. Parameters: Input symptoms, constraints.

170

evaluate_symptom_categories; do_frame_check(constraints); propagate(relevant_input_findings); DO IF check_conclusion(active_structure, explanations, relevance, user) THEN IF no_constraint_violation THEN solution_found; EXIT_recursive_loop; ELSE check_condition(active_structure, relations, pointers, links); follow_relevant_paths(active_structure, relations, pointers, links); IF no_solution_found THEN do_heuristic_rule-based_retrieve (symptom_categories, diagnosis_categories); IF no_solution found THEN do_model-based_retrieve(knowledge-intensive_relations); Alg. 6.9 – Problem solving in integrated knowledge structures

Handling this programme is not easy because of its high degree of required user interactivity during the problem solving process. The user must confirm or reject in continuation. However, it may be a powerful tool, if given to the right hands. E.g., for a company big enough to employ at least one knowledge engineer which leads a lot of aftersales service and repairs on machines of a high complexity.

6.2.10.

Conclusions to the Retrieval Task

We described a lot of different retrieval methods in this approach. All of them were complete and correct and more or less efficient. We worked top-down and have got an overview of the task in general and the methods to be used. The following table 6.6 compares all these retrieval approaches to one another with the task executable methods linked to existing architectures. Existing architecture:

Follow direct indexes Search index structure Search general knowledge Calculate similarity1 Explain similarity2 Use selection criteria3 Elaborate explanations

PATDEX/1 PATDEX/2 INRECA CASEY MOLTKE PROTOS CREEK

• •

•

• •

• •

• • • •

• • • • •

•

• •

• •

• • •

• •

• • •

CcC +

• • •

Table 6.6 – Retrieval techniques used in the analysed systems Follow direct indexes: Links are directed and may be followed in one certain direction which leads to the cases. This does not mean moving without restrictions in a semantic network.

1 2 3

See also section 5.2 For explanations see also the section 5.2 For selection criteria see also sections 5.1, 5.2, and 6.2.

171

Search index structure: Moving in a structure of indexes which are connected by relations, until your situation is above a certain evidence limit. Not inductive specialisation. Search general knowledge: There are additional explicit knowledge representations to be used to make the retrieve task more efficient or effective. It is not knowledge extracted from the case base. Calculate similarity: The similarity is a function of a given query and a statistically proposed hypothesis. Explain similarity: The result of the similarity measure is explained implicitly by given relations in the general domain knowledge or by a simulation in a model of the general domain knowledge. Use selection criteria: There is the possibility to do preference checks during the test selection. Elaborate explanations: The calculation of explanations is done step by step during the problem solving process.

6.3

Reuse

In this section we introduce methods for an effective and efficient reuse of problem solutions and the possibilities to create adapted problem solutions from past diagnoses. The results should be directly stored by the system to further elaborate them during the following revise task. The possibilities up to now are still limited, because of the limited knowledge descriptions for modification (or adaptation) knowledge as described earlier. The methods already existing may be described in the task-method decomposition model in the following mode (fig. 6.15).

reuse copy

adapt symptom modification

substitutional reuse

modification of traces

transformational reuse

derivational reuse

Fig. 6.15 – The reuse task

Based on these methods the following integrated reuse algorithm 6.10 can be described. Sources of the algorithm: PATDEX/1/2, CHEF, INRECA, CASEY, PRODIGY/ANALOGY, CAPLAN/CBC. INRECA related aspects:

INRECA provides the transfer–solution, the substitutional_reuse, and the user_reuse methods, the latter with a special support of applying user constraints. The transformational_reuse and derivational_reuse methods are not supported because of the CBR perspective of the approach.

172

IF sufficiently_similar (query, current_case) THEN transfer_solution ELSE IF symptom_modification_sufficient (query, current_case) THEN substitutional_reuse ELSE IF solution_adaptable (query, current_case) THEN transformational_reuse ELSE IF solution_trace_adaptable (query, current_case) THEN derivational_reuse ELSE user_reuse (query, case_base, constraints). Alg. 6.10 – Integrated reuse

6.3.1.

Copy (PATDEX/1,/2)

This is a direct transformation of the diagnosis on a therapy1. This is done directly by the following basic transactions (alg. 6.11). Sources of the algorithm: PATDEX/1,/2, INRECA. INRECA related aspects:

This can be directly done also by INRECA, because queries do have the same concept as the cases. cancel_non_similar_features(query, diagnosis); transfer_diagnosis; Alg. 6.11 – Copy solution

This method is rather typical for pure case-matching systems. In the most diagnoses in technical domains the problem solution will be directly embedded. E.g., if „relays rotten“ then „change relays“. How this is to be done may be found directly in a simple database system, which does not need to be directly integrated into the inference module, if multi-tasking and drag&drop is available.

6.3.2.

Case Adaptation Methods

Conditioned by the problem solving methods knowledge-intensive retrieve and case adaptation must be differentiated. When doing knowledge-intensive retrieve one gets a case adapted by systematically substituting sub-classes by retrieved classes. For developing a therapy first the parts to be adapted by the information the diagnosis provides must be selected. This must be decided by strong domain knowledge. For that purpose it is necessary to mark the adaptable parts early enough, because usually every search process becomes a time-critical task with an increasing amount of data.

6.3.2.1.

Substitutional Reuse (CHEF, INRECA)

This is the easiest form of case adaptation methods. For modification rules are used, dedicating the components to be exchanged in the therapy. CHEF uses heuristics defining a type of critics of

1

A therapy is the problem solution to the current problem, namely the solution to the already found diagnosis.

173

the current solution for the current situation which also defines substitutable attributes. Generally these are mainly addressed by valid contexts or in less structured domains by symptom combinations of the current situation. This way they can be ordered in symptom categories like the already existing ones in the MOLTKE workbench (Althoff, 1992a). Substitutional reuse works also without a domain model, but then user interaction to a high degree may be recommended. The algorithm of substitutional reuse may be described in the following way (alg. 6.12). Sources of the algorithm: especially CHEF; INRECA. INRECA related aspects:

The INRECA algorithm works always on explicit rules, which are provided by the user to check exchangeability. search_different_features(query, current_case); search_common_contexts(different_features); FOR all_different_features_in_case IF checked_exchangability(valid_context, case_feature, query_feature) THEN exchange_feature_in_solution(checked_feature); Alg. 6.12 – Substitutional reuse

This is no problem if the features are ordered by abstraction levels or classes, sub-classes, and instances, because this way it would be very easy to find common contexts and abstraction levels (Pews & Weiler, 1992; Pews, 1994; Wess, 1995). The context is very important as the following example shows. Take the wheel exchange example: One can take the therapy of the right wheels to change the left wheels of a car by changing the position of the wheel. It would not be necessary to exchange also the hands in which the tools are held. However, this would have to be done for a left-hander indifferently, which wheel to be changed. The direction of the screws, in contrary may be always the same if not working on gas tubes. Thus, the context is very important to propagate the right rules.

6.3.2.2.

Transformational Reuse (CASEY)

This means the reuse of the former therapy stored together with the diagnosis. For doing that the following task executable methods must be executed (alg. 6.13). Source of the algorithm: CASEY INRECA related aspects:

INRECA has no planning component, because of the CBR-perspective of the approach. select_adaptable_components; DO index_transformal_operators; select_on_smallest_difference(feature); search_forward_transformal_links(feature, action); UNTIL plan_complete Alg. 6.13 – Transformational reuse

174

What is needed here is an adapted control structure to guide the planning process. A problem here is that it is always possible, that a primary modification of the current problem solution does not match following conditions. This is because of the np-completeness of the planning problem and requires usually backtracking to a high degree. All search is done in a strict domain model (strong theory). In general, the more easy approach is done with evaluation rules of the following form: (Situation ∧ Explanation)n-1 |= {T-Operator} ⇒ (Action ∧ Explanation)n E.g., a diagnosis of a machine where not all parts are directly reachable by hands. If in the diagnosis it was found out that one of these parts is out of order, and no earlier plan was stored, but there is a topologic domain model, then a plan to insert the spare part may be developed conditioned by the topological dependencies of the environment. CASEY (Koton, 1988; 1989) originally was developed to classify heart diseases. What this system is doing is retrieving cases as problem solutions and modifying them with causal general domain knowledge-based of the heart failure model (Long, 1986).

6.3.2.3.

Derivational Reuse Planning the Modification (PRODIGY/ANALOGY, CAPLAN/CBC)

This means the reuse of the former method to find the problem solution to a diagnosis. Here the method to develop the therapy will be adapted. Certainly this method had to be linked in some way to the diagnosis and not only the therapy like in transformational reuse. Therefore, the following parameters of the current diagnosis and the former problem solution method are required: • • • • • •

Problem solving method of the current case; Used operators; Considered sub-goals; Generated alternatives; Paths not taken into consideration; The success of the former solution.

The following shows the abstracted algorithm for derivational reuse in CBR systems with strong general domain knowledge (alg. 6.14). Source of the algorithm: ANALOGY/PRODIGY

INRECA related aspects: INRECA has no planning component, because of the CBR-perspective of the approach. evaluate(initial_situation, old_plan); search_sub-goals(old_sub-goals, initial_situation, old_plan); search_links(alternatives, operators); extract_recursively_sub-plans(new_sub-goals, links); do_plan_in_new_context; IF not_useful(plan) OR not_found(plan) OR rejected(plan) THEN user_plan; Alg. 6.14 – Derivational reuse

175

Certainly, the new plan must be related in some kind to the diagnosis and being linked to it. As an example, let us refer to the upper example, only that now also the Δ-data of a whole group of almost similar machines (e.g., different models, different outfit, or equipment) is available. What must be done now is to develop the method to change spare parts and rotten elements. Thus, one modifies and assembles already existing sub-plans. This method reduces data because of Δ-data, as it is possible to transform different machine models in different configurations into one extended functional model. Another form of derivational reuse is an approach, where the solution method is copied and reused. It is the case, when no adaptation is necessary and may be neglected for theoretical purposes. However, for a system developer it may be a useful alternative and for that purpose we mentioned it at this place.

6.3.2.4.

Case-Based Derivational Reuse (INRECA)

This method is taken directly from the reuse of Smalltalk-80 methods (section 4.4), where empirically tested results may be found in (Bergmann, Maurer & Pews, 1994). For the reuse of modification traces only an abstraction of the similarity measure was necessary (section 5.2.1.9) and an adapting modification of the concepts for the cases. The reuse of modification methods by CBR-methods follows the general principles of the CBR task and, thus, will not be shown in any algorithmic description. The assumptions are only that the main retrieval task terminated with the result that the solution was not found, but that a case was found whose solution may be adapted to the problem description to become the current problem solution. Then the next retrieval process is started to find a fitting adaptation trace in the case base where all modifications, which were already done, are stored. current situation solution method similarity situation to be modified

adaptation trace

reminder to be created reuse

current situation, user-adaptable

modified solution-method

Fig. 6.16 – Case representations consisting of reuse-situations

This way one can find modifications that may create a new case, if the situation is adapted by the user or which may create a reminder from the earlier solution to the current solution. These can then be integrated into the case base. Certainly, the user must confess first the selected trace before doing the case transformation. If he changes the trace it would be useful to (at least) store the new trace with its situation in the modification-case base. This is an effective method to store empirical knowledge about already applied case adaptations. 176

6.3.3.

Conclusions

By the cognitive view it is clear that these models are improvable. By the neuro-cybernetic1 view they are trivial. However, develop a better one is not trivial. Not at least neuro-cybernetic classification is understood very well at this point of research. These reuse-methods provide a great help, if strong general domain knowledge is available. The exceptions are copying the solutions, CBR-based reuse, or modification by the user, where no strong general domain knowledge is needed. The reuse methods are a clearly extension of the problem solving process that began with the retrieve task and, therefore, they are a real help for work to be done in complex systems where a number of similar (or analogous) problems must be resolved several times. The following table 6.7 references where all these methods have already been implemented. Existing architecture:

PATDEX/1 PATDEX/2 INRECA CASEY MOLTKE PROTOS CREEK

Copy solution Modify solution Copy solution method Modify solution method

•

•

• •

• •

• •

•

• •

CcC +

• •

Table 6.7 – Reuse techniques used in the analysed systems Copy solution: Non-fitting features are abstracted away and the case is copied to the solution. Modify solution: There are rules dedicating the components to be exchanged in therapy. Copy solution method: Take an earlier used method to adapt the case to the solution's conditions, like in CAPLAN (Muñoz-A., Paulokat & Wess, 1994; Muñoz-A. & Huellen, 1995), PRODIGY/ANALOGY (Veloso & Carbonell, 1989; Veloso, 1992). Modify solution method: A method is replanned to adapt the current solution to the application's requirements, like in DAEDALUS, ICARUS2 (Langley, 1993). INRECA can retrieve Smalltalk-80 methods (Bergmann, Maurer & Pews, 1994) and adaptation cases.

6.4

Revise (CHEF, PATDEX/2)

Revision may be seen as a preparing process for knowledge acquisition (cf., e.g., Morik, 1989). The solutions found by the earlier tasks and the knowledge structures may be controlled and confirmed or repaired, if they do not work as expected. The task-method decomposition model allows the following view on the integrated revise task (fig. 6.17).

1 2

While neuro- or biologic cybernetic is called the part of cybernetic, which investigates the information flow in natural systems, the research about the simulation of the information flow of natural systems is called electronic cybernetic. Both „pure“ machine learning and planning systems, i.e. not case-based.

177

revise evaluate solution

evaluate by teacher

evaluate in real world

evaluate in model

repair fault

self-repair

user-repair

Fig. 6.17 – The revise task

First, the results should be registered or stored, if they are necessary to be used by the following learning algorithm. The parameters for that task are the developed problem solutions. The algorithm for case revision looks as follows (alg. 6.15). Sources of the algorithm: All related aspects, especially CHEF, PATDEX/2, further S3+/INRECA, MOLTKE. INRECA related aspects:

INRECA and S3+/INRECA provide several strategies to check the validity of their knowledge structures. Because of the evaluate by teacher philosophy there is no further system internal check of the validity of the problem solution to be accepted by the expert. save_results IF NOT knowledge_base_consistent THEN repair_knowledge_base. FOR all_non_evaluated_cases IF no_risky_consequences THEN evaluate(solution); IF rejected_solution THEN learn_from_failure; repair_fault(domain_knowledge); ELSE learn_from_success; Alg. 6.15 – Integrated revise

Although CHEF is no diagnosis but a planning system we present it here as an example because to our knowledge there is no other CBR system containing a revision component like this. However, there is a principally similar approach, developed by Stefan Wess for PATDEX/2 (alg. 6.16). Sources of the algorithm: especially PATDEX/2, INRECA. INRECA related aspects:

Based on their relational indexing retrieval structure (fig. 3.3), PATDEX/2 and INRECA provide the possibility to reject proposed problem solutions and to continue the problem solving process without respecting rejected solutions. In PATDEX/2 the update of the relevance matrix, as a revise step, means that the weights for the rejected diagnosis will be recalculated such that the diagnosis threshold for this diagnosis prevents the suggested case to be suggested again. While this works for every case and diagnosis directly and efficient, the problem is that an additional recalculation of the weights, for the originally suggested diagnosis, can lead to the problem again, namely that the wrong diagnosis would be suggested. INRECA bases, therefore, not on a threshold per diagnosis but on combined local intra- and inter-class similarities (sections 5.2.2.3 and 6.4.3.1, especially alg. 6.19; also chapter 7). Using these betterinformed measures the computed weights are more stable and lead to a better classification accuracy (Wess, 1995). However, in one update step, INRECA cannot guarantee that the rejected diagnosis will be prevented to be suggested because of the weights alone. Instead the so-called exclusion rules can be used in INRECA that can work as 178

user-specified contraints that can temporarily cancel all cases in the case base that do not fulfill these contraints, again using the relational indexing structure. user_checks_diagnosis; IF failure THEN update_relevance_matrices; temporary_cancelling_cases_with_rejected_diagnosis; return_to_retrieve; ELSE learn_correct_case; Alg. 6.16 – Revise based on user feedback and relational indexing

Thus, this works without any planning component. However, take into consideration that the user interactivity is much higher and, certainly, that works only efficient during relational retrieval.

6.4.1.

Evaluate Solution

The algorithm 6.17 for case evaluation works as follows: Sources of the algorithm: MOLTKE/MoCAS/KALES. INRECA related aspects:

INRECA has no simulation component. This is because of the evaluate by teacher philosophy of the approach. For further research it is also planned to integrate a MoCAS-like integrated model-based approach which provides the possibility of simulation on a symbolic level. select_case_to_evaluate; mark_as_evaluated_in(numerical_simulation, symbolic_simulation, real_world); IF general_domain_knowledge_available THEN generate_aim; ELSE ask_user; DO evaluate_in(marked_environment); check_on_differences(evaluation, aim); UNTIL failure OR confirmation Alg. 6.17 – Evaluate solution

6.4.2.

Repair Fault

The mechanism for the repair of cases follows directly on the evaluation task. Certainly, it is necessary only if differences to the aim to be achieved were found during the evaluation process. The system tries to build up a structure which provides the possibility to change certain parts of the solution or to increase the solution evidence. Before rejecting a solution proposed by the system (what should be done rarely, especially when there was only one solution), one may check the protocol of the solution finding process up to all statistically relevant paths. Sources of the algorithm: MOLTKE/KALES. INRECA related aspects: 179

INRECA has no automatic repair component. This is because of the evaluate by teacher philosophy of the approach. Integration is not possible without integrating further deep general knowledge. find_causes(differences); generate_explanations(causes, differences); IF fault THEN save_failure(situation, diagnosis, solution); DO modification(case); UNTIL no_more_faults RETURN repaired_structure; Alg. 6.18 – Repair fault

Certainly, the repaired case may optionally be re-evaluated. Faults may be discovered by a planning algorithm and a comparison of attribute values in specified nodes of the simulated model. The repair task may succeed in a separate repair planning module that changes sub-plans to avoid earlier made faults. In this module the following knowledge types of general domain knowledge need to be available: • General causal knowledge to generate explanations, why specified aims were not achieved (in the machine-diagnosis example, at the end of section 6.3.2.3, this would have been knowledge about accessibility and approachability of machine components). • Domain knowledge about compensation of error causes (this would have been knowledge about mounting and reassembling the components). Changes may be saved and learnt in the following acquisition process for empirical knowledge.

6.4.3.

Validity Measures to Detect Faults in the Knowledge Base

The revision of cases is not sufficient when the whole case structure needs to be updated because of inconsistencies or because of a bad clustering or structural deficiencies. This can happen easily especially when knowledge acquisition can be done manually. For some methods approaches were developed to measure the validity of available knowledge structures or to repair them. These may be divided up, conditioned by the structures supported: • Heuristic quality of clustering (INRECA). • Consistency checks in symbolic models (MoCAS). • Structural deficiencies in knowledge structures (S3+/INRECA)

6.4.3.1.

Similarity as an Effectiveness Measure for Case Cluster Revision (INRECA)

When doing an integrated retrieve with k-d trees, it is clear that the system can handle classes and probably the internal model will not take much difference of classes and clusters if the knowledge structure is built in an at least partially geometrical way. To measure the validity of 180

the knowledge structure where the cases are stored, one must take into account how good classes are divided up by the similarity measure and, certainly, how good it clusters the classes. Therefore, in Wess (1995) a method is explained, based on similarities explained earlier (section 5.2.2.3), which provides this kind of validity measure for a set M with m sub-sets M 1 ,..., M m (of cases). • Global intra-class similarity: simIntra = Mi

1 | M i |2

∑ sim( c, d )

c , d ∈ Mi

• Global inter-class similarity: simInter = Mi

1 ∑ sim(c , d ) | M i|| M \ M i | c ∈M i ,d ∈ M \ M i

From a more abstract point of view, it is possible to compare a consistency check with the interclass similarity measure and a completeness check with the intra-class similarity measure. Here the intra-class similarity must have a high value and the inter-class similarity,by contrast, must have a low value. For providing an efficient method to find sub-optimal solutions it is possible to combine them to the following:

• Combined global intra-inter similarity: Inter . = simIntra simKomb Mi M i (1 − sim M i )

This way it is possible to check the quality of one's similarity or distance function in dependence on how it is adapted to the kind of problem to resolve. For the combined intra-inter similarity the following statements are valid: . • simKomb ∈ [0,1] Mi Komb . • simM i = 1 for an optimal similarity measure.

So it is necessary to calculate the combined intra-inter similarity for all clusters or buckets of one's structure and then look for an adequate spatial partitioning as described earlier (section 5.1.1.1) or like in the following algorithm: Source of the algorithm: INRECA. INRECA related aspects:

This algorithm is the core of INRECA's validity check.

181

recluster_set = Ø; FOR all_buckets calculate_ new_ combined_intra_inter_similarity; IF new_ combined_intra_inter_similarity < old_ combined_intra_inter_similarity THEN recluster_set = recluster_set ∪ current_bucket; cluster(recluster_set); FOR all_buckets recalculate_ new_ combined_intra_inter_similarity; IF mean_value (new_ combined_intra_inter_similarity) < mean_value (old_ combined_intra_inter_similarity) THEN cluster(all_case_base); Alg. 6.19 – Case cluster revision based on intra-inter similarity

6.4.3.2.

Consistency Check in Symbolic Models (MoCAS)

In the symbolic models described here it is very easy to do a consistency check. First one must check for all components if all incoming and outgoing ports are connected. If not there should be a justification which is interpretable to the system's control structure. Then for all components connected to one another it is checked if the outgoing ports are of the same type like the incoming ports of the connected components. This is the basic requirement for the work with effectpaths and an effective simulation of the behaviour of the real world application becomes possible, which provides the failures that happened.

Examples: • If the base of a transistor is not connected it will never conduct until it burns. • If the message 4,5V is passed on a port of the type electricity (yes/no), then the system does what it is told by its internal machine-code representation, but not what the control structure originally wanted to. It is clear that internal inconsistencies must be banished to be able to find the external ones. Thus, an algorithm could be as follows (alg. 6.20). Sources of the algorithm: MoCAS, MOLTKE INRECA related aspects:

This algorithm is specific for symbolic models, which are not integrated in the current INRECA approach, because of the knowledge structures that would be necessary.

182

FOR all_components FOR all_ports check_if_connected; FOR all_components FOR all_ports FOR all_components1 FOR all_ports IF connected THEN check_if_same_types; Alg. 6.20 – Consisteny check in symbolic models for technical devices

6.4.3.3.

Detecting Deficiencies in Context-Graph and k-d Tree (S3+/INRECA)

Structural deficiencies always arise when certain properties of the knowledge base are not respected by the provider of explicit knowledge. These structural properties, which must be guaranteed, are always entailing the representation formalism. Structural deficiencies may be divided up in two different kinds (Priebisch, 1995): • Critical deficiencies: Lead to a failure of the system. • Non-critical deficiencies: Lead to a performance loss. The process to find these structural deficiencies is called functional validation. In S3+/INRECA the algorithm of functional validation must cope with the following context-graph specific conditions: • • • •

All rules on the lower edges of one node must be disjunctive. Rules to specialisations must not be unrealisable. No rule to a specialisation may be contradictory to the condition part of its context. Conjunctions of one edge should not be redundant (non-critical).

6.4.3.3.1. Structural Deficiencies in the Strategy Knowledge It was shown with test domains that in general the user quickly looses the overview of the definitions of the test order. This may lead to the following to structural deficiencies: • He defines unreachable tests. • He forgets to define tests. Unreachable tests are only a memory overhead, but if tests are forgotten, it may happen that specific diagnoses are not reachable anymore. In this case the inference process terminates in a context which is more general than the one which is not reachable anymore.

1

The second loop may be obviously avoidable by direct indexing.

183

Context graph

Strategic decision-tree

motor block defect

motor block defect yes

cylinder block defect head defect

If the compression test has been forgotten, then motor block defect is the most specialised diagnosis in this sub-graph because its further specialisations are not reachable anymore

compression test

sealing rotten

no

yes

noise when turns Non-reachable test because there exists no alternative in the context-graph

oil on top

if (no compression) if (no compression) and (oil on top) if (compression) and (noise)

Non-reachable rules because of inclusion

Fig. 6.18 – Structural deficiencies in the strategy knowledge of the context-graph

For this purpose so-called context-buckets have been developed in S3+/INRECA. These are buckets at the leaves of the decision tree, where the reference to the contexts is stored, where it leads to.

Example: • For „oil on top“ it would say that two contexts are reached (sealing rotten, head defect). For the knowledge engineer, there are the following possibilities to avoid the upper deficiencies: • If there are several contexts in a bucket whose preconditions are not disjunctive1 (like in our example), then they are not separable by defining additional tests. So, (s)he needs to change at least the succession of the access structure. • Define tests, until all leaves have only one context. • Cancel decisions which lead to empty buckets. The algorithm would be: Sources of the algorithm: S3+/INRECA. INRECA related aspects:

This algorithm is directly integrated in S3+/INRECA.

1

To avoid dizziness with theoretical definitions in the sense of different.

184

FOR all_buckets IF more_then_one_context(bucket) THEN IF preconditions_disjunctive THEN check(permutations_of_test_order) IF resolvable THEN mark_to_be_user_confirmed ELSE user_repair FOR all_decisions IF leads_to_empty_bucket THEN delete_decision Alg. 6.21 – Detecting structural deficiencies in strategy knowledge of context-graphs

This is clearly a user repair approach. Measuring the validity of the whole context-graph considering the whole application is only possible if there is something to compare with, as shown in the next section.

6.4.3.3.2. Revision of Knowledge Structure Deficiencies with Functional Impact on the Selected Solution These are easy to specify by comparing results of the different inference processes. At this point it should be clear that every context in the context-graph must correspond to a specific case in the case-base, and vice versa. Then the k-d tree retrieval is started with the situation of a specific context or rule-based retrieval with the situation of a case. Comparing the results of all context diagnoses and all case diagnoses in the following way1, it is possible to measure the completeness and correctness of the methods relatively to each other as well as the system's internal consistency of the knowledge representations. • The case's diagnosis is more specific than the context's diagnosis. Then the context-graph is correct and consistent related to the case base of the k-d tree, but not complete. There is the possibility to merge the new tests of the k-d tree in order to receive a more specific context, where that certain diagnosis can be stored. • The case's diagnosis is the same as the context's diagnosis. That is perfect. • The case's diagnosis is less specific than the context's one. Then the context-graph is correct, complete and consistent, but the k-d tree retrieval is not complete. A new case should be created with the same properties of that certain context. • If the case's diagnosis and the context's diagnosis are incompatible for at least one trial, then the context-graph is inconsistent with the knowledge base of the k-d tree. Therefore, the two validity checks may be expressed like in the following algorithm: Sources of the algorithm: S3+/INRECA. INRECA related aspects:

This algorithm is directly integrated in S3+/INRECA

1

For large case-bases and context-graphs this revision method and its repairs should be done as automatically as possible and overnight, because of its complexity and the high amount of database- or swap-operations.

185

FOR all_contexts IF context_diagnosis ≠ retrieve_k-d(diagnosis_situation) THEN IF more_special(context_diagnosis, case_diagnosis) THEN create_new_case; IF incompatibility THEN mark_for_user_repair(incompatibilities); status_context_graph=inconsistent; FOR all_cases IF case_diagnosis ≠ retrieve_rule(diagnosis_situation) THEN IF more_special(case_diagnosis, context_diagnosis) THEN insert_tests; create_new context; IF incompatibility THEN mark_for_user_repair(incompatibilities); status_context_graph=inconsistent; Alg. 6.22 – Detecting knowledge structure deficiencies by comparing case-bases and context-graphs

If it is now ensured that the systems work well together for a certain application, it is now possible to value the current solution by doing the same test. Take the solution as a query to the other method with its specified knowledge base and compare the current solution with the newly retrieved one. There may result: • The current solution is optimal for the knowledge-bases. • The current solution can still be specialised. • The current solution is inconsistent to the other knowledge base.

6.4.4.

Conclusions

Some of the methods for case revision working on general knowledge in symbolic models are comparable to the planning algorithms of the reuse task of derivational analogy. However, there is a possibility to make them even more efficient, because also the result to be achieved by the problem solving process should be known and only the method must be checked, at least when evaluating symbolic models. It is possible to use planning algorithms based on backward chaining (Langley & Allen, 1993). It is necessary to consider is that evaluation based on symbolic models is probably more efficient than means-end analysis. Thus, if strong theories are available the bset thing to do is to use them. Concerning validity measures, it is clear that every commercial system should have one in order to avoid problems during the problem solving processes. These would be problems with the knowledge-base, which could cause especially in „high-stress situations“ a lot of trouble. Therefore, we propose these measures to be used at least once before closing the system to avoid the registration of defective or defiant knowledge-bases, but also before integrating new knowledge to avoid confusion during the following retain process (section 6.5 Retain). The following table 6.8 lists the CBR systems where the explained revision methods are already implemented: 186

Existing architec- PATDEX/1 INRECA ture: PATDEX/2 Evaluate by teacher Evaluate in real world with ongoing stimuli interpretation Evaluate in model Self repair User repair Validity measure

•

•

•

• •

CASE

CHE MOLTKE PROTOS CREEK

Y

F

•

• •

•1 •2 •3

•

•

•

• •

CcC +

•

S3+/ INRECA

•

• •

Table 6.8 – Revise techniques used in the analysed systems Evaluate by teacher: There must be an expert available to value the found solution's usefulness. (S)he guides the integration process (section 6.5). Evaluate in real world: The system can notify relevant changes in the environment, which are related to earlier problem solutions, because of an integrated real time sensor system. If there is then also a function to value, if the solution was appropriate for the problem, then the system can guide itself for the following integration tasks (section 6.5). Evaluate in model: There are now two possibilities on two different levels of abstraction. The first is to find inconsistencies in symbolic models. If there was an inconsistency to be found for the retrieval of the solution, then, if trying out the solution in the model, this inconsistency must not exist anymore. If trying out the solutions in topologies with numerical simulation, there must be defined evidence-categories (section 5.1.2.1), based on evaluation functions, on which may be decided if a solution works, or not. Self-repair: If the system can generate repaired solutions by itself. This may be the case, e.g. based on general domain knowledge or on simulation and adaptation. User-repair: The user must find the repaired solution by himself. This, then may be integrated into adapted structures (section 6.5). Validity measure: There is a possibility to „measure“ validity of the knowledge structure. This includes finding contradictions and quality of clustering, but not numerical effectiveness of certain formulas.

6.5

Retain

The retain task implements an update of the knowledge which is available to the system. It describes the experience-based learning process. For CBR systems it is mainly the integration of structures and concepts directly related to the cases, which may be: • • • • • •

Problem situations; Case descriptions; Problem solutions; Problem solution methods; Explanations or justifications; New general domain knowledge used during the problem solving process, prompted by the user.

1

This can be done by MoCAS or KALES. KALES can adapt cases because of changes in the underlying model of the technical device (De la Ossa, 1992; section 6.5.1.4) Validity checking is done by several systems associated with MOLTKE, namely iMAKE, MoCAS,.KALES, and GenRule.

2 3

187

Doing that means mainly elaborating the steps described in the following figure 6.19.

relevant problem descriptors, solutions, explanations

build or modify case

modified case base

select relevant data

update of access

activated knowledge structure

improved knowledge structure

Fig. 6.19 – Knowledge acquisition in integrated structures

The main problems to be respected before building the system are the following: • • • •

Select the information or knowledge to be stored with the case. Select the structure of information and knowledge. Select the index structure to access cases. Select the kind of integration into the knowledge structure(s).

Doing retain means carrying out the following actions on the selected data structures (fig. 6.20).

retain

extract

index determine indexes

extract relevant descriptors extract justification

adjust indexes

generalise indexes extract solutions

extract solution method Fig. 6.20 – The retain task

188

integrate

update general knowledge rerun problem

6.5.1.

Extraction (CASEY, CREEK)

The work to be done by the extraction task is to select the knowledge structures from where the information is taken that should be integrated. Further one must elaborate this information in a way that it can be easily integrated. The learning sources are mainly the following ones: • • • • •

The problem solutions. Activated structures of the diagnosis path. Evaluated concepts in the semantic net or symbolic model. Activated structures and traces of the adaptation, modification and the repair of solutions. Protocols, explanations or justifications to the above liste learning sources.

The extract task may be described in sequential terms in our meta-language in the following way (alg. 6.23). Sources of the algorithm: CASEY, CREEK, and all related aspects. INRECA related aspects:

•

Because of the evaluate by teacher philosophy accepted cases are considered correct.

•

In the case base only positive examples are stored.

•

New cases need to be integrated into the case base manually.

•

Explanations do not need to be extracted because of the obvious k-d tree features. IF successful_solution THEN extract_task_status=success; ELSE extract_task_status=failure; IF solved_by_new_case_or_method_or_user THEN create_new_case; find_used_learning_sources(descriptors, solutions, protocols); IF created_new_case THEN store_with_case(explanation); IF reused_case THEN store_with_case(solution, method_trace, justification); Alg. 6.23 – Integrated extract

In the systems named in the section title, the solution explanation is directly stored with the case in an adapted structure like the descriptors of the path. If a case is reused, its information may also be used for modification processes. The result is an increased performance of the system, because the search in the general domain knowledge is not necessary as far as the provided information is enough to elaborate the given problem.

6.5.1.1.

The Inference Method and Explanation

What is done here is a strategy protocol of the inference path during the problem solving process. The result is the problem solving method by the view of the inference or planning module. This may also enable the system to do a derivational reuse like explained earlier. The only exceptions to be mentioned at this place are failure solutions. They may be extracted as the following structures: • Separate failure cases in the failure buffer. • Marked as total problem cases in the knowledge structure. 189

If noting methods used during reuse and revise it is possible to use the failures not only to change weights of descriptors, but also to create constraints for modification or adaptation of plans. This marks knowledge acquisition by failure, which is to be done like in the following algorithm (alg. 6.24). Sources of the algorithm: CASEY, CREEK, and all related aspects. INRECA related aspects:

•

Because of the evaluate by teacher philosophy accepted cases are considered correct.

•

A deep general knowledge model which supports the following executable methods is not implemented.

•

Constraints can be integrated manually by so-called exclusion rules. IF failure THEN remind_previous_failures; understand_present_failure; correct_present_failure; IF can_be_avoided(present_failure) THEN create_constraints; Alg. 6.24 – Learn from failure

If failures may be registered anyway depends on the diagnosis type and the real world application. One must decide application-specifically if failures are necessary to be used during the construction of a solution. Especially in applications with possibly critical outcomes, the user must be warned early enough before doing something irreversible or destructing expensive items.

6.5.1.2.

Relevant Descriptors

It is certain that one must decide about the selection of relevant descriptors for the retrieval of a case. This is equal to a structuring of the search space and extraction then is an analysis of the case-specific domain knowledge. A simple approach would, e.g., be to use all features as indexes like in instance-based reasoning where every available feature is used to calculate similarities. Another approach would be the explanation-based case-matching where all relations are respected occurring in the explanation-chain. The integrated form of the algorithm is as follows (alg. 6.25). Sources of the algorithm: CREEK, MOLTKE, and all related aspects. INRECA related aspects:

•

In INRECA relevant descriptors are determined during the precalculation phases when the k-d tree is (re-)built.

•

The consequence is that the determination of relevant descriptors for the INRECA system coincides with the revise task. This algorithm needs, thus, not to be repeated. find_relevant_features(current_problem); filter_out_wide_spread_features(current_problem); filter_out_redundant_features(current_problem, case_base); IF validity_measure(case_base, accessing_structure)< validity_threshold THEN adapt(relevances); Alg. 6.25 – Learn relevant descriptors

190

So what is done here may be explained by the standard example implemented in a system called MBRtalk. The first to do is a parallel match on all cases available to select the relevant features of your problem. After that the features that your case has in common with lots of very different cases are filtered out. This works similarly for neural classification in the pattern-match sense. After that the case can be tested against the case base. If classified correctly the descriptors were selected in the right way, if not it is necessary to adapt the used relevance matrix, e.g., by using an adapted learning function. This approach is universal and may, conditioned by the objects of the general domain knowledge, be implemented implicitly like explained in section 6.5.2.

6.5.1.3.

The Justification

The input parameter for an extraction of a justification is the activated knowledge structure. The output is a hypothesis with its justification. The method of justification or explanation searches in the activated structures of the knowledge representation (e.g., parts of a semantic network) and follows the paths, which have the highest accumulated explanation strength. For that purpose every knowledge-intensive relation or combination of knowledge-intensive relations must have a factor describing its explanatory strength. For strong theories this is no problem, but in weak theories it is necessary to rely on expert knowledge or better, also on heuristics of already made experiences. These may be selected and validated conditioned by the following items: • • • • •

The context, which was valid for the relations. Used knowledge-intensive relations. Activated concepts. Activated filters. Activated cases.

The calculation of these explanatory strengths is done in runtime as the knowledge structure is searched. The following algorithm shows a possiblity to extract a justification. Sources of the algorithm: CREEK, CASEY, MOLTKE, and all related aspects. INRECA related aspects:

Because of the structure of the retrieval process, respectively of the k-d tree, there is no justification or explanation component. The aim was to structure the inference process in the most efficient way and not to create the most effective explanation. However, explanations can be extracted by the decision tree. These explanations can also be used as preferences during the retrieval with k-d trees to improve the performance (chapter 7). explain_features_existence; cancel_features_existing_in_one_structure_in_the_other_not; check_knowledge-intensive_similarities_of_knowledge-poor_differences; ask_user; IF confirmation THEN set_relevant; ELSE set_irrelevant; Alg. 6.26 – Integrated extract of justifications

For example, if characterising differences of cases in an EMOP or a category, it is first necesary to explain features existing in one case but not in the other one. For the ones equal to each other nothing needs to be explained, because these features do not characterise any differences 191

relevant to the system. After calculating the knowledge-intensive similarity of syntactically different features, their names and their explanation strengths can be used directly as a part of the explanation. If no relevant knowledge-intensive relation was found between those, then the user is asked to insert it. If he does not want to, they are declared irrelevant. If the user confirms this relation, his/her explanation is inserted as justification into the protocol. In general, the system would highly interact with the user, also to check hypotheses, for the purpose of a limitation of the search space. The extracted protocol is not only used explicitly as an explanation to the user, but also implicitly as a locally extracted control-structure for the elaboration of dead ends or cycles. Thus, it is always possible to do backtracking and check still not used alternatives like relations or methods. For hidden hints one may also, certainly user guided, check rules leading in contrary directions, as a last alternative, if nothing works anymore (abductive inference). These kinds of backtracking very much increase the complexity of earlier optimised algorithms, but in the end one can be always certain that everything possible has been done, because finally it ends in a wide area search providing always optimal solutions. This yields a kind of implicitly extracted control-structure guiding the problem-solving task by differentiating advantages of wide-area and profound search, respectively completeness and efficiency, conditioned applicationspecifically by the risk associated with possibly critical outcomes and the preferred balance of complexity and accuracy.

6.5.1.4.

Extracting Empirical Knowledge by Analogy Using Symbolic Models (MOLTKE, KALES1)

The MOLTKE workbench was designed for fault diagnosis of technical systems. Therefore, it was also necesssary to implement symbolic models of the technical devices to be diagnosed, which were used as general domain knowledge. The KALES sub-system uses the iMAKE (Schuch, 1992) sub-system, which builds a library of all the different machine parts (section 1.3.1). That is done by the following methodology. iMAKE associates to each part's model a pattern of diagnostic knowledge which specifies its behaviour when the model is simulated. iMAKE operates incrementally, allowing the user to select machine parts that should be modelled. When iMAKE now provides two different models, but only one with a case-base, then KALES was implemented to extract the other one. This was done because of the following purposes: • Reducing the costs of redoing the modelling, simulation and the causal diagnostic knowledge extraction task during initial expert system construction. • Making case-specific knowledge reusable that has already been acquired for another expert system. KALES builds analogies between the old and the new machine, trying to transform cases from the old case base to the new one. KALES uses explanations of earlier problem solving for two purposes, first to identify strategies and second to transfer these identified strategies into the target

1

Knowledge Adaptation in Learning Expert Systems (De la Ossa, 1992).

192

domain. The components, which are necessary to build this task, are shown in the following figure 6.21. domain knowledge

machine parts library

model of the source machine

model of the target machine

diagnostic search space

diagnostic search space

diagnostic case-base

diagnostic case-base

evaluation function

learning goals

analogy

hypothesis space

adaptation case-base

hypothesis

learning function domain expert

Fig. 6.21 – An overview of KALES Domain knowledge: Descriptive causal knowledge that describes the source and the target machine. Learning function: It uses the domain knowledge and experiences to perform analogical inference from the source to the target domain. Hypothesis space: Here contemporary hypotheses of analogy identification and explanation are stored, where they are associated with causal and heuristic domain knowledge. Learning goal: It associates a diagnostic source case with general causal domain knowledge from the source and the target machine model to state a diagnostic strategy implicitly included in the diagnostic source case. Evaluation function: It specifies the validity of the adapted strategy. Adaptation case-base: Direct acquisition of adaptation knowledge by the user. These may be hints for feature analogy in terms of feature associations (section 6.3.2.4).

KALES builds plausible diagnostic situations by analogy, guided by order of symptoms in the source cases and incorporates instantiated validated diagnostic strategies into the target case base like in the following algorithm (alg. 6.27). Sources of the algorithm: MOLTKE, KALES, and all related aspects. INRECA related aspects:

Learning by analogy is not possible in INRECA, because there is no deep general knowledge model implemented. Input:

•

The source and the target object and their observed features.

•

The domain of the source and the target objects.

Output: The target domain's case base. 193

FOR all_cases_in_source_case_base check_model_dependencies(source_model, symptoms) build_case_explanations extract_strategies(explanations) explain_test_selection(strategies) generalise_on_library_level(test_selections) FOR all_new_machine_components check_functional_dependencies create_effect_chains new_strategies = map(source_strategies, component_library, new_effect_chains) Alg. 6.27 – Learning case-specific knowledge by analogy using symbolic models

6.5.2.

Integration of Knowledge and Indexes

The algorithm of integration of cases into the case base is the central problem of all CBR systems, because without that no case knowledge acquisition would be possible. We will not elaborate on inserting objects or relations into the general domain knowledge, because this must be done interactively anyway. However, we will talk about integrating cases directly into the general domain knowledge. A case must be embedded into the case base and the accessing structure must be adapted in the way that similar cases correlate more than non-similar ones. This is done by adapting descriptors, filters, weights, or the similarity measures that work on these. First, let us introduce the integrated retain algorithm (alg. 6.28) and list examples for the task executable methods. Sources of the algorithm: CREEK, MOLTKE, and PROTOS. INRECA related aspects:

•

Queries can be integrated directly, because they contain the same concepts as cases.

•

Generalised rules may be extracted for further use by the integrated decision tree.

•

Adaptation is done by the update of the relevance matrix. A correctness and completeness check is would be very complex, because INRECA has been built to handle huge case-bases, but a test platform is implemented.

•

In INRECA only successful cases are stored. IF no_similar_past_case(current_case) THEN construct_new_case; ELSE lazy_generalise(old_case); IF current_case_successful THEN integrate_into_successful_cases; ELSE integrate_into_total_problem_cases; DO adaptation UNTIL system_behaves_as_wanted Alg. 6.28– Integrated retain

6.5.2.1.

Changing the Concept Structure of the System (CcC+)

There are only two different kinds of changes, that may be done in knowledge-bases' concepts. 194

• It may be enlarged by more features. • Features may be modified. The first point of view only creates problems if attributes must be added to already good classifiable cases. Their representation becomes more inaccurate and it is possible that in the following classification tasks also this additional feature must be tested to find the right solution. The second point is the more delicate one, because methods must be found to transform the old attributes to the restructured ones. In CcC+ there is such a component, which helps to define socalled transformation rules. These must be prompted during the restructuring, which works in the way described in algorithm 6.29. Sources of the algorithm: CcC+, INRECA. INRECA related aspects:

INRECA also uses an integrated rule-based component, but not for changing the concept structure. insert_new_attributes user_defines_transformation_rules FOR all_cases propagate_transformation_rules delete_transformation_rules delete_unused_attributes Alg. 6.29 – Changing the concept structure

6.5.2.2.

Integrating Nearest Neighbour Links (CcC+, CREEK)

In the calculation step all the similarities between cases of the case base are calculated and then a certain number of most similar cases is stored with every case. The algorithm1 builds these links is in the following way (alg. 6.30a). Sources of the algorithm: CREEK, CcC+, and all related aspects. INRECA related aspects:

Nearest neighbour search was not implemented, because it is neither complete nor correct. FOR all_cases FOR all_other_cases calculate_similarity IF other_case_among_the_most_similar_cases THEN store_sorted_index_in_case2 Alg. 6.30a – Explicating nearest neighbour links

This algorithm is not very complicated, but the problems arrive when integrating a new case into a huge case base. If there is not time enough to recalculate all similarities there are two other possibilities. The following is the less complex one (alg. 6.30b).

1 2

2

Its complexity is O((#cases) ) In nets a Δ-Link may be created.

195

Source of the algorithm: CcC+. INRECA related aspects:

This algorithm was not implemented, because it may create inconsistencies in the INRECA knowledge base. FOR all_cases IF case_among_the_most_similar_cases THEN store_sorted_index_in_case store_new_index_sorted_in_nearest_cases Alg. 6.30b – Heuristic explication of nearest neighbour links for new cases

Only the nearest neighbours are searched and integrated into one another. This method may create faults in the neighbourhood relationships. Therefore, one can also take benefit of the following recursive algorithm 6.30c (not yet implemented). Sources of the algorithm: Goos (1995). INRECA related aspects:

This algorithm was not implemented, because it may create inconsistencies in the INRECA knowledge base. FOR all_cases 1: IF case_among_the_most_similar_ones THEN store_sorted_index_in_new_case store_new_index_sorted_in_nearest_cases DO 1 (with all nearest cases); UNTIL no_more_near_case_among_the_nearest Alg. 6.30c – Improved heuristic explication of nearest neighbour links for new cases

This algorithm recursively checks all near neighbours of the neighbours until they are too far away. It is clear that it is possible that this method can also create relationship faults, especially when the distance defining near is too small, but the probability of faults is much smaller. Statistical tests with equal and gauss scatter distributed random clusters and cases should follow. Analytically the faults should be decreased to O(1/d2): d = distance for evenly distributed cases.

6.5.2.3.

Integrating Noise-Resistant Prototypical Cases with Gauss-Filters (WheelNet)

This application is based on more dimensional Kohonen-networks used for world models (Weis, 1994). Their main advantage is their applicability. They have already been successfully used for domains with a high amount of noise and provide retrieval rates of 98% after 20 learning steps for clusters with 20% of random noise during the learning and the retrieval process in a four dimensional universe. Here they are used for building prototypical cases, relevance implicit. Source of the algorithm: WheelNet. INRECA related aspects:

With this algorithm it is possible to store empirical scatters of attributes in gauss-filters. This may be used for noise resistant pattern association during the retrieval process or the selection of an application dependent inference method. This algorithm was not implemented, because current research has not yet reached the desired level of inference methods' integration. Another problem is also the high complexity. 196

FOR all_possible_attribute_values create_neurone; initialise_connections; FOR all_filter_clusters initialise_filter; normalise; initialise_local_evidence_threshold; initialise_D-links(distances); Alg. 6.31a – Building noise-resistant prototypical cases

The result is an empty prototypic case. When entering cases do the following processes: FOR all_cases select_cluster; DO adapt_learning_rates; competetive_learning(worst_classified_case); hebb_learning(worst_classified_case's_D-links); kohonen_training(worst_classified_case, neighbour_cases); UNTIL classified_correctly_all_filters; Alg. 6.31b – Entering cases to build a prototypic case

For Kohonen training it is important to take care that the neurones do not get too near because, otherwise, they become too similar. Take an adapted function (Szepesvári, Balázs & Lörincz, 1993), like the following, for that purpose. It should fulfill the following criteria: • • • •

Be positive. Start from zero. Increase towards a maximum. Decrease for larger arguments down to zero.

Fig. 6.22 – Filter division function for Kohonen training: 3

f(x) = a (1 - x ) (e

-bx

-b

-e )

The filter result should be a kind of normalised gauss function around the scatter of attribute val⎡ (x − η )2 ⎤ 1 exp⎢ − ues, type of p( x | η , σ ) = ⎥ if η is near the mean value and σ the mean 2σ 2 ⎦ 2πσ ⎣ 197

value of the distance of the attribute values. If the parameters are known, the result is a normal gauss scatter. It is also possible to start with the gauss function to create the filter if the mean value is known in advance.

p(χ|ησ)

η − iσ

η

η + iσ

Fig. 6.23 – The Gauß scatter

For a large amount of really similar cases to be stored together in only one prototypic case, one can take the filter value to calculate distances and similarities. How „normal“ a query is can be computed by taking its filter values multiplied with the amount of neurones in that cluster, that summed up from every cluster divided by the number of clusters used. Further the filter represents the scatter of the features for the class represented by the cluster.

6.5.2.4.

Automatic Integration into Experience Graphs (PATDEX/1)

When integrating a case into the case-base, the problem situation is sent from the root over all the edges to determine similar symptom combinations where the case may be integrated. For the algorithm1 6.32 itself following the following parameters are needed: • • • • • •

D: Set of symptom formulas to be determined. F: Case to be integrated. M: Symptom pattern determining F (to be integrated). D(K): Formulas determined by the edge K. L: List of nodes. Kn: Node where the edges OK finishes and UK begins.

Source of the algorithm: PATDEX/1. INRECA related aspects:

The algorithm was not implemented because of its high complexity.

1

Algorithm of Stadler & Wess (1989) changed by Weis (1995b).

198

L=∅ FOR all_pattern_to_be_integrated 1: IF ( D∩M ≠ ∅ ) ∧ ( D=M ) THEN IF not_marked_edge1 THEN increase_weight mark_lower_node2

A,B

A,B

A,B

Fig. 6.24 – Integrating „A,B“ in „A,B“

IF ( M⊂D ) ∧ ( D⊄M ) THEN IF not_marked_edge THEN increase_weight integrate_into_lower_edges_of_lower_nodes ( D\M ) integrate_into_upper_edges_of_neighbour_nodes ( D\M ) integrate_into_lower_edges_of_neighbour_nodes ( M∩D ) DO 1:

B A,B

A

A A

B

Fig. 6.25 – Integrating „A“ in „A,B"

1 2

If edges are marked, it means that their weight needs to be increased. They mark an gain of information for the problem solution. Nodes are marked for the further elaboration process of the algorithm.

199

2: IF ( D⊂M ) ∧ ( M⊄D ) THEN divide_edge D(OK)=M∩D increase_weight(OK) D(UK)=M\D integrate_into_upper_neighbour_edges (M\D) integrate_into_lower_neighbour_edges (M∩D) L=L∪Kn

C

A,B, C

A,B

A,B

A,B

C

Fig. 6.26 – Integrating „A,B,C“ in „A,B"

IF ( D⊄M ) ∧ ( M⊄D ) THEN DO 2: integrate_into_lower_edges (M\D)

B A,B

A,C, D

A

B

A

C,D

C,D A

Fig. 6.27 – Integrating „A,C,D“ in „A,B"

IF ( D∩M )=∅ THEN create new path END FOR FOR all_nodes_in(L) FOR all_nodes_in_graph IF equal_symptom_pattern THEN unify_nodes FOR all_marked_nodes_in_graph unify_marked_nodes_by_unifying_edges insert_pointer_on_referred_cases Alg. 6.32 – Integrate cases into experience graphs

In the experience graph explicit negative examples must not be integrated. However, failures (false solutions) may be integrated in another way: It is clear how to make heuristics of the suc200

cesses. In the analogue way it would also be possible to represent heuristics of the failures, by simply counting in the nodes or edges during the search task how often they have been passed. This way no empirical knowledge referred to the successes is lost. This may be an advantage that helps survive problem situations with possibly critical outcomes, but this only with an exponential complexity. Summing up the failures and scaling this value by the number of nodes and the number of already done retrieval processes, one can compare if different applications of your system are more or less effective. Further a relatively low value of a sub-tree is a sign that in this part failures of the system are rare. The experience graph is a complete and correct way to select and reject problem solutions, but it must be also said that it is a delicate structure that is not easy to handle.

6.5.2.5.

Update of Relevance Matrices (PATDEX/2)

During the training phases weights should be tested and adjusted. This should be done in a way that all cases of the case base can be classified correctly. Every case on its own may be tested against the whole case base. This classification is done, as explained earlier, with the derivative of the contrast measure of Tversky. If a case is not classified correctly the parameters, respectively the integrated relevances, must be adapted, e.g., by competitive learning. This can be guided by the following items: • The local evidence threshold of the case is too low. • The situation of one case is a sub-set of the situation of another case. Let be S1⊂S2. If the diagnoses of the cases are different, this can cause always a false classification, if S1∩S2 is always tested at the beginning. This may result in a global inconsistency of all the domain knowledge. • Symptoms being shared by different cases are weighted too high. • Discrminating symptoms are weighted too low. Sources of the algorithm: PATDEX/2, INRECA. INRECA related aspects:

An improvement of this algorithm is directly integrated into the INRECA system (section 6.4.3.1, alg. 6.18). The INRECA algorithm works without thresholds and makes effective use of local similarity measures. IF too_low(evidence_threshold) THEN increase; IF S1⊂S2 THEN change_order (S2); IF relevance_too_high(S1∩S2) THEN decrease; IF relevance_too_low(S1\S2 ∪ S2\S1) THEN increase; Alg. 6.33 – Learning an update of the relevance matrix

It is clear that termination cannot be guaranteed because of dynamic changes and the fact that the separability of the different cases cannot be guaranteed (Nilson, 1965). However, what is possible is a heuristic adaptation by statistics of the success to adapt your learning rate of a certain case or to select the case to be adapted, what leads to an acceleration of the learning process.

201

6.5.2.6.

Generating Shortcut Rules (MOLTKE)

A basic item for rule generation with the GenRule sub-system (Althoff, 1992a) is the comparison of situations. These may be situations of cases for the inductive rule generation or situations of cases and paths for analogue rule-generation. Generated rules are validated by there determination factor (section 5.2.2.1). If the determination is above a certain user specified threshold, the rule is called shortcut rule1 and it is used to derive uninvestigated symptom values in order to confirm a hypothesis. If the determination factor of that rule does not reach that threshold it is called redundancy rule. This specifies symptoms not to be tested to confirm this certain hypothesis, because their action part does not seem to be relevant to resolve the current problem, because of its low empirically estimated probability. These rules are derived in the following way2 (alg. 6.34). Sources of the algorithm: MOLTKE, GenRule, and all related aspects. INRECA related aspects:

In INRECA inductive rules can be extracted from the decision tree. Such rules are diagnostic rules, while GenRule generates completion rules, as the shortcut rules are called in the INRECA terminology. These shortcut rules are valued by estimated conditional probabilities called determination factors. besides an inductive rule generation genRule also supports rule generation based on analogies between cases and paths extracted from the context-graph (section 1.3.1). Input: Case base, paths. Output: Shortcut and redundancy rules.

1 2

The idea is directly motivated by expert behaviour. (S)he is in general capable to infer the situation following the current situation after a certain action. In GenRule, different algorithms can be used. Here we show the integrated inductive and analogue algorithm.

202

FOR all_cases_and_paths cluster_cases_and_paths_by_their_diagnoses; FOR all_diagnoses FOR paths look_for_minimal_paths°1; cancel_non_minimal_paths°; FOR all_cases_and_paths look_for_include_relations(situations)°; order_paths_and_cases_by_the_sub-set_relation°; IF sub-set_of (path_situation, case_situation)°2 THEN create_completion_rule°3; create_unified_situation(case_situations)*4; ask_user_for_ obviously_redundant_symptoms*; sort_out_obviously_redundant_symptoms*; FOR all_cases a_rule= symtoms(case) → unified_situation \ symtoms(case)*; FOR all_cases FOR all_paths a_rule= symtoms(path) ∩ symtoms(case)

→ symtoms(path) \ symtoms(case)°; FOR all_generated_rules calculate_determination_factors; determine_shortcut_or_redundancy_rule; Alg. 6.34 – Generating completion rules

The analogue part compares all cases and paths with similar5 situations. It selects the shortest one of the similar paths. If there is an attribute included in the path, which is not included in the case, it will generate a shortcut rule for exactly this specified value. The inductive part does the same with the situation of the cases and the unified situations of „similar“ sub-sets per diagnosis. The analogue part needs a context-graph from which GenRule can automatically extract the paths. While this requires an integrated system like MOLTKE or S3+/INRECA it offers the possibility of checking the validity of the knowledge base (what GenRule in fact does) when compared to the case-base, and vice versa. Finally, one may note that here we show always the generation of multiple rules, while GenRule (Althoff, 1992a) creates always one rule for one attribute in the action part.

1 2 3 4 5

This (°) is only necessary for the analogue part of the algorithm. This is interpreted as an inconsistency of the knowledge base. The completion rule generated may confirm a solution after the retrieval process by trying out the proposed value. This specifies the integration of rules and cases. This (*) is only necessary for the inductive part of the algorithm. Here similarity is based on the sub-set relation. Similar cases or paths must have the same diagnosis and their situations need to partially include each other. For further information see Althoff (1992a).

203

6.5.2.7.

Integration of Symbolic Models (MOLTKE, MoCAS)

This section describes the integration of general domain knowledge organised in form of a symbolic model with case-specific knowledge. Therefore, the following definition of case-specific knowledge follows the definition of the symbolic model given earlier:

Def.: A case in the MoCAS knowledge base is a quadruple (S,d,I,Z) with: • A set S of symptoms si = (b,p,v) with: – b: component descriptor – p: port descriptor – v: a value from the value range of p. • A diagnosis d (descriptor of the component not working anymore). • A intended behaviour I with cj = (p,v) the input signals of the control unit. • A status description of the machine zk= (b,p,v) when p describes the internal state. The algorithm 6.35 for the integration of case-specific knowledge is now as following divided up into the parts of „integration of the case into the model“ and the „abstraction of the case“. Sources of the algorithm: MoCAS, MOLTKE, KALES, and all related aspects. INRECA related aspects:

It is not integrated, because knowledge-intensive integration of symbolic models into INRECA was not within the scope of the project. With the integration of S3+/INRECA analogue rules may be integrated to support the inference process. create_net_object(case); FOR all_components_in_S create_symptom_link(pi,vi); FOR all_components_in_Z create_state_link(pk,vk); FOR all_components_in_I create_intended_behaviour_link(pj,vj); create_link_to_diagnosis(case); Alg. 6.35a – Integrating a case into general knowledge in form of a symbolic model

C1 Cn

value value

Ci: component or symptom

signals status

case value

value

C1

Cm

correct

index

Ci: component or symptom

fault

abstracted case Fig. 6.28 – An integrated, abstracted case 204

control unit

create_abstract_net_object(case,Fabstract); create_link_to_abstract_case(case); FOR all symptoms_in_case IF no_link_physical_effect_to_diagnosis(symptom) THEN cancel(symptom, Fabstract); FOR simulate_machine(I,Z) IF symptom_values_not_equal(I,Z); THEN create_link_symptom(F,bi,fault_in_pi) ELSE create_link_symptom(F,bi,correct_in_pi); create_link_to_diagnosis(Fabstract, simulated_input_ports_values); Alg. 6.35b – Abstracting a case for its use within general knowledge in form of a symbolic model

6.5.2.8.

Sustained Learning in Integrated Knowledge Structures (PROTOS, CREEK)

PROTOS learns with every solved problem. The learning model integrates a case-based learning and a model-(explanation)-based approach with a learning apprentice view (Bareiss, 1989). If a case-based solution was found by the first trial it increases the relevance of the descriptors. If it needed several trials it looks for the cause of the failure. The relevance of the descriptors of these failure cases will be decreased. If there was no solution proposed by the system it will create a new case. build and/or modify case relevant problem descriptors, solutions, explanations

modified case base create and/or update remindings and descriptors and save cases

select relevant parts of the current case activated knowledge structure

improved knowledge structure

Fig. 6.29 – Main steps in learning by retaining past cases (Aamodt, 1991)

For that purpose the new case will be integrated into a category preselected by the teacher. Then the system integrates the already known dependencies to symptoms and asks the user about additional relevant symptoms and structural dependencies to be integrated into the semantic network. PROTOS provides always only proposals. This means that the user plays a very active role during the knowledge acquisition. This content forms an algorithm (alg. 6.36) in the following way. 205

Sources of the algorithm: CREEK, PROTOS. INRECA related aspects:

These algorithms could not be integrated in INRECA because of the integrated knowledge-bases, which are in use in PROTOS and CREEK. While these systems work on one integrated knowledge structure, especially CREEK was one motivation of the seamless integration of induction and CBR in INRECA, which also uses one integrated knowledge strcuture, namely the improved k-d tree called Inreca-tree (chapter 7). IF CBR_successful; THEN IF solution_case_found_at_once THEN strengthen_relevances(activated_knowledge_structures, solution_path); IF failure THEN find_faults(activated_knowledge_structures, failure_cases); decrease_relevances(findings_on_failure_paths); IF reminder_link_used THEN strengthen_reminder_relevance; ELSE integrate_reminder(failure_case, solution_case)1; IF modified_case OR NOT solution_mergable OR NOT findings_generalisable(name, feature_values)2 THEN create_new_case; DO INTEGRATE(new_case); ELSE create_new_case;3 weaken_remindings_to_retrieved_case; DO INTEGRATE(new_case); user_confirmation Alg. 6.36a – Retain cases

INTEGRATE(solution: new_case): ask_user_for_best_category; integrate(solution, selected_category); ask_user_for_dependencies; If some_explanation_incorrect THEN acquire_model_knowledge(user); integrate_dependencies; IF NOT rerun_successful THEN DO adjust_indexes; UNTIL success; mark_case_as_not_confirmed Alg. 6.36b – Integrate cases into the knowledge structure

1 2

3

Reminders exist only in CREEK. They point from a failure case to a proposed solution-case. CREEK is a „lazy generaliser“(Aamodt, 1991), where the retention of cases and not induction is the primary learning paradigm. Cases are only generalised if they match perfectly on feature names and when some features values have common parents in the subclass_of inheritance hierarchy. This means that there is a totally new solution for the system.

206

The „strengthen relevance“ task gets its training set from the activated knowledge structures in the semantic network. When integrating a new case it is checked first, whether it can be merged with an already existing case by calculating the similarities of the cases of this category to the current case. If it will be merged, the relevances of the merged case will be strengthened. The new case will be marked as non-confirmed, which results in that this case will not be retrieved if there is another relevant case above the evidence threshold. With the case the following items will be stored: • The problem description (realised by links to the features in the semantic net). – relevant findings – differential features • The solution. – successful diagnosis – successful treatment • The trace of the successful problem solving path. – differential cases – prototypicality of the case • Dead ends (attempted paths failed). – failed diagnosis – failed treatment • Extracted explanation chains. – failures – successes • Dependencies, constraints. • Remindings to other cases. The explanation-based methods are related to the notion of their knowledge. This means that a rule is derived by the explanation of its usefulness. In other words, new knowledge is learned in the sense that it is understood by being explained on the basis of which is already known (Aamodt, 1991). This is equivalent to our definition of models at the beginning of this chapter, when considering that it is possible that not all relevant relations are already known. This means implicitly that it is possible to complete models by any experience based knowledge acquisition, like in real life.

207

thing generic concepts

general domain concepts

cases

case 039

case 76

case 112

Fig. 6.30 – The integrated case base of CREEK (adapated from Aamodt, 1995)

6.5.3.

Conclusions to the Retain Task

It is clear, that a specified knowledge structure requests adapted knowledge acquisition methods. Therefore, in totally integrated structures only one integrated and complex algorithm is necessary or complexity must be changed for high user interaction. In general, one selects the automatic or at least the semiautomatic alternative. It is easier for stand-alone concepts. Integrating the knowledge into the available structures requires that the interactivities of the events are programmed universally, as it should be. Existing architecture:

Extract relevant descriptors Extract justifications Stand-alone method All-over integration Hybrid-integration Extract generalisations

PATDEX/1 INRECA CASEY MOLTKE PROTOS CREEK PATDEX/2

• •

•

• •

• •

•1 • •

• • •

• • • •

CcC +

•

•

S3+/ INRECA

•

• • •

Table 6.9 – Retain techniques used in the analysed systems Extract relevant descriptors: It means that it is possible to determine the items that were important for the correct problem solution. These items must be considered relevant for that solution. It does not mean building up an access structure but adapting it to the current problem solution process. Extract justifications: This means that it is possible to provide an explanation or justification of the inference process for the user. This way, he can understand how the system works. If it works correctly, he further would be able to correct obvious errors. This does not mean that the system may do things which are that strange that they must be justified or explained to the user.

1

Firing rules may be interpreted as justifications. Further the MoCAS module provides directly justifications for the proposed diagnoses.

208

Stand-alone method: This means, here is one method of one inference process implemented, working alone without any support. It does not mean that there is one method implemented, but it may be done integrated retrieval with this method. All-over integration: This means here is one integrated method implemented, working on one integrated knowledge structure. It is not the case, when there are different inference modules together in one program working separately. Hybrid-integration: This is the case, when there are different inference tools that work with more or less different knowledge representations, but all of them to solve the same kind of problem. The modules may use results of one another in some case, but no „integrating element“ (e.g., in form of an (integrating) algorithm) exists. Extract generalisations: The system can extract inductive rules by the knowledge structures, which are supported. These rules are not always correct, but with forthcoming experience of the system on the application they can effectively cut the search space without notable impacts on correctness (Althoff, Wess & Traphöner, 1995).

Another point is the balance of pre-processing and the later use of knowledge, respectively the retain and the retrieval time. The point is for our methods the pre-processing phases, namely the preparation of the case base for the retrieval process, is more or less reciprocal to the retrieval time. E.g., sequential retrieve has no pre-processing phases, but a linear complexity. k-dretrieval has the pre-processing phase to build the tree, and a logarithmic complexity.

6.6

Summary

We analysed a number of CBR-related research prototype systems according to a set of systematically chosen decision support criteria. The criteria set again has been completed by the introduction and application of several technical criteria which directly relate to the main phases of the CBR cycle (Aamodt, 1993b), namely retrieve, reuse, revise, and retain. The analysis of the CBR-related systems was based on methods and knowledge modelling techniques described earlier. We newly introduced a set of abstracted algorithms (from the analysed systems) that have been integrated into the task-method decomposition hierarchy for CBR of Aamodt and Plaza (1994) to supplement the more structure-oriented static criteria by more dynamic control criteria. These algorithms have been developed for all the sub-tasks of the task-method decomposition model and can be seen as concrete examples for task-executable methods. For all algorithms it is stated how they relate to the INRECA system.

209

CHAPTER 7: ONE RESULT OF THE EVALUATION OF CBR TECHNOLOGY: THE SEAMLESS INTEGRATION OF INDUCTION AND CBR IN INRECA This chapter focuses on the seamless integration of induction and CBR. In the INRECA system, we perform CBR as well as TDIDT (top-down induction of decision trees) classification by using the same data structure called the Inreca-tree. We extract decision knowledge using a TDIDT algorithm to improve both the similarity assessment by determining optimal weights, and the speed of the overall system by inductive learning of preference rules and their integration into the retrieval structure. The final integrated system evolves smoothly along application development time from a pure CBR approach, where each particular case is a piece of knowledge, to a more inductive approach where some sub-sets of the cases are generalised into abstract knowledge. We evaluate the system on a database of insurance risk for cars and the COILLTE application involving forestry management in Ireland (also section 4.3).

7.1

Introduction

In this chapter we want to focus on a deep kind of integration of TDIDT and CBR that we call a seamless one. The INRECA system has been influenced by both the KATE system (Manago, 1989) and the MOLTKE workbench that includes a.o. the PATDEX CBR system (section 1.3.1). While PATDEX is well-suited for tasks like technical diagnosis, it cannot handle more general decision support tasks (section 9.3) that require arbitrary kinds of value ranges, complex similarity assessment including local similarity measures, a hierarchical case representation as well as the use of general domain knowledge. PATDEX bases on relational indexing (fig. 3.3) and dynamically defined categories of similarity (section 5.1.2.1). This approach works very efficient for symbolic and/or small value ranges. In addition, for PATDEX/2 local similarity measures have been introduced and combined with PATDEX's extension of the contrast measure of Tversky (section 5.2.1.6). However, the „full“ propagation of local similarities cannot be efficiently handled with relational indexing. The main idea was to extend the PATDEX indexing mechanism such that it can process also numeric attributes and make efficient use of local similarity measures. The foundation for this extension was to view cases as points in a n-dimensional space and to view similarity in correspondence to the distance of points (Richter, 1992). For this purpose a k-d tree (Friedman, Bentley & Finkel, 1977) has been introduced in INRECA as an additional basic structure for indexing and retrieval (Althoff, Wess et al. 1994), besides relational indexing and linear lists, because it allows a recursive partitioning of the search space (fig. 7.1). 210

negative

A2 50 40 30 20 10

• Cases are viewed as points in a n-dim. space • Similarity corresponds to the distance of points

positive

? 10 20 30 40 50 60 70

A1

Idea: Recursive partitioning through k-d trees Fig. 7.1 – Foundation: Geometric interpretation of cases

The basic idea of k-d tree building is to structure the search space based on its observed density (respectively on its classification goal) and to use this pre-computed structure for efficient case retrieval (respectively for efficient diagnosis). It functions as a binary fixed indexing structure for case retrieval. Within the k-d tree an incremental best-match search is used to find the K most similar cases within a set of n cases with k specified indexing attributes. Thus, using a k-d tree the search space can be stepwise refined and the case retrieval focused on such sub-spaces where the potentially similar cases are located (fig. 7.2). Partition the search space and search only the part including potentially similar cases

Problem Stepwise refinement Q X

of the search space

Y Q X

Hyperball Found cases Distance Fig. 7.2 – Basic idea

The basic procedure for building a k-d tree can be described as follows (alg. 7.1). If Split?(CB) THEN RETURN(MakeBucket(CB)) ELSE Diskriminator := SelectAttribute(CB); Value

:= SelectValue(CB, Diskriminator);

RETURN(MakeInternalNode(Diskriminator,Value, CreateTree(LowerPartition(Diskriminator,Value,CB)) CreateTree(UpperPartition(Diskriminator,Value,CB))); Alg. 7.1 – Generating a basic k-d tree 211

According to algorithm 7.1 a specific k-d tree can be built by choosing a discriminator attribute, a partitioning value as well as a termination criterion. If we take inter-quartile distance to select discriminating attributes, the median to determine a partitioning value, and if we define the bucket size to be smaller than 3, we obtain the following simple k-d tree as shown in figure 7.3. A1

A2 50 C 40 30 B 20 D 10 A

F

35

G A2 30

35

C(20, 40) H(70, 35) F(50, 50) E(35, 35) I(65, 10) G(60, 45)

D(30, 20)

A1

Chosen parameters: • SelectAttribute(CB) • SelectValue(CB) • Split?(CB)

Inter-quartile distance Median Bucket size < 3

Fig. 7.3 – An exemplary k-d tree

Retrieval in a k-d tree is done by recursive tree search with backtracking. Here two test procedures are recursively applied: Ball-Overlap-Bounds (BOB) and Ball-Within-Bounds (BWB) (Wess, Althoff & Derwand, 1994). Both are relatively simple geometrical procedures. BWB is linear with the number of indexing attributes and BOB is a binary test. While the search is going on, a priority list is defined that contains the ordered list of the current K most similar cases and their similarity to the query. The similarity SIM is computed as follows: for each attribute Aj, one computes a local (i.e., relative to this attribute) similarity simj between the values of the query and the target case. Then, all the local similarities are aggregated in a weighted sum. The priority list is modified when a new case comes in the „top K“. The recursive procedure runs as follows (alg. 7.2; also section 6.2.6).

212

currentNode = k-d-treeRoot; otherNodes = ∅; priorityList = ∅ k-d-treeSearch(Query, currentNode) previousNode = father(currentNode) IF currentNode = leafNode THEN computeSimilarity(query, currentNode) modify(priorityList, currentNode) ELSE nextNode = Value(Query, currentNode) otherNodes = childNode(currentNode) - nextNode k-d-treeSearch(Query, nextNode) END IF inspect(otherNodes) = TRUE THEN

*** BOB-test ***

FOR ALL node ∈ otherNodes DO k-d-treeSearch(Query, node) IF priorityList ≠ full THEN

*** BWB-test ***

k-d-treeSearch(Query, previousNode) Alg. 7.2 – Retrieval using k-d trees

In addition to improving PATDEX's indexing and similarity assessment abilities, the basic capabilities of the KATE induction system, which bases on the TDIDT approach, had to be included because, on the one side, PATDEX lacks them and, on the other side, they are necessary to handle general decision support tasks (also section 9.3). A decision tree is induced from a database of training cases, described by a set of attributes Aj, j = 1, ..., p. Each case belongs to a specific class Ml, l = 1, ..., L. The partitioning procedure attempts to find the most informative attribute in order to create the shortest tree possible. Traditionally, it uses a hill-climbing search strategy and a preference criterion often based on the information gain (Shannon & Weaver, 1947; Quinlan, 1986), like the C4.5 system (Quinlan, 1993)1. At each node in the decision tree, the criterion is evaluated for all the attributes which are relevant and the one is picked which yields the highest increase of the information gain measure. The main steps of the TDIDT approach are described in algorithm 7.3. IF currentNode = leafNode THEN label(currentNode, mostProbableClass) ELSE develop(currentNode, relevantAttributes) select(bestAttribute, relevantAttributes) split(currentNode, bestAttribute) Alg. 7.3 – Top-down induction of decision trees

1

However, many other criteria have been proposed: Impurity reduction (Breiman, Friedman et al., 1984), χ2 statistic (Hart, 1984), G-statistic (Sokal & Rahlf, 1981) etc. For a comparison of selection criteria, see for instance (Mingers, 1989).

213

The capabilities that had to be integrated where the generation of explicit generalisations from a database of cases and the entropy criterion (information gain measure) for test selection. Thus, the final integrated INRECA system used as basic features two structures: decision trees and k-d trees. The purpose of this chapter is to present an implementation of an integrated structure, called the Inreca-tree to avoid confusion with the original k-d tree, that permits both data structures to be used efficiently for diagnostic kinds of tasks. The new structure fulfils the following objectives: • • • • • • • •

Extracting decision knowledge by induction; Use this knowledge in the definition of the similarity measure; Relating the current problem to the learned experience by CBR; Maintaining classification accuracy of CBR when meeting unknown or noisy queries during consultation; Maintaining the incremental behaviour of CBR; Gaining speed by using inductive learned and explicit knowledge; Being flexible as in CBR; Being straight forward as in decision trees.

Some of these objectives are explicitly geared towards the system end-user (e.g., flexibility and speed). For instance, a help-desk support technician needs sometimes to be guided by the system (induction), and at other times drives the search (CBR). On the other hand, some others are compelling for the system builder (i.e., definition of similarity assessment). Finally, some others are dedicated to system maintenance (i.e., incremental learning). However, in their initial form both k-d tree and TDIDT mechanisms are limited in several ways (Manago, Althoff et al., 1993). We summarise the main points briefly: • TDIDT: lack of flexibility and incrementallity, sensitivity to errors and noise in the data during consultation1; • k-d trees: lack of knowledge generalisation, problems with treatment of unknown and unordered symbolic data. Table 7.1 aims at contrasting decision trees and k-d trees and at presenting the improvements gained with integration. The contrast calls especially for extending k-d trees with efficient classification abilities, also for symbolic attributes and unknown values in the query. Therefore, the extension of the usual k-d and decision trees attempts to encompass the objectives we presented above.

1

Note that many TDIDT algorithms have been developed in order to encompass these problems (e.g., Utgoff, 1988, for incrementality, Quinlan, 1986, for tree pruning).

214

Aspects Similarity measure

Decision tree not necessary

Nearest neighbour classifi- not possible (there are exceptions, e.g. ID5) cation using similarity measures Result obtained classification Backtracking in the tree not necessary Cases during retrieval not necessary Order of tests fixed Unordered value ranges possible Ordered value ranges not necessary Unknown attribute values costly copying of cases in during tree building all branches

k-d tree necessary, must meet monotony requirements possible

Inreca-tree multiple, class-specific similarity measures possible

K nearest neighbours necessary necessary query case must be completely described w.r.t. indexing slots impossible necessary not possible

both possible possible arbitrary presentation of query possible possible extension of k-d trees to handle unknown values

Table 7.1 – Contrasting k-d trees and decision trees for diagnosis tasks

Based on an Inreca-tree both a decision tree and an extended k-d tree can be generated (also section 5.1.1.1). Thus, an Inreca-tree can serve two different roles: it can be used for consultation like a decision tree or as an indexing tree to support CBR. To avoid confusion and to differentiate an Inreca-tree from other decision and indexing trees, we use italic letters to note whether an Inreca-tree is used as a decision-tree or an indexing-tree. In an Inreca-tree generalised knowledge can be used to improve case retrieval. This can be achieved by the automatic determination of weights for the relevance matrix using a decision-tree (sections 5.2.2.3 and 6.4.3.1; alg. 6.18), or by extraction of rules from the decision-tree that are then included in the indexing-tree as preferences. The more cases are available, the more general knowledge can be extracted and integrated in the indexing-tree for retrieval improvement. Thus, the INRECA system evolves smoothly along application development time, from a pure CBR approach, where each particular case is a piece of knowledge, to a more inductive approach where some sub-sets of the cases are generalised in abstract knowledge. In the following section we summarise the basic improvements of the Inreca-tree structure on the k-d tree approach. We describe and evaluate the version of the integrated system, which bases on automatic determination of weights using a decision-tree, on a database of insurance risk for cars and the COILLTE application on forestry management in Ireland (section 4.3). We also describe the approach of learning rules from the decision-tree that are then included in the indexing-tree as preferences. We evaluate both approaches according to our project goals (section 2.3) and show that these techniques are successful steps in the direction of the INRECA-CBR approach to knowledge-based system development.

7.2

The Inreca-Tree

In this section, we present an extension of the k-d tree building and search methods, called the Inreca-tree. We also describe how optimal weight vectors can be computed from such a structure by using intra- and inter-class similarity measures (alg. 6.18). The resulting system is completely integrated. The same tree can be used simultaneously as an extended k-d tree in the CBR process for case indexing and retrieval, or as a decision tree in the induction process for case gen215

eralisation or during consultation. The interactions between both approaches are greatly enhanced. As we will illustrate, by this enhancement we obtain a more flexible and acceptable tool that is easier to use and to maintain.

7.2.1.

Extending the k-d Tree Building and Retrieval Methods

This extension aims at creating a single Inreca-tree that can be used indifferently as a k-d tree for retrieval and as a decision tree for knowledge and rules extraction. Like in TDIDT and in k-d trees, the branches of an Inreca-tree represent constraints for certain attributes of the cases. Since we need to handle ordered and symbolic attributes as well as unknown attribute values, we introduce different kinds of branches. These branches are shown in figure 7.4. Unordered Value Ranges

? A i = vij

? A i = vij


vi1

unknown

...

... vij

v im

unknown

Fig. 7.4 – Branches of the Inreca-tree for ordered and unordered value ranges

As stated previously, the current k-d tree algorithms cannot handle unknown values and symbolic attributes. However, this arises often, whatever the attribute choice strategy. Therefore, the retrieval strategy in the k-d tree procedure must be modified, such that it can deal with cases that contain symbolic or unknown attributes, as it can be done in TDIDT algorithms. This modification is performed through an extension of the BOB and BWB Boolean tests presented in section 7.1, that work primarily on binary trees, towards more general multi-dimensional trees. Beside these two tests the basic retrieval procedure remains unchanged.

7.2.1.1.

Extending the BOB-Test

The BOB-test is executed in order to recognise whether a node may contain some candidates that are more similar to the current query than those that have already been found. Therefore, the geometrical bounds of the node are used to define a test point that is most similar to the current query but still lies within the geometrical bounds of the current node. If this test point is in the ball it means that the ball overlaps with the node and then there may be a candidate to the priority list in this node. The extension of the BOB-test requires that the way how these test points are constructed also takes symbolic attributes and unknown values into account. The definition of such a test point requires the assignment of a value to each attribute used in the case description. The value xj for each attribute Aj must 216

a) lie within the geometrical bounds of the dimension defined by the attribute and the node to be investigated. b) be most similar (with respect to the local similarity for this attribute) to the value qj of the attribute in the query. For attributes Aj with ordered value ranges, we determine the bounds (lj...uj) defined by the attribute and the current node in the tree. The value xj of the test point is as follows (1). In this definition, * denotes a new special value assumed in every value range. The similarity between * and every other value in the value range of attribute Aj is equal to 1. ⎧unknown if the node is in the unknown branch of Aj ⎪* if the node is not in a branch of Aj ⎪ x j = ⎨q j if q j ≥ l j and qj ≤ u j ⎪l j if q j u j j ⎩ j

(1)

For attributes with an unordered value range xj is defined as follows (2): ⎧unknown if the node is in the unknown branch of Aj ⎪ x j = ⎨v j if the node is in the v j branch of A j ⎪⎩* if the node is not in a branch of Aj

7.2.1.2.

(2)

Extending the BWB-Test

The extension of the BWB-test for correctly handling unknown values and unordered value ranges can be done similar to the extension of the BOB-test. Let us recall that this test acts as a shortcut during the retrieval process by deciding at a given node of the tree whether all nearest cases have been found, or not. This results in a reduced number of examined cases. Unfortunately, this extension significantly increases the computational cost for executing this test. The gain obtained in the case of symbolic attributes is illusory, because the test must verify each dimension of the attribute space, what can be very high for symbolic attributes. Instead of being in O(k) where k is the number of indexing attributes in the tree, the complexity is in O(k×r) where r is the average number of values for the indexing attributes. However, the executable of this test can nevertheless pay-off, if the access to the cases is very expensive, for example if the cases are stored in some external storage.

7.2.2.

Generality of the Inreca-Tree

Up to now, we have introduced a new unified data-structure, called Inreca-tree, which allows to perform inductive reasoning as well as nearest-neighbour retrieval. For its use as decision-tree, the Inreca-tree can be built by using an entropy criterion for attribute selection (section 5.2.1.4). The classification is then performed in the standard way by a top-down search through the tree until a leaf node with an assigned classification is reached. In this case, no backtracking is required to reach a conclusion. If a nearest-neighbour retrieval should be performed, the Inrecatree can be constructed by using the inter quartile distance or the average similarity criterion 217

(section 5.2.1.4) and the retrieval using the extended bounds test(s). In this case, however, backtracking guided by the BOB-test is required to guarantee that the most similar case(s) can be found.

7.3

Compiling Inductive Knowledge into the Similarity Assessment

The Inreca-tree can be used to realise inductive reasoning as well as nearest-neighbour retrieval of cases. Now we explain how an intermediate approach between TDIDT and CBR can be reached. We start with pure CBR and move more to the side of induction as more cases arise. With this procedure, we want to avoid the problem that the retrieval time increases if more and more cases are contained in the case base. The more cases are available, the more reliable is the classification by the induced decision tree, because the inductive hypothesis is based on a larger set of known examples. Additionally, traversing a decision tree is much faster than nearestneighbour retrieval because no backtracking in the tree is required (fig. 7.5).

retrieval effort

nearest neighbour search INRECA integration induction

case-based reasoning

induction

number of cases

Fig. 7.5 – Retrieval effort with induction, CBR, and the INRECA integration

The general idea of how to realise this shift from CBR to induction is to include more and more nodes in the Inreca-tree for which backtracking is not required. As explained before, backtracking in the Inreca-tree is guided by the BOB-test. Therefore, the only way to reduce the amount of backtracking in the Inreca-tree is to reduce the number of areas that overlap with the computed ball that is drawn around the query. Because the areas themselves cannot be changed without rebuilding the tree, the size of the ball must be reduced in order to reduce the number of overlaps. Please note that the size of the ball is defined by the similarity between the query and the cases currently visited and stored in the priority queue. Reducing the size means increasing the similarity between the query and the cases with the „correct“ classification. We now explain two different ways of how additional knowledge, derived by induction, can be integrated into the similarity measure to increase the similarity between the query and those cases with the „correct“ classification. The first way proposes to automatically adjust the weights that determine how the different local similarities for the attributes influence the global similarity. The second way explicitly compiles the symbolic decision rules that emerge when viewing the Inreca-tree as a decision-tree into a more informed local similarity measure. Both approaches 218

require that the Inreca-tree is constructed as a decision-tree by using the entropy-criterion for attribute selection. The resulting tree can, nevertheless, be used for a nearest-neighbour retrieval, too.

7.3.1.

Determination of Optimal Weight Vectors

A global similarity measure SIM between two cases a and b can be defined as a weighted sum of local similarity measures simj between the p attributes Aj that make up the cases (3). The weights wjl evaluate the relative importance of the attribute Aj for each class Ml, l = 1, ..., L. p

SIM(a, b) = ∑ w jl × sim j (A j (a), Aj (b))

(3)

j =1

where Aj(a) (resp. Aj(b)) stands for the value of case a (resp. b) for attribute Aj.

It is usual that the system user sets up these weights manually. They are normalised afterwards. This kind of approach presents the advantage to strongly implicate the user during the system building process. He generally enjoys this task. However, this kind of job is difficult and errorprone. After several steps (setting and normalisation), the resulting weights are no longer easily understandable. Moreover, one usually evaluates only global weights for each attribute, without taking care about the class to which the cases belong. We propose that the system computes automatically the weights used in the similarity measures. For this purpose, we compute local intra- and inter-classes similarities. The class of each case is given by the leaf node of the Inrecatree in which the case is located. This corresponds to a classification if the Inreca-tree is used as a decision-tree. The Inreca-tree builds a partition of the set of cases in a set of classes. We know for each case its associated class. Let M = {M1, ..., ML} be the set of classes. The Inreca-tree may be viewed as a representation of a set of sufficient conditions (the rules) for a case to fall in some class Ml. Each condition refers to a single attribute Aj contained in the cases. On the basis of this knowledge acquired during tree building, we can compute the intra- and inter-class similarities for each attribute Aj as follows: • The local intra-class similarity measures to which extent a case a is similar to the cases b that belong to its own class Ml with respect to attribute Aj are given by: (4) 1 ∪ sim j,l =

Ml

2

∑ sim

a ,b ∈M l

j

( A j ( a), Aj (b))

• The inter-class similarity measures to which extent a case is similar to the cases that belong to other classes with respect to attribute Aj are defined as follows: 1 ∩ (5) sim j,l = ∑ simj (Aj (a), A j (b)) Ml M − Ml

a∈M l ,b∈M − M l

The goal of setting optimal weights is to maximise the intra-class similarity and to minimise the inter-class similarity between cases. For this purpose, the relative weight of each attribute Aj for each class Ml is defined by (6):

219

{ = ∑ max {0,sim ∪

w jl

} − sim } ∩

max 0, sim j ,l − sim j ,l

L

l=1

∪ j ,l

(6)

∩ j ,l

The knowledge discovered in the Inreca-tree about the class Ml of each case, is transferred to the similarity assessment through the definition of the weights. In section 7.3.3 we will go a step further using knowledge extracted from a decision-tree to adapt the local similarity measures themselves.

7.3.2.

Experiments

This section presents initial results on two databases. The database of insurance risk for cars (called the CAR database) contains 205 cases described by 26 attributes. The goal is to determine a risk estimation factor (scaled from -3 to +3) for car insurance companies, based on several attributes such as manufacturing, price, technical data, etc. (section 3.1.2.1). The second database (268 cases) comes from an application realised by IMS for COILLTE (section 4.3) involving the forestry management in Ireland (it is called the FORESTRY database). The underlying problem is to come up with a thinning strategy which maximises the overall value of the crop while minimising the risk of wind damage. There are 10 attributes. The choice of using nearestneighbours or TDIDT classification for this problem is not fixed. It depends mainly on the forester and the location of the crop. Therefore, the integrated system offers the opportunity to be tuned with respect to the user's needs.

Let us recall that the objective of the k-d tree indexing mechanism is to avoid the computation of the similarity function between a new problem and all the cases of the case base. The goal of the experiment is the following: through the extension of the k-d tree into the Inreca-tree and the computation of optimal weights, we intend to significantly decrease the number of „visited cases“ (i.e., the number of cases on which the similarity is computed). Therefore, we have tested the average number of visited cases by cross-validation (80% of the database is used for training and the remaining 20% is used for test, repeated five times with different training and test sets) for the standard k-d tree and for the Inreca-tree where the weights have been pre-computed. The number of cases to be retrieved varies from one to 30. Figure 7.6 displays the number of visited cases in the standard k-d tree versus in the Inreca-tree.

220

180 160 140 120 100 80 60 40 20 0 1

2

5

10

15

20

25

30

Number of retrieved cases

Fig. 7.6 – Number of cases visited vs. cases retrieved (CAR and FORESTRY databases)

For the CAR database, we built the Inreca-tree with an inter quartile distance because there are many ordered attributes available. On the other hand, we built the tree with an entropy measure for the FORESTRY database because all the attributes but one are symbolic. The retrieval has always been performed without the BWB-test because of the presence of many unordered attributes. The CAR database experiment can be interpreted in the following way. Due to the large number of attributes, it is necessary to browse many cases in the database even if we want to retrieve a small amount of cases only. The extraction of weights from the Inreca-tree allows the similarity measure to be influenced by the acquired knowledge about the space structure. Therefore, the first retrieved cases reflect this structure, whereas the following ones do not. We have to note that the optimisation was achieved based on local criteria and, thus, turns out to be adequate only for a limited number of nearest neighbours. However, it is rarely necessary to return more than ten cases to the user. We obtain the same kind of results using the FORESTRY database. We can also notice that the introduction of automatically computed weights turns out to make the retrieval conducted on fewer cases and, as a consequence, quicker. Note that since the k-d tree mechanism ensures to always retrieve the most similar cases, the accuracy of results is the same for both k-d and Inreca-trees.

7.3.3.

Learning Preferences

In this section we describe the kernel idea of the seamless integration in INRECA. While the k-d tree, and especially the Inreca-tree, is a powerful retrieval structure, it can be observed that backtracking steps during retrieval could be avoided if the respective knowledge is available. The basic idea is to build an integrated structure that can be used as indexing-tree and as decisiontree – this is the original motivation for building the Inreca-tree – and use the latter one to generate symbolic descriptions to be included in the similarity measure. Thus, the underlying similarity measure becomes better informed based on explicitly generalised knowledge. The effect is that unnecessary backtracking during case retrieval is avoided if the decision is already „certain“ (fig. 7.7).

221

B1

Observation: Many backtracking steps can be avoided if the respective knowledge is available. Idea: Interpret the indexing structure as a decision tree and generate symbolic descriptions to be included in the similarity measure.

B2

Effect: Avoids unnecessary backtracking if the decision is already “certain”.

B3

B4

“Tree Pruning”

Size? big

small

Part is not usable

medium Bucket

Small means always:

Bucket

Bucket

Fig. 7.7 – Improving efficiency through learning

The simple exemplary decision-tree given in figure 7.7 should be interpreted that some machine part is not usable if it is small (it must be at least medium-sized). If this general knowledge is extracted from the decision-tree (as a rule) and included in the similarity measure of the indexingtree in form of preferences, then this results in a „soft“ tree pruning. If a machine part is small, it is already clear that it is not usable and backtracking can be avoided, because this case obtains the maximal similarity. Backtracking is only necessary if more similar cases are requested than cases including small parts are available in the indexing-tree. A decision tree is a representation of a set of sufficient conditions for an example to fall in some class. Each condition refers to a single attribute contained in the cases. We now want to compile such a set of conditions for the attributes, required to fall in a leaf node of the Inreca-tree, into the local similarity measure. Thereby, the user-defined similarity measure is modified on the ground of the induced general knowledge.

222

A1 ? v1 A2 ? v2

A3 ? v3

...

...

...

...

if A1 ? v1 then sim 1 =1 if A2 ? v2 then sim 2 =1 if A3 ? v3 then sim 3 =1 Leaf node L1 Fig. 7.8 – Modification of the local similarity measure by induced conditions

Figure 7.8 briefly demonstrates this idea. A section of an Inreca-tree is shown together with a set of constraints. These constraints can be collected by following a path from the root of the tree to a leaf node, e.g. L1. If this Inreca-tree is used as a decision-tree, this set of constraints allows to directly conclude that a new example that fulfils these constraints falls into the class of L1. We can move towards such a classification behaviour by modifying the local similarity measures simi to evaluate to 1 if the respective condition for the attribute holds in the query. This approach requires that besides the class-specific weights introduced in 7.3.1, class-specific local similarity functions are allowed, too. This extension is reflected in the following definition of the global similarity. In this definition, the first argument of SIM refers to the cases of the case bases (which are already classified) and the second argument refers to the query case. p

SIM(a, b) = ∑ w jk × simjk (Aj (a), A j (b)) with a is from class Kk j =1

The difference to the previous definition is that we now have a specific local similarity measure for each class Kk. These classes relate to the leaf nodes of the Inreca-tree. Now we define how a new local similarity measure sim' can be defined which uses the set of conditions induced for a certain class.

⎧1 if y ? v is a condition for the class of case of x and simlk′ (x, y) = ⎨ ⎩siml (x, y) otherwise

y ? v holds

Please note that this definition increases the similarity between the query and those cases that are reached when using the Inreca-tree as a decision-tree. Please note that increasing this similarity decreases the diameter of the ball constructed in the BOB-test and consequently reduces backtracking. If all attributes which occur in the cases are also used as branches in the INRECA tree, the global similarity will increase to 1. In this case, no backtracking in the retrieval process is required at all because the ball constructed in the BOB-test has a diameter of 0. The whole procedure of learning preferences is summarised in algorithm 7.4. 223

• Generate an entropy-based Inreca-tree with bucket size equal to „same class“ (decision-tree). • Extract rules that guarantee the optimal local similarity for the respective attribute value included in the path of „same class“. • Insert these rules as preferences in an Inreca-tree based on average similarity, or another generation proccedure (indexing-tree). • Overwrite the similarities computed by the similarity functions underlying the indexing-tree with the „preferred similarity“ of the rules in case they can fire. • Extract new preferences, or updated preferences, from the „growing“ decision-tree during application development and include them into the indexing-tree. • IF

– the decision-tree does not grow any more in spite of newly incoming cases, or – the learned preferences already avoid backtracking in the indexing-tree completely, or – application development is finished, or – the flexibility offered by the CBR component is not necessary any more

THEN

the CBR component can be substituted by a decision-tree based component without backtracking (i.e. for consultation). Alg. 7.4 – Learning preferences with the integrated Inreca-tree

7.3.4.

What Did We Really Gain?

The introduced improvements allow the implementation of a CBR system and an inductive system within the same structure. The integrated system combines these approaches in a seamless way (fig. 7.9) that is completely transparent for the user. A2 Decision tree classification cases Nearest-neighbours classification A1

Fig. 7.9 – Combining case-based and inductive classification

The final integrated system evolves smoothly along application development time, from a pure CBR approach, where each particular case is a piece of knowledge, to a more inductive approach where some sub-sets of the cases are generalised into abstract knowledge. The Inreca-tree meets all the requirements presented in table 7.1. The objectives we wished to fulfil (section 7.1) are completed: • A single structure can be used either for CBR or for TDIDT retrieval. It can be straight forward and extract knowledge by induction, or flexible as in CBR; • It enables incremental learning and adaptation of the similarity measure; 224

• This learning makes the retrieval quicker (section 7.3.2); • It permits to evolve smoothly along application development time, from a pure CBR approach, to a more inductive approach; • It offers a flexible tool configurable with respect to the end-user's needs.

7.4

Conclusion

In the recent past, more and more research activities in AI have been re-directed towards application-oriented research and technology transfer. However, scaling-up from toy applications to complex real world tasks is not trivial. Further, single-paradigm systems seem to have reached their limit in several respects, and it is time to study more actively how to integrate multiparadigm methods (Michalski & Tecuci, 1994). In this chapter we studied integration along two dimensions: Integration of CBR methods with methods based on general domain knowledge, namely rule-based methods (Golding & Rosenblum, 1991), and integration of problem solving and incremental learning from experience (Aamodt, 1995). Examples of approaches in this direction are improving indexing in CBR by TDIDT (Kibler & Aha, 1987; Cardie, 1993; Ting, 1994), improving similarity assessment in CBR by learned rules (Sebag & Schoenauer, 1994), and combining nearest neighbour classification with symbolic learning (Zhang, 1990). Our approach corresponds to Golding and Rosenblum (1991) on a very abstract level. It includes also the aspect of improving case indexing with a TDIDT algorithm like Cardie (1993). We extract decision knowledge by inductive learning to improve both the similarity assessment and the performance of the CBR system. Our main goal is to combine nearest neighbour classification with symbolic learning. Here the main motivation is similar to the underlying ideas proposed by Wettschereck (1994a) and Salzberg (1991), though the means we selected to achieve it are very much different. The approach taken in the final integrated INRECA system always keeps the full flexibility offered by CBR, including also the non-generalised cases, as long as it is necessary during application development. For applications with regular updates where it is not foreseeable when a knowledge/case base is complete, it might be the case that it will never switch to decision tree consultation, i.e. that the cases will never be completely removed. The INRECA system has been built according to the set of decision support criteria introduced throughout this document. The hybrid combination of nearest neighbour algorithms and nearest hyper-rectangle algorithms as described in Wettschereck (1994b) reflects a set of – interesting – experiments, but remains on the level of toy domains and has not been used in application development. Actually, INRECA appears to be the only CBR tool, at least if compared with commercial tools, that supports an optimisation of the executable system based on the system's experience (section 3.2.3.5, table 3.31). The experiments demonstrate that for both applications the integrated system leads to a reduced number of visited cases and, thus, to an improved retrieval time. From these results, we can conclude that the introduction of knowledge about classes has improved the similarity measure in

225

the sense that it reflects the underlying space structure in a better way. The system can learn incrementally from new cases. On the other hand, the integration of k-d tree mechanisms and TDIDT algorithms in a single Inreca-tree makes the system much more flexible and enables the user to tune it to its own needs. The system can handle efficiently unordered symbolic attributes and unknown values in queries. It can produce decision knowledge by compiling cases or interpret the cases in a CBR manner.

7.5

Summary

We described the seamless integration of induction and CBR in the INRECA system. This integration has been directly motivated from the evaluation of commercial CBR tools (Althoff, Auriol et al., 1994; 1995a) and, thus, shows that the evaluation task successfully guided the development of the INRECA CBR tool. The seamless integration of the final integrated INRECA system is based on a unified data structure, called Inreca-tree, that results from the integration of k-d trees, decision trees as well as the CBR-qualities of the PATDEX system. This structure can be used for top-down induction of decision-trees (including their consultation) and supports the CBR process as indexing-tree. Using the Inreca-tree as decision-tree, a set of classes can be dynamically derived and for these classes weight vectors be determined based on combined local intra- and inter-class similarities. These weights are used as relevances. The resulting update process for relevances yields a performance improvement, as shown for two different applications (including the real-life COILLTE application). The update process is also more stable and better informed as the update process used in PATDEX because it considers the whole case base as well as local similarity measures (the latter ones have only been implicitly covered by PATDEX). In addition to the determination of weight vectors, INRECA can also learn preferences being extracted from a decision-tree and then included into the indexing-tree to avoid backtracking in situations where the underlying decision is clear. INRECA appears to be the only CBR tool, at least if compared with commercial tools, that supports such an optimisation of the executable system based on the system's experience.

226

CHAPTER 8: COMPARING INRECA WITH RELATED WORK There is a large number of topics that can be viewed as somehow related to the INRECA system, the INRECA project, or the technology underlying INRECA. While the INRECA system has already been compared to industrial CBR tools (chapter 3), to CBR-related research prototype systems (chapters 5-6), to approaches from the MOLTKE and INRECA system families (section 1.3), to other approaches to integration (chapter 7), according to feedback from real-life applications (chapter 4) as well as to approaches from other scientific fields (section 1.2), we now want to complete this by discussing many different approaches, systems, techniques, and applications that – hopefully – are helpful within our context of evaluating INRECA. As topics related to the INRECA we considered the following to be of special interest (in brackets we note the places where the respective issues will be or already have been discussed): • Applications: Applications that – carried out with CBR technology or not – appeared to be of interest for the INRECA technology (sections 8.2; 8.5; 8.7; 8.10.4; 1.3.4; 1.3.3). • Case-based reasoning: Alternative CBR approaches to INRECA (in addition to those already covered in our evaluation in the earlier sections) (sections 8.2; 8.3; chapter 6). • Clustering: Approaches to clustering that appeared to be of interest for the INRECA technology (sections 8.3; 5.1.3; 1.2.5). • Databases: Interesting approaches to integrated information systems based on database and CBR systems (sections 8.2.3; 8.2.4; 8.10.1; 1.2.4; 1.2.3). • Decision support: Approaches to decision support (systems) that appeared to be of interest for the INRECA technology (sections 8.9; 8.10.1; 9.3; 1.2.4). • Diagnosis: Approaches to diagnosis that appeared to be of interest in addition to the topics already discussed (sections 8.5; 9.3; 6.2.1.1; 5.1.7; 1.3). • Dynamic diagnosis: Approaches to diagnosis that explicitly covered the aspect of „time“ (section 8.8.1). • Evaluation: Approaches to the evaluation of CBR or knowledge-based systems in general. Such approaches influenced our overall approach of evaluating INRECA (chapter 9; sections 3.1.1; 3.0; chapter 2). • Explanation-based learning: Approaches to explanation-based learning that appeared to be of interest for the INRECA technology (sections 8.3; 8.8.4). See Althoff (1992a) for an explication of the explanation-based learning view on the iMAKE system (section 1.3.1.3) and on other diagnostic learning systems that base on causal knowledge. • Help desk: An interesting solution to a real-life help-desk problem (section 8.10.4). • Indexing: Interesting approaches to indexing have already been discussed in earlier chapters (sections 6.5; 6.2; 5.1; 3.2.1.2). • Induction: Approaches to induction that are related to the inductive methods used in INRECA (section 8.4; chapter 7; section 1.2.3).

227

• Information retrieval: Approaches to information retrieval that appeared to be of interest for the INRECA technology (sections 8.4.10, 8.6.3; 6.2; 1.2.4). • Integration: The variety of approaches dealing with integration along many different directions (sections 8.10; 6.5.2; 1.3.6; 1.3.5; 1.3.2; 1.3.1). • Knowledge acquisition: Approaches to knowledge acquisition that appeared to be of interest for the INRECA technology (sections 9.2; 6.1; chapter 4; sections 1.3.1; 1.2.2). • Knowledge representation: Knowledge representation issues have already been covered to a high degree (section 9.3; chapters 7; 6; 5; 4; sections 3.2.1; 1.3). • Learning: Work done using the general label of learning that of interest for a system that continuously learns form experience (sections 8.4; 6.5; 6.4.0; 1.3.7.3; 1.2.3). • Meta-level reasoning: Approaches to meta-level reasoning that appeared to be of interest for the INRECA technology (sections 8.8.5; 8.8.6). • Model-based diagnosis: Approaches to model-based diagnosis have already been discussed in the context of the MOLTKE and INRECA system families. In this chapter some more general topics will be discussed (sections 8.6; 8.7.3; 8.9.2; 8.10.2; 5.2.1.7; 5.1.5; 1.3.6; 1.3.1.3). • Nearest neighbour search: This issue has already been discussed (sections 6.5.2.2; 5.2; 1.2.5). • Real-time: Solutions to real-time problems that appeared to be of interest for the INRECA technology (sections 8.7; 8.9.3). • Repair: Approaches to the repair problem in diagnosis and classification that appeared to be of interest for the INRECA technology (sections 8.6.5; 6.4.2). • Retrieval: Approaches to the retrieval of data, information, and/or knowledge that appeared to be of interest for the INRECA technology (chapter 7; section 6.2; chapter 5; sections 1.3.6.3; 1.2.4). • Rule-based reasoning: Approaches to rule-based reasoning that appeared to be of interest for the INRECA technology (sections 8.8; 8.9.1; 1.3.4; 1.3.2) • Similarity: Approaches to similarity assessment have already been discussed in detail (sections 6.5; 6.4.3.1; 6.2; 5.2; chapter 4; sections 3.2.1.3; 1.3.7.2; 1.3.6.2; 1.2.5; 1.2.3). • Technical diagnosis: Solutions to the problem of fault diagnosis for engineering systems that appeared to be of interest for the INRECA technology (sections 8.5; 8.6; 1.3). We searched interesting journals, conference and workshop proceedings from 1991 to 1995 looking for interesting contributions using the above key word list. By this we decided to consider about 300 papers for more detailed study. Every paper was classified into one of the following categories: • Remove („Remove it from our list“): The paper was of no further interest for our evaluation task. • Reference („Refer to it in the document“): A reference to the paper appeared to be helpful on at least one aspect. • Abstract („Consider an abstract of the paper“): A few number of aspects included in the paper have been used during our evaluation task. • Complete („Consider the complete content of the paper“): The contents of the paper has been of high importance for our evaluation task. 228

Of course, a comparison with related work is always a very dynamic process. So, also papers that have been published in 1996 or before 1991 have been considered. Though the selection of related work has been very systematic, it was mostly limited to the literature available at our campus (i.e. in the library or within our group) or available at a known ftp/WWW-side.

8.1

Introduction

Knowledge is – in the global competition – one of the strategic factors of success. Documented knowledge about processes is the basis for every planned and prepared improvement (cf., e.g., Jarke, Jeusfeld et al., 1996). Knowledge, which is registered in any recoverable representations, can be the basis for innovation and technical progress. Especially the integration of data, information, and knowledge is of central importance here (Aamodt and Nygård, 1995). We want to discuss the approach taken in INRECA along different dimensions of interest like fuzzy systems (Kruse, Gebhardt & Klawonn, 1993; Zimmermann, 1993) and their application (e.g., Kurtz, 1996), commercial approaches to diagnosis (e.g., Waleschkowski, Schahn & Forchert, 1995), or quality management (Pfeifer, Grob & Klonaris, 1995). Since CBR as a technology appears to offer new approaches for building integrated and flexible information systems (Bartsch-Spörl & Wess, 1996a), issues like method-oriented programming in the large (Nagl, 1990), the integration of already existing software systems (cf., e.g., Nagl, Eversheim et al., 1995), or the integration of information systems based on relational databases (cf., e.g., Jarke, 1992) – among other topics – are of special interest here. Further topics discussed are approaches to CBR which are complementary to the INRECA approach concerning their application, their practical realisation, or their used methods. While there exists a variety of CBR applications (cf., e.g., Wess, 1996; Ehrenberg, 1996; Woltering & Wess, 1996; Althoff, Auriol et al., 1995), we selected these approaches from our evaluation point of view, and did not to give an overview on the state of the art in CBR. These statements also apply to the other work covered in this chapter: We give an overview on a number of approaches to induction with a strong focus on top-down induction of decision trees (for a discussion of the relationships between CBR and logic-based approaches to induction see Morik, 1996). We also cover topics like model-based diagnosis, real-time approaches, and rule-based approaches, which all overlap in their technological effect to at least some degree with the technology underlying INRECA. Finally, we will shortly summarise the main issues of this chapter.

8.2

Case-Based Reasoning

The basic idea of CBR is the solution of new problems on the basis of similar problems of which the solution is already known. Problems are registered as cases which contain the description of the (analytic) problem (section 9.3) in form of pathologic symptoms, the diagnosis, and the fitting problem solution. The problem solving process in CBR consists of finding and comparing similar cases and the adaptation of the retrieved cases to fit for the current problem situation (Kolodner, 1993). CBR is based on the view that a significant portion of human cognition and 229

problem solving involves recalling entire prior experiences or cases rather than just pieces of knowledge at the granularity of a rule like in rule-based systems (see also ???). The idea is intuitively appealing because it is evident in many human problem solving tasks, particularly in classification, diagnosis, and decision support, which are the kind of tasks the INRECA system has been built for. By using cases as structured knowledge as the main source of system knowledge, and deriving „intelligent“ indexes based on problems already solved by the case, we can take advantage of integrated similarity measures, contexts or logical inference. Because of the independence of the cases we can increase the system's knowledge simply by adding new cases. With a CBR approach, users can express the current problem in a simple „natural“ way by prompting for information as needed, and quickly retrieve relevant cases without having to know something about the items you look for.

8.2.1

A Selected CBR Application: Legal Reasoning

There are domains which are „natural“ applications for CBR systems because of their structural properties. One of these is the legal reasoning domain (e.g., Ashley, 1990). State-of-the-art CBR tools should support indexing schemes and structures used for such standard domains. INRECA provides the possibility to emulate all these indexes in the Inreca-tree, but it does not provide a semantic net.

8.2.1.1.

Semantic Nets for Legal Reasoning

The aim was to develop different access paths to information by applying related types of indexes (Rissland, Skalak & Friedman, 1994). These are mainly: • Symptom categories which provide access to any case which contains a certain attribute. • Family resemblance or prototypicality factors. These are psychological measures by Rosch and Mervis (1975). • Legal theories. They specify features to be considered. • Rules to generalise case-law. • Domain taxonomies of law and legal „index opinions“. The system searches the domain network of cases and the other legal knowledge to support arguments in favour or against an individual relief from debts. It generates a heuristic argument together with further case retrieval using multiple indexing strategies in a semantic network. The application was developed for the U.S. federal statute that governs the approval of bankruptcy plans. The case base for any bank consists of a semantic network called case-graph. Cases are represented by factual situations, bundles of citations, stereotypical stories, terms of legal factors (typical symptoms), and measures of their prototypicality.

230

8.2.1.2.

An Integrated Case-Based and Rule-Based Approach for Legal Reasoning

A hybrid approach to represent the legal sources and the use of meta-knowledge seems to be appropriate to solve problems in this domain. This is the architecture of DANIEL (Distributed Architecture for the knowledge-based Integration of Expert systems in Law) in Brüninghaus (1994). The main knowledge sources are legislation and case-law. It has been shown in Rissland and Skalak (1991) that case-law and legislation are best mapped on a case base and a rule-base, respectively. Since in a given case each knowledge source is likely to contribute to the solution and can be assumed to obtain a result on its own, the problem solvers are applied concurrently and autonomously. Their local results are evaluated by a rule-based co-ordination component, whose meta-knowledge is derived from general legal doctrine and from the capabilities of the problem solvers. In the case of differing local results, the most probable solution is chosen according to the legally determined binding force of the respective legal source, the amount of super-contexts and the similarity between the given and the retrieved case. Merging the reasoning chains of a case-based and a rule-based reasoner is generally not advisable for legal reasoning domain because of the following reasons: • Different binding force (validity) of the knowledge forces. • Incompatible partial results of the problem solvers due to their different internal representations and semantics. • Redundant and contradictory knowledge in the problem solvers (only a part of it is supplementary). • Interdependency of the legal sources (existing case-law influences future legislation, and vice versa). This is also an appropriate application for systems similar to INRECA. Here the important facts of the rule chains could be chosen to built the tree which stores the case data in its buckets.

8.2.2

Selected Integrated CBR Applications

In all areas where knowledge from different sources interleaves and processes need to be controlled or planned, AI applications can be used to solve already standardised problems. When we are talking about AI applications in relation to the INRECA project we always mean applications for integrated CBR systems which provide a wide spread problem solving capability. DaimlerBenz proposed in (TopBusiness, 1995) the following applications for AI systems especially for the German market1: • Real-time process control; monitoring: Expert systems guide plants to optimal working conditions. They note in parts of a second if working parameters are wrong, they analyse the causes and provide tools for counter measures or antidotes. E.g., in the power plant of Preussen-Elektra in Staudingen the steam circulation is optimised by a system of ABB.

1

Detailed information about the Japanese relative to the American market can be found in (Feigenbaum et al., 1994).

231

• Diagnosis; maintenance: The expert system contains rules of which symptoms point on certain faults or failures. The computer may also be connected to the object to be diagnosed in order to achieve further information about the system on its own. E.g., Daimler-Benz plans a diagnosis system for motor car workshops. It will be implemented application-independently and may be used also for other engineering systems like aeroplane systems. • Working plans: The experience of the organisers, which parts of which materials can be manufactured on which machine in what succession is stored in the knowledge base of a system which allows to build the working plans. E.g., the Deutsche Aerospace generates working plans of its plant in Hamburg-Finkenwerder automatically and plans to automatically generate also the assembly plans in the nearer future. • Credit worthy checks: This is similar to a legal-reasoning application (see also section 8.2.1). E.g., it is known that the Citibank Germany uses a programme like that. • Construction; development: The software proposes blueprints or constructs immediately components based on construction rules and system component's parts properties. E.g., Mercedes-Benz installed a CAD/CAM system like this. • Securities' business: Depending on the client's wishes and his financial possibilities a strategy for his investments may be inferred. E.g., the Citibank developed rapidly this strategic sphere of activity with the help of its expert system RAMSES. • Risk analysis: Profession, age, hobbies, illness, stay abroad, nationality may influence the client's wishes on a health or life insurance. E.g., a basic model of a commercial system developed by Kölnische-Rück which can substitute in 95% a qualified human risk examiner. There are also other interesting applications for integrated CBR systems like classification which is very interesting especially for natural domains. Decision support domains are likely interesting for economic purposes and diagnosis applications for technical or medical domains. The usage of CBR systems is as wide spread as also the possibilities to build them. With the system resulting from the INRECA project all these domains can – in principle – be effectively handled. A simple approach to build a CBR system which is convenient especially for commercial applications is building a CBR system for queries on the tophead of already existing data in commercial data bases as described by some examples as following.

8.2.3.

CBR Systems as Tophead on Databases

Important information is generally stored in large databases. Conventional database systems offer only limited query flexibility. Therefore, approaches exist to build CBR systems using a conventional database. This has the great advantage that access mechanisms are provided by interfaces to the programming language. Therefore, it is not necessary to program the access mechanism on one's own like it was done in the INRECA project. Here it was necessary because it was stressed especially to provide also a very efficient and effective access to the queried data. However, most data to be used in commercial CBR systems are already stored in conventional 232

databases. Therefore, in the context of the INRECA system also the following approaches need to be noted.

8.2.3.1.

Similarity Based Queries with SQL

Similarity based queries on databases would enhance the utility of data resources. The approach by Shimazu, Kitano and Shibata (1993) uses Standard Query Language (SQL) to achieve nearest neighbour matching. This is done in the context of a database management system, which does the data resource management. The system is called CARET (CAse REtrieval Tool). Similarity values can be used by defining an abstraction hierarchy within the database or by weighted sums of weighted local similarities of attributes. First they integrate so-called neighbour-sets for any attribute in the database. These neighbour specifying sets are valued with their specific similarity. To avoid overheads it is possible to define thresholds when doing an n-best match. Complex indexing schemes are difficult to maintain in standard relational databases. The system addresses ordinary system engineers in a corporation-wide quality control domain.

8.2.3.2.

Statistic Analysis with CLIPS

The Spotlight Data Exploration System (Anand & Kahn, 1993; fig. 8.1) helps users to cope with the large amount of marketing data generated by supermarket scanners. It delivers different types of information with different degrees of exploration depth, depending on the user's interests. Data are acquired automatically and stored in a large database. The tophead module to do the statistics is programmed in CLIPS. It is used to identify regional characteristics, machine faults, or threats on the marketing strategy. Mainframe market researchers databases

mainframe filter module PC-front-end communication link

PC database data retrieval functions

user interface

XPS-module

Fig. 8.1 – The Spotlight system's architecture

233

• The mainframe filter module extracts data about the target product and its key competitors in a selected region. It interprets how the key competitors and exceptional regions should be selected and which product segments are important for the current analysis. • The user interface generates pick lists of products and regions. • The expert system modules contain heuristics for analysing the data extracted by the mainframe, rules for selecting graphs to present data, and rules for generating text to describe the current analysis. • The library of data retrieval functions allows to query the database, which was downloaded to the PC-front-end for dynamic needs. This approach enables users to create reports with different contents without changing the heuristic for analysis. It is possible to change analytic knowledge without changing the generic presentation algorithm. That is why knowledge needed by the expert system module for data analysis is separated from its representation capabilities. They needed four person-years to create the front-end module to an existing database. The system can provide full executive overviews. The rule-based approach allows to give the user the flexibility to customise heuristic parameters. Everything in the knowledge base had to be validated manually. INRECA provides its own very efficient database. It is based on the CASUEL language (Manago, Bergmann et al., 1994) used for case-specific and general and domain knowledge description and exchange.

8.2.4.

Goal-Directed Queries as Tophead on Databases

Commercial applications of AI technology often entail integration between information systems already in use and tophead expert systems. In Milan Bonarini and Maniezzo (1991) developed an expert system that performed reasoning in the field of financial decision making based on data which are stored in conventional databases. It uses rules to do forward chaining to find inconsistencies during the initial plausibility control, and backward chaining for a goal-directed analysis of the given decision problem. The rules may be partitioned in several knowledge-bases and the user can define an additional rule-base where he decides how to use the different knowledge-bases. It was possible to implement the knowledge-bases for qualitative trend identification and automatic natural language report generation and context-sensitive data pattern recognition. The suggested rules were much more complicated than those previously included in the knowledge base. Their generation could be performed only by highly skilled personnel and it required substantial effort from them.

8.3.

CBR Systems with Cluster Trees

To bridge the gap between the difference of top-down and bottom-up categorisation, namely induction and causal models, a method was proposed in Hanson (1990) to cluster any kind of interrelated cases. On the one side, a large amount of preconstructed information (knowledge in234

tensive) is used while, on the other side, the featural similarity is analysed by a given set of objects. However, knowledge structures can be more general than objects and can contain more complex information than features. These are mainly: • Abstract concepts; • Actions; • Relations. The following approaches (cluster trees) are related to the discovery of category structure and the use of feature interrelations and their interaction with generalisation, inheritance, retrieval, and memory organisation. One main criticism of the induction approach was often that the classes of objects or events were clustered together without any kind of context, goal, abstracted knowledge, or any other information. The construction of complex knowledge structures seems to be the more potential approach. However, it reclaims a much higher amount of time to develop an application.

8.3.1.

Integrated Conceptual Clustering and Categorisation

The WITT project involves learning a category structure from little previous information outside some featural analysis and using a syntactic similarity based method to construct and refine new knowledge structures (Hanson, 1990). The clustering is realised by neighbourhood relations, when the primary access on features is determined with the entropy information measure to detect test attributes. In the clustering algorithm the following relations were used at the same time: • Centroid clustering of circles (fig. 8.2a); • Nearest neighbour clustering of circles (fig. 8.2b).

(b) Nearest neighbour clustering of circles

(a) Centroid clustering of circles

Fig. 8.2 – A clustering example showing the sensitivity of the solution to the group (class) membership rule

For further group determination the following methods are used: • The nearest neighbour group membership rule (fig. 8.3a); • The farthest neighbour group membership rule (fig. 8.3b); • The centroid group membership rule (fig. 8.3c). 235

(a) The nearest neighbour group membership rule

(b) The farthest neighbour group membership rule

(c) The centroid group membership rule

Fig. 8.3 – Clustering examples illustrating three different but commonly used methods for determining group membership

To determine the degree of intercorrelation of features to select the relevant features the feature to feature uncertainty relationis used:

∑ p( f | f i

j

) for all i ≠ j

This is what the researchers proved by implementing the WITT system: • Correlation of features are important for the support and elaboration of categories. • Models may be constrained by their category properties (constraint satisfaction paradigm: Holyoak & Thagard, 1987). • To bridge the gap between top-down inductive and bottom-up knowledge intensive approaches a step in the right direction needs to embed causal relations.

8.4

Induction

INRECA's induction capability is achieved by generating the indexing structure (k-d tree) in a special way. The current implementation allows to select from different approaches for choosing the attribute and its value for the next split. Due to this fact it was easy to add an entropy-based splitting criteria. Using this criterion for k-d tree generation results in a tree very similar to the one ID3 would produce. In the next section, we want to compare INRECA's induction capability to recent work in this field. We have a short look on approaches which do not base on the TDIDT idea. Then we will mention some improvements for TDIDT algorithms, especially concerning continuous-valued attributes. Finally, we will discuss induction-based knowledge engineering.

8.4.1

Non-TDIDT Approaches

Heath, Kasif, and Salzberg (1993) presented an induction algorithm based on simulated annealing. In contrast to INRECA which separates a set of points with axis parallel hyper-planes, their approach generates oblique hyper-planes to separate the different categories. To overcome the computational problems of finding this hyper-planes (NP-completeness) a randomised approach was chosen. Heath, Kasif, and Salzberg (1993) ran experiments which showed that their approach achieved high accuracy combined with small trees.

236

With SIA (Venturini, 1993) a multistrategy approach was presented which is based on a AQ-like algorithm in cooperation with a genetic algorithm. In contrast to earlier work in this specific field it allows to handle numerical and incomplete data.

8.4.2

Improvements on TDIDT Algorithms

From the numerous work on TDIDT algorithms we should mention some improvements proposed by Quinlan (1990). The article presents some addition to the original ID3 algorithm which allows to deal with uncertainties in classification. This is achieved by soft thresholds and a probabilistic method for handling unknown values and imperfect leaves. Compared to this INRECA's integration of CBR and induction provides a more elaborate way for handling unknown values. An interesting idea which could also improve INRECA's k-d tree was presented in Fayyad (1994). The paper focuses on the problem of choosing values to branch on. A method called attribute phantomisation is presented which maps irrelevant values to a single default-branch. Thus, smaller trees can be generated while the accuracy is increased, as Fayyad proved by experiments. To reduce problems typically appearing while using greedy algorithms for tree generation, Wallace and Patrick (1993) present their implementation using a look-ahead for attribute selection. Slightly better results could be achieved, but on the cost of a significantly higher computational effort.

8.4.3

Continuous-Valued Attributes

As the performance of ID3-like algorithms dramatically depends on the number of continuousvalued (e.g., integer, real) attributes, a lot of literature dealing with this problem can be found. ID3 handles numerical attributes by calculating the entropy for each candidate cut point. To do so the examples must be sorted first which leads to a very high complexity (Remember that this calculation must be done at each numerical node of the tree). The second objective for changing ID3's continuous-value handling is that the algorithm performs quite bad if the values for one class lie in an interval. Most of the suggested approaches base on a discretisation of continuous values. Catlett (1991) suggests to calculate more than one cut point based on entropy. Thus, splitting on the same attribute more than once can be avoided. This results in a large saving of CPU time while sometimes also the accuracy is improved. The criteria when to stop discretisation can be derived from the minimum description length criteria as shown in Fayyad and Irani (1993). Further problems related to binary splits are pointed out in Van de Merckt (1992). Using a clustering-based selection method for the split point, robustness against noise can be increased. Even more improvements appear if this method is extended to a discretisation method, as shown in Van de Merckt (1993). The clustering algorithm used here is derived from the Error Sum of Squares which is used to find a contrast-based cut. Experiments on different domains reveal that

237

the presented approach increases the prediction quality especially if the target concept is partially presented and highly disjunctive. A different method to attack the above mentioned problems in large, noisy domains is presented in Catlett (1992). The method called peepholing applies two main ideas: shortlisting and blinkering. The aim of shortlisting is to scratch out non-promising attributes. Blinkering is an elaborate method to narrow the amount of possible splitting points on bases of small sub-sets from the whole database. Empirical results show that peepholing nearly reduces the order of ID3 from super-linear to approximately linear. This substantial reduction allows to use induction algorithms on datasets with far more than 10.000 cases. In experiments on INRECA with different domains we also recognised a relation between the number of numerical attributes and the building time for the k-d tree. Due to this fact future work on INRECA will have to include some of the above mentioned techniques. Especially the clustering method used by Van de Merckt seams promising as it is quite near to the methods INRECA uses for the selection of splitting values.

8.4.4

Random Splitting

Mingers (1989) considered a number of different splitting measures. He came to the surprising conclusion that randomly chosen attributes do not significantly decrease classification accuracy. By carefully reviewing Mingers experiments it could be shown that this conclusion was wrong (Buntine & Niblett, 1992). The reason for this was pointed out in Liu and White (1994). Further experiments discovered two factors which influence the degree of accuracy decrement. The first factor is the greater chance to omit informative attributes while the second is the risk of overfitting the tree with non-informative attributes. Only if the regarded domain consists of attributes of equal importance the random selection method may produce trees of comparable accuracy with minimal computational effort. Due to this we decided not to implement a random attribute selection for INRECA.

8.4.5

Induction as Knowledge Acquisition Technique

Due to significant performance improvements described above it is now possible to utilise machine learning techniques in fields like knowledge acquisition or data mining. For example, Yasdi (1991) applied a learning technique to learn classification rules from a database. He used rough sets to design the learning algorithm. A group at General Motors applied an outgrowth of ID3 to a database with about 300.000 cases of vehicle symptoms (Uthurusamy, Means et al., 1993). While developing this system (INFERRULE) they discovered a property of data not yet addressed in the literature: inconclusiveness. This states, real world cases often contain attributes which are not sufficient to exactly classify an example1. Since ID3 does not perform well on

1

However, see Richter and Wess (1991): The approach taken here (i.e. in the PATDEX/2 system) is to choose an appropriate class of similarity measures („Tversky-type measures“: Tversky, 1977) and to automatically improve them with a

238

inconclusive data, certain changes needed to be made. INFERRULE has been deployed on sub-sets of the mentioned database and seems to provide a better way for handling inconclusive data. Finally, we want to have a short look on an interesting issue presented in Tsujino, Dabija et al. (1992). The system KAISER - a knowledge acquisition tool - detects improprieties in domain knowledge. From the induction point of view, the idea to reveal structural improprieties by evaluating decision trees is most important. For example, inseparable examples, similar classes etc. can give the domain expert hints how to improve his knowledge representation. Especially the last article shows an interesting domain INRECA could be applied to. With its database capabilities and the different measures and splitting techniques incorporated, INRECA could easily be extended to discover implicit knowledge in existing databases.

8.4.6.

Decision Tree with Hyper-Planes

Standard decision tree approaches such as ID3 set points with axis-parallel hyper-planes. Here a method is presented (Heath, Kasif & Salzberg, 1993) to find hyper-planes at any orientation. The simulated annealing algorithm finds a hyper-plane to partition the training set recursively in two partitions each. This is done in a stochastic approach. Applications are cancer diagnosis and identification of galaxies, where the approach worked especially good and created trees with a low depth.

Fig. 8.4 – Tree generated by ID3 and SA

competitive learning strategy (Richter, 1992; Althoff, Wess et al., 1994). By this procedure it is possible to discover incomplete cases that have only been a good guess. In addition, the revise mechanisms used in the PATDEX system and the INRECA system can be used to quickly discover incomplete cases by temporarily canceling „wrong“ cases (section 6.4, alg. 6.16).

239

8.4.7.

Decision Tree with a Contrast Measure for CutPoints' Selection

In numerical attribute spaces the combinatorial complexity of finding the correct disjunctions of the target attribute is a highly complex task. In Van de Merckt (1993) an approach is proposed using heuristics based on numerical similarities among the training instances. It uses an unsupervised heuristic issued from cluster analysis for the selection of the generalisation steps. The algorithm is like TDIDT for building the classification rules. The idea is to search for clusters that are as much contrasted as possible from the instance space proximity point of view, even if they contain elements belonging to distinct classes. The principle is to rely on the similarity hypothesis and to select in an unsupervised way (without looking at the class labels) the cut-points which produce the more contrasted partitions. The following methods have been used: | A||⋅ B| ⋅|| x ( A) − x (B )|| | A ∩ B| − x ( A) is the vector of average values of the elements belonging to the sub-set A, where A, B represent the present set of features. | Ci |⋅| C j | • Monothetic Contrast (Ci,Cj,Ak) = | C ∩ C | ⋅ ( mki − mkj ) i j • Edwards-Cavalli Criteria =

− where Ci,Cj are the resulting binary partition, mki is the mean value of Ci for the attribute Ak. Contrast (C i , C j , Ak ) • Entropy weighted Monothetic Contrast (Ci,Cj,Ak) = Entropy(C , C ) i k According to the researchers publications these are the main results of their work: • Using unsupervised heuristics may be useful to find disjunctive descriptions, especially in numerical attribute spaces, where the combinatorial complexity of finding disjunctive attributes is sufficiently high. • The use of similarity measures is less important for sufficiently large training sets. • Selecting cut points based purely on attribute similarity without looking for class information may lead to build overly complex trees with less predictive power. • This approach is highly effective when target concepts are partially represented or when they are highly disjunctive and weakly conjunctive.

8.4.8.

Binary Decision Trees with Bayesian Diagnosis

Both, the expert's knowledge about the base rates of the classes and about the conditional probabilities of the data within the class can be expressed by beta probability distribution. The Bayesian point probability that a single case belongs to a given class is treated as a parameter. Its probability distribution is derived and called „credibility distribution“ (Kleiter, 1992). Because the numerical treating of the distribution is difficult, an approximation by a beta distribution is proposed. The credibility distribution expresses the impression of a diagnostic probability. In 240

cascaded inference, the total degree of diagnostic expertise is not additive, but recursively discounted at each level of inference. The Bayesian theorem including parametrisation, probability distributions and statements of assumptions may be expressed by the following formula for hypothesis H and symptom sets D:

p( H |D ) =

p( H ) p( D| H ) p( H ) p(D| H ) + p(¬H ) p( D| ¬H )

It is that the data generating process concerns the structural and probabilistic model of the process that generates the expert's experience. The expert's belief is expressed by subjective probability distributions. The base rates which specify the percentage of cases belonging to the classes are expressed by beta distributions. The symptom information specifies the percentage of symptoms present within a class. It is presented as conditional probabilities. The researchers had to cope especially with the following properties of the implemented method: • Combinatorial explosion of the full problem space when too many symptoms are used. • By presenting only sub-sets to the system and ... • By approximation of confidence intervals it is possible to enlarge the systems capabilities.

8.4.9.

Binary Decision Trees with ID3 and Entropy

Since most real world applications of classification learning involve continuous valued attributes, the discretisation process is an important problem. This is done like in the following method: For each evaluation of a cut point the data are partitioned in two sets and the class entropy of the resulting partition is calculated. Then the one with the minimum class entropy is selected, as described in the following: • Let T partition the set S of examples into sub-sets S1 and S2. Let there be k classes C1,...,Ck and let P(Ci, S) the proportion of examples in S that have the class Ci. The class entropy, which specifies the needed information in bits, of the sub-set S is defined as: n

Ent (S ) = − ∑ P( Ci , S )log 2 ( P( Ci , S )) i= 1

• The class information entropy of the partition induced by T is: E ( A, T; S ) =

| S1 | |S | Ent (S 1 ) + 2 Ent (S 2 ) | S| |S |

A binary discretisation is determined by selecting the cut-point T with the minimal value. The training set is sorted once and then the algorithm is recursively applied. This method is suboptimal. Therefore, in Fayyad and Irani (1993) an approach exists for rejecting a given partition if the costs are too high. It is called the MDLPC (Minimum Description Length Principle Criterion). • The partition induced by a cut-point T for a set S of N examples is accepted if: 241

Gain( A, T; S ) >

log 2 ( N − 1) Δ( A, T , S ) + N N

• When Gain (A,T;S) = Ent(S) - E(A,T;S) • and Δ(A,T;S )= log2(3k-2)-[kEnt(S) -k1Ent(S1)-k2Ent(S2)] The main results of the work considered by the researchers themselves are: • The efficiency can be increased without changing the final outcome when using attribute values' intervals. • It is possible to derive better decision trees from the same data, compared to decision trees with no strategy, random cut or always cut strategy, which cuts until all examples of one class are clustered in one partition.

8.4.10.

Decision Tree with Failure/Success Statistic for Information Retrieval

Conceptual information retrieval systems use structured document indices, domain knowledge, and a set of heuristic retrieval strategies to match the user's query with a set of indices describing the document's content. DEDAL (Baudin & Pell, 1994) is such a conceptual information retrieval system for a mechanical engineering design domain which uses structured patterns rather than keywords or words from a text. E.g., if the system is looking for a document about hybrid engines but there is no perfect mach on this keyword, then the system can look in a keyword hierarchy for more abstracted keywords of this type. In this example the new query keyword may be the more general expression motor, which may be found in a domain abstraction hierarchy. There may be also the possibility to store a heuristic rule that information about hybrid engines is often found nearby documents about newly invented locomotion techniques. DEDAL stores multimedia mechanical design documents such as meeting summaries, pages of a designer's notebook, technical reports, CAD drawings, and video-taped conversations between designers. • In order to refine retrieval heuristics first examples of query/retrieval pairs, generated by successful and failed retrievals, are gathered. • Successful and failed retrievals are stored in a set of concepts that discriminates positive and negative examples. • These examples are the input information for an inductive decision tree generation algorithm (ID3) that refines the set of heuristics.

242

query

exact match

no

retrieval strategies

decision tree

query/index

yes positive ? retrieve assign high priority

assign low priority

Fig. 8.5 – Using induction to refine retrieval heuristics

To use ID3 the examples, generated by the heuristics, had to be processed to select features that could be used to discriminate positive and negative examples and to remove noisy examples. In DEDAL three different types of features were used corresponding to three levels of generality: • Documentation dependent features such as a particular reference to a segment of information. • Domain dependent features: These are elements dependent on a particular context. • Task dependent features: They depend only on the type of information retrieved. The main results can be summarised as follows: • It is necessary to create a decision tree for every level of generality. • The branches of the decision tree pointed out flaws in other parts of the system such as the indices themselves or the domain model (in the interactive mode). • Using relevance feedback the DEDAL system improves the retrieval's effectiveness in noisy domains.

8.5

Diagnosis

The main steps of diagnosis are suggesting a fault on the basis of a problem description, predicting its consequences and checking whether the expected consequences are confirmed by observed or inferred problem findings (Aamodt, 1991). Many problems in the industry are diagnosis type problems, being either troubleshooting or data interpretation problems. In the mid 80's many approaches in artificial intelligence and cognitive science were caried out to develop generic models of diagnostic processes (Chandrasekaran & Mittal, 1983; Feltowich, Johnson et al., 1984; Clancey, 1985; Bennet, 1985; Sembugamoorthy & Chandrasekaran, 1986; Keravnou & Johnson, 1986; Puppe, 1987; Fink & Lusth, 1987; Breuker, Wielinga et al., 1987). Diagnosis is unequivocally a classification task (also section 9.3). Given a set of symptoms the system needs to determine the problem. Case-based diagnosis systems (like also INRECA) try to retrieve past cases that are similar in nature to the input symptom list and suggest diagnoses based on the diagnoses from the best retrieved cases. The current CBR tools available on the market have been designed to solve diagnostic tasks and have produced the largest number of fielded applications. 243

Not all diagnostic tasks are the same, however. In some diagnostic domains, all of the information is available at the beginning of the retrieval process. In this situation, the system only needs to retrieve cases once, since no additional data will be collected that would change the best matching cases. In other domains, the initial information about the problem may be sparse or incorrect. In these domains, the system performs case retrieval over several iterations or will query the user for information along the way to retrieve good cases. As more information is gathered, a diagnosis can be confidently presented to the user. Some domains have a well defined set of diagnostic features that tend to be well filled out at runtime, other domains have very large feature sets that are very sparsely populated at runtime. Clearly the latter situation will present difficulties in representation and indexing that must be addressed. Here are some of the issues that present themselves in general diagnosis and fault recovery systems (Althoff, Auriol et al., 1995a). • Diagnosis in domains with structured case representations. Diagnosis in domains like medicine require working with complex patient records. Patient records have important diagnostic information in them that cannot easily be represented using a flat feature vector. For example, the patient record will have history information on the patient as well as chronological information on recent doctor visits or test results. This structured and chronological information is important in performing good diagnosis. In order to use these types of structured representations with existing CBR tools, developers are usually forced to aggregate the structured patient record into a single feature vector representation for indexing and retrieval purposes. Chronological information is summarised into single aggregate features like averages, maximums, slopes, etc. While the aggregation process seems to work well, it is admittedly going to lose potentially vital information in the summarisation process. • Multi-phased diagnosis. Sometimes, in order to simplify the case representation issues, a case-based diagnostic system will use a multi-phased approach to performing diagnosis. The first phase of the diagnosis is to determine the general diagnostic category for the input symptoms. By eliminating the general diagnostic categories that are not appropriate for this case, the system can focus on a more detailed representation in the subsequent phases to retrieve cases with better overall accuracy. This approach simplifies the case representation task and allows cases in the same diagnostic category to be stored together with their own unique representation and indexing mechanism. • Diagnosis and recovery as two classification tasks. Once a diagnosis is determined the diagnostician must then choose a course of action for fixing the problem. Determining the recovery action is also a classification task that can be addressed with the same cases as were used to perform the initial diagnosis. In fact, the diagnosis is used as an important indexing feature in determining the appropriate recovery action. Case based fault recovery systems first determine the diagnosis using the symptoms to find relevant cases with a case index that is tailored to the diagnostic outcome feature. Then they take the proposed diagnosis from the first step and look for recovery actions by retrieving cases (from the same library) using a case index that is tailored to the recovery outcome feature in the case.

244

• Integrating other available domain knowledge into the CBR process. While cases are often a well-suited form of knowledge to use in a diagnostic system, it is by no means the only source of knowledge that is available in most situations. Many times there are well-established rules and procedures for handling many of the most common symptom signatures. There is no need to use cases if the rules are well defined for certain common circumstances. In other, rarer instances, a model-based system is in use that can typically handle a larger set of problems than collections of rules. In either event, a hybrid system can be built that uses the rules or model to handle the commonly seen problems and the cases can be used for the situations that fail to fall into the knowledge area that is encoded in the rules or model (also section 1.3).

8.5.1

Ambient Conditions of Commercial Diagnosis

The technological progress and the usage of electronic systems lead to very complex technological systems. A lot of variants of a certain type of a machine are produced and need to be diagnosed for the purposes of error detection and maintenance. This can be implemented by a modelbased approach using object libraries and also by a heuristic approach (Weis, 1995a). WDS1 is a hybrid commercial diagnosis shell developed by Daimler-Benz AG and DANET GmbH used for diagnosis in technical domains (Waleschkowski, Schahn & Forchert, 1995). Three main aspects influenced the development of the WDS system: • Parts of the technical knowledge were already available on computers. • The diagnosis should be clearly related to objects. • Further knowledge should in general be automatically generatable. The WDS system consists of three sub-modules: • The model-base development kit. • The run-time diagnosis system. • The revision system which revises the heuristic knowledge base using feedback information. The development and the usage of the system must be seen within the framework of the following figure 8.6:

1

Wissensbasiertes Diagnosesystem.

245

engineer data and information evaluation

knowledge acquisition

Central office

knowledge-bases

error messages, sensor data

technical application

runtime system

protocols

tests, results

vehicle identification symptoms

after-sales service

Workshop customer

mechanician Fig. 8.6 – Framework of information flow for the development and employment of a diagnosis system

It became an important item for the industry to ensure the quality of products and that way also the quality of internal and external processes. All different strategic and organisational parts of the enterprise work together and rely on information about current processes (fig. 8.7). Guarding the processes became more and more difficult with the ongoing rationalisation and automisation of production processes. It is important for knowledge-based systems which work on several processes that they can communicate with one another in the case of needing information about situations that happened earlier. This is especially important in conspicuous or faulty situations. The aim is the analysis of these situations to find causes and typical occasions in which they occur. If the problem situation already happened a repair solution needs to be found. In Pfeifer, Grob and Klonaris (1995) a framework for knowledge-based systems technology can be found which can cope with the entire scope of problems which can occur in usual enterprises (fig. 8.8). The central point of view is the information flow between the processes.

246

after-sales service assembly

customer advisory service report analysis

failure analysis stock/ shipment

test

concept

knowledge base statistic process control

plan generation

construction

planning realisation

supply

Fig. 8.7 – Organisational parts of enterprises which need support of knowledge-based systems (adapted from Pfeifer, Grob & Klonaris, 1995)

It is important that necessary information is at the right place, and in time. Therefore, the different programme modules, which were designed to solve special problems, need to communicate with each other. This information flow integrates also related modules which can provide or demand knowledge, information or data like, e.g., CAD/CAM-, CIM-, CNC-, AMR systems or production process planners or controllers. Therefore, general interfaces need to be specified which provide the possibility to send information in a unified representation.

247

quality report preparing compression quality data

registration

error detection knowledgebase

success control

failure analysis

weak points

finding measures taking measures measures

Fig. 8.8 – A framework of commercial failure analysis (adapted from Pfeifer, Grob & Klonaris, 1995)

To solve the existing problem, detailed models of the production lines, the production processes, and the resulting product have been implemented. The basic knowledge acquisition activities are described in the following figure 8.9.

248

key data

: means, that the upper knowledge is a source for the lower one.

machine description production process attaching functional units to elementary works product structure machine faults causal structure confidences abductive rules therapies tests Fig. 8.9 – Steps in the acquisition of knowledge

1.

Key data: A clear description of the production process, of the person responsible, and the machine.

2.

Structure tree of the machine: The components of the used machines are recursively retrieved on different levels of abstraction.

3.

Description of the production process: Determination of elementary processing steps to be done and of the produced intermediate and final products.

4.

Every elementary processing step is associated wit the corresponding functional units, i.e. the respective machine components.

5.

Structure tree of the products: All existing products in the production process are decomposed into their components.

6.

Definition of faults of the product components: Every product may have specific attributes. All relevant features of a product component need to be systematically retrieved that may specify failures in the respective production machine's component.

7.

The causal structure: It determines relations between causes and consequences.

8.

Confidences: They define the relevance of the relations and help to find also combinations of plausible failures.

9.

Evaluation of abductive inference: Where and up to which degree can implications be used like equivalencies.

10. Description of repair measures for the determined failures. 11. Description of tests: To increase the quality of the result of the analysis, the test to be done needs to be described in detail.

8.6

Model-Based Diagnosis

Model-based expert systems use explicit knowledge about the structure of the „world“. In diagnosis domains, this world is the device or system which is diagnosed. These models are used to

249

predict or to evaluate the behaviour of the device in a given situation. The behaviour of a device can be modelled in two different ways: First, a model can consist of logical states and their causal dependencies. In general, these states have no explicit relationship to any physical or functional unit of the system, but they describe aspects of the system's behaviour as a whole. Some of the causal dependencies are references to observable attributes. Another type of models divides the system into physical or functional units. The behaviour of the system is then a result of the behaviour of the single units and their functional or physical connections. Most models of this type are hierarchical, so that the units of the top-level model are described in a model of smaller units, down to some kind of elementary unit. These elementary units can either be primitive functional components or „smallest replaceable units“ (SRU's). The behaviour of these elements is represented by mathematical expressions. The quality of a model depends strongly on its degree of detail and exactness. A precise submodel can usually give a better diagnosis than a high-level model of a complex unit does. However, the degree of detail is not only limited by hardware and software resources but also by the problem of uncertainty and inexactness. While a digital system can be easily modelled down to single logical grids (as far as this makes sense), this is not possible for most analogue devices. Very often the function of a component cannot be predicted by exact calculations, because they cannot take into account any kind of noise. Many model-based diagnosis systems work around this problem by replacing exact calculations by a qualitative calculus like QPE or QSIM. Another classification criterion for model-based diagnosis systems is the distinction between fault models and models for the correct behaviour of a system. If a model describes the correct behaviour of a system, the diagnosis engine can name the component of the device which does not behave as specified in the model. If a given situation can be explained by a fault model, the model can also give information about the cause of the fault or on how to repair it. However, fault models can only consider faults which are known in advance. The compromise used by some systems consists of modelling the correct behaviour of the device, while storing fault models for the most common or most probable predictable faults. The diagnosis process itself is started with a set of observed symptoms as its input. In a first step it tries to find one component in the model which has a (direct or indirect) causal influence on all the given symptoms whose value is abnormal. The assumption behind this step, which most model-based diagnosis system make, is that only one component of the system is faulty at one time. This first search can yield several possible causes for the observed failure. In the next step these hypotheses must be verified. One possibility to do this is to start a simulation of the system. The hypothesis, which assumes that a certain component behaves abnormally, is propagated through the causal links of the model. If the result of the simulation matches the observed situation to a certain degree1, the hypothesis is a possible diagnosis for the given situation. Otherwise, it is rejected. Another strategy consists of making additional observations along the paths in the model between the hypotheses and the abnormal symptoms. If one of these newly observed symptoms shows a correct behaviour, the corresponding component cannot be along 1

In a qualitative calculus, this means: within a certain interval.

250

the path from a hypothesis to an abnormal symptom, which means that such a hypothesis is incorrect and can be rejected. Of course, these strategies can also be combined. As a final result, the remaining hypotheses (in the optimal case there is only one) are given as output.

8.6.1

Some Example Systems

The approach described in Console, Portinale and Dupré (1991) uses a state-oriented fault model. The behaviour of the given device (in this example: an engine) is modelled by a set of logical states which can be represented as definite clauses, while the cause-effect relationship is modelled by an implication. One path in the example model is, e.g., irreg_oil_comsumpt → oil_lack → oil_gauge (red). In this path, only the last expression denotes an observable state. The states of this model form an acyclic directed graph whose root nodes or „initial state“ represent the causes of faults. The diagnosis process is a search in this graph for an initial state with a path to all observable states which show an abnormal behaviour. As a search of the whole graph is very inefficient, the approach in Console, Portinale and Dupré (1991) contains a method to restrict the search space. This is done by associating every node in the graph a formula representing a necessary condition for the state denoted by the node. These conditions are formulated in terms of observable values and are, thus, easy to verify. They are generated by a compiler which only uses the information coded in the model. So, this approach is an example for the use of compiled knowledge in diagnosis. However, the compiled conditions cannot determine the diagnosis directly, but they can reduce the number of hypotheses by eliminating those which certainly do not apply. A different kind of compiled knowledge is used in the Intelligent Alarm Processor (IAP) described in Pfau-Wagenbauer and Nejdl (1993). Its field of application is an electric power distribution network, which has only a few kinds of different components. The knowledge about each of these components (like, e.g., circuit breakers) is compiled into a set of rules. These rules map observable values to states, which can be „ok“ or „abnormal“. The results of these component rules serve as input for another set of rules in a higher hierarchical level, in which the components are grouped into larger devices like switching fields. This second level also contains heuristic rules to represent knowledge that cannot be coded in the model itself. Finally, there is a third level of model-based and heuristic rules which produces the diagnostic output. The approach in Vinson, Grantham and Ungar (1992) is an example for the usage of qualitative reasoning in model-based diagnosis. This approach has been implemented in the DISARM system (DIagnosis System with Automated Rebuilding of Models). DISARM's model of the diagnosed device (here: a chemical plant) consists of measurable values such as temperatures, material quantities, or reaction rates. These values are represented by qualitative states which describe their location relative to landmarks such as boiling point or maximum pressure, and their first derivative. The links between these values indicate an influence which can be positive or negative. During the diagnosis, DISARM first searches for a unit which has an influence on all the detected abnormal values. Then the measured values are propagated through the model along the paths 251

that affect the suspicious unit. DISARM now tries to change the model of this unit in order to get an explanation for the misbehaviour. This can be done by removing variables and links from the model or by adding some from a library of fault models. E.g., if a leakage is detected, a model for a material flow is inserted. These changes in the model have the advantage that the system is still able to diagnose new faults before the first one has been repaired.

8.6.2

Monitoring and Diagnosis

A behavioural model of a device is not only suitable for a model-based diagnosis. The information stored in such a model is also needed for monitoring tasks. In this case the model is needed to determine, whether the device currently behaves correctly, or not. If it does not, a diagnosis process is triggered. Thus, monitoring and diagnosis systems are often linked together very closely, as it is the case for the IAP system (Pfau-Wagenbauer & Nejdl, 1993), which receives messages from a monitoring system (SCADA - Supervisory Control And Data Acquisition). In some other cases, both tasks are accomplished by one tool, as the required data is the same. One of these systems is the MIMIC system (Dvorak & Kuipers, 1991). It uses a model based on QSIM, which can handle semi-quantitative values, i.e. positions relative to landmarks and consistency constraints. In the monitoring stage, MIMIC tries to predict the behaviour of the system for the next period of time. As such a prediction is often ambiguous, MIMIC stores several instances of the model, each one representing a possible prediction. In the next time cycle, these models are compared to the new observations. If there are inconsistencies, the instance of the model is discarded, otherwise, the system continues by predicting the next step. However, before an instance of the model is really discarded, a diagnosis process is run on it to bring it into agreement with the observations again. During this diagnosis, MIMIC tries to identify a component which is responsible for the inconsistencies and tries to find a state of the component which removes the inconsistencies. This diagnosed state can denote a correct (but unpredicted) behaviour or a fault. In both cases, the model can still be used to continue tracking the behaviour of the device. If the diagnosis fails with this instance of the model, MIMIC assumes that the prediction that produced this model was not correct. In this case it discards this one and continues tracking the other instances. Another QSIM-based monitoring and diagnosis system is the DIAMON (DIAgnosis and MONitoring) system described in Lackinger and Nejdl (1993). This system uses a hierarchy of models so that it can handle more complex devices. This hierarchy is mainly used for the diagnosis. The monitoring process only uses the top level of the hierarchy. In this model, the correct behaviour is described by a set of constraints. During the monitoring process, DIAMON checks, whether the observations of the most important parameters fulfil these constraints. If this is not the case, the diagnosis algorithm is started. This algorithm examines the constraints in the deeper models, together with additional observations in order to find the component which behaves incorrectly.

252

8.6.3.

Model-Based Information Retrieval for Design Tasks

DEDAL is an interface that facilitates the reuse of design experience of engineers by providing an intelligent guide for retrieving text, graphics, and videotaped documents (Baudin, Underwood & Baya, 1993). It uses concepts from a semantic model of the designed artefact to index the design documents as also a set of heuristics that reason from the model if the retrieval fails. The domain knowledge was built by specifying a language to index the design information. The domain model includes a representation of the artefact structure, its function and qualitative relations used by the designer to model the device. Main relations are: isa, part-of, attribute-of and depends-on. Heuristics work causally on model-rules and spatially in the universe.

8.6.4.

Symbolic Models for Diagnosis

There are a lot of different approaches to build an effective diagnostic system. One of them is to design a simulation system and to look for differences in the models and the real world's behaviour. De Kleer (1992) proposes a formal representation (not implemented) of a straightforward or straight-backward search for inconsistencies in a symbolic model which represents the real world application. The diagnosis is found, where the difference of the model and the real world's behaviour has its origin. The approach is based on the framework of Reiter (1987) for minimal diagnosis with symbolic models:

Def.: A system is a triple (SD, COMPS, OBS), where SD is the system description, COMPS the system components' constants description and OBS the set of observations. The algorithm is based on the propagation of observed values and finding contradictions to the system's normal behaviour in a substitution and unification approach to realise resolution by subsumption as sentences are represented by clauses (Richter, 1989).

8.6.5

Model-Based Diagnosis and Repair

A relatively new approach in model-based diagnosis consists of integrating the repair process in the diagnosis process (Friedrich, 1993). This approach applies to systems with very high breakdown costs, in which therefore repair actions must be started as soon as possible, sometimes even before the exact diagnosis is known. Repairing a device is not an analytical task anymore, but a planning problem. Based on first hypotheses, a diagnosis and repair system builds plans to re-establish the purpose of the device. The parameter which is used to choose one of these plans is an overall cost factor which includes the costs for additional observations, the breakdown costs, and the repair costs.

253

8.7.

Real-Time AI Approaches

There are many real-time applications in several engineering domains, such as • Process control; • medical monitoring; • on-board diagnosis. Therefore, a system needs to be created which can always react to get further information which can be useful to resolve problems. The system needs to be able to respond at any time and, thus, the complexity of elementary tasks needs to be very low. Tasks need to be programmed in a way that they can be interrupted at any time without damaging the performance of the system. The requirements for task simplicity are that the data and the knowledge should be reliable. Static or unreliable knowledge could be described by certainty factors. The solution space should be small. This corresponds to the requirements of diagnosing engineering systems.

8.7.1.

Monitoring and Diagnosis with AND/OR-Nets

Morizet-Mahoudeaux (1992) describes the so-called SUPER system (fig. 8.10). It was developed for maintaining and updating information about an evolving system, as its states change due to abnormal effects such as faults or as a consequence of external actions taken on the system. A knowledge acquisition module was developed and an inference engine which can work on an incrementally increasing number of rules (fig. 8.10).

transducers

database

knowledgebase

diagnosis and monitoring module

knowledge acquisition module

USER INTERFACE Fig. 8.10 – The real-time expert system SUPER

254

The system development followed the assumptions that most of the industrial processes can be monitored by a system following a logical model like the one shown in the following example (fig. 8.11): gaz tank full

pump okay

AND

ignition switch okay

battery okay

AND

ignition switch dead

battery dead

AND OR

gaz injection

engine rotates

no spark

ignition coil dead

engine does not rotate

AND OR starting fault electricity Fig. 8.11 – A simple example of a network of rules

The proof for the determination of a diagnosis is done by a backward planning algorithm. The proof can only be stopped by three events: • An exceptional event: Then the system goes back to the data update step. • A failure: Then the system looks for the next goal to be proved. • A success. The efficiency of the resolution algorithm has made it possible to implement the system in a real-time multitasking environment.

8.7.2.

Real-Time Decision Support Modules with Constraint Satisfaction

The growing capabilities of information gathering systems in the submarine combat environment (Benjamin, Viana et al., 1993) continues to generate interest in real-time decision support modules. The Manoeuvre Decision Aid (MDA) is a knowledge-based decision support system which has many similarities with the navigator architecture for Autonomous Mobile Robots (AMR; von Puttkamer, 1993). It is a knowledge-based planner which was developed in six person-years at the Naval Undersea Warfare Center, Rhode Island. The model of the manoeuvre generation progress decomposes into three main parts: 255

• Situation assessment (or sensor data elaboration in AMR). • Goal formulation. • Constraint satisfaction (or navigation in AMR). It is essential for an automated reasoning system in this domain to consider all factors and concerns in a given situation in order to put forth its best recommendation (representing the Pilot architecture in AMR). Each of the goals identified in a situation directly leads to a set of constraints on the four motion variables: • • • •

Course; Speed; Depth; Time.

The typically over-constrained nature of the problem is countered by propagating the priorities of manoeuvre goals to produce rated constraints. Initial constraints are posted on a so-called blackboard with their individual ratings. These are combined by conjunctions creating and rating so-called constraint regions in the resolution space. For these regions functions are predefined. The aim of the last stage of the algorithm is to combine all these region functions derived from the rated constraints to build a collective region function representing the actions to be done. To build this set of mutually exclusive region functions all non-mutually exclusive region functions must be split individually to receive non-contradictory solutions. Before intersection:

After intersection:

Fig. 8.12 – 2-d analogy for combining regions

The rated constraint combination technique allows the implementors to build knowledge structures in a format similar to the ones given by the experts. The blackboard architecture has provided the ability to represent and develop the solution process in a modular and incremental manner. The organisation of knowledge sources allows the incorporation of existing expert systems to support the difficult tasks of strategic situation assessment and target motion analysis with Ljapunov functions (von Puttkamer, 1993).

256

8.7.3.

Symbolic Models for Real-Time Applications

Hydro Quebec has developed the SCADA system (Bernard & Durocher, 1993) to perform a continuous analysis of alarm messages, automatic detection of protection or restoration control and then to produce a real-time diagnosis identifying the origin and the consequences of the fault of modern electric power systems. The alarm messages contain the following information: date, time, location of the system, system's component, description of the abnormal behaviour and its current state. The alarm messages are taken as symptoms of events occurring in various facilities from which the operator can deduce the resulting status of the equipment here. Hypothesis generation based systems take these symptoms as a starting point for constructing a detailed model of the power systems behaviour, in order to determine the exact cause of the events from which the equipment status can be calculated. This task is extremely difficult in a real-time context. To go directly from the symptoms to the equipment's states it uses a high-level power system model which establishes direct symbolic relations between the equipment, the alarms, and the behaviour of the automatic controls. This added level of abstraction makes it much easier to analyse alarm message sequences. There are also rules included to account for the possible abnormal operating of the control and possible noise in the alarm messages. What the system does with the model and the provided rules is a symbolic pattern match and a forward chaining of the related rules and the model-internal relations.

Read real-time alarm messages

Create or complete diagnosis LOOP Output diagnosis, if any

Eliminate obsolete messages and diagnosis Fig. 8.13 – The real time main event loop of the SCADA's diagnosis system

The main results of Bernard's and Durocher's (1993) work in the real-time context are: • It is possible to establish a model-based system in a real-time domain. • The forward chaining of the symbolic values, done by a system in OPS5, created many difficulties and drawbacks during the implementation, but it provides a remarkable efficiency.

257

8.8.

Rule-Based Approaches

During the last years the MOLTKE expert system has been developed at the University of Kaiserslautern as a platform for various advanced rule-based diagnostic techniques. It represents a static diagnostic framework as (partially) described in section 1.3 (also Althoff, 1992a; Pfeifer & Richter, 1993). Here approaches will be presented which extend the functionality of the MOLTKE system, which where developed in different basic contexts. Since the INRECA project bases to some degree on experiences made and work done in the MOLTKE project, the following approaches are also complementary for INRECA and point out some intesresting reseacrh directs that might be of interest for the future development of INRECA.

8.8.1.

Integrating Time Dependent Symptoms in RuleBased Systems

For various reasons many expert systems do not explicitly represent time. This may be a useful abstraction for many applications. However, some symptoms evolve over time in a characteristic manner and require several measurements that must be carried out in a particular temporal order. In Nökel and Lamberti (1991) such symptoms are called Temporary Distributed Symptoms (TDS). The authors described methods to integrate a temporal matching algorithm into the – otherwise static – framework of the MOLTKE workbench. In MOLTKE a diagnosis in this context is done by prevision and compare, realised by straight-forward propagation of rules. To integrate TDS's a temporal matcher was implemented, which carried out the following task if it was provided with the specification of a TDS as input: • Plan expected values for individual measurements. • Compare predictions and measurements and check for earlier similar situations. Therefore, a new type of test was implemented for TDS's because time-independent symptoms still used the same matcher. Once the symptom has been selected, it is marked as in progress and the user can actually return at any time to retest the symptom. According to Nökel and Lamberti (1991) their approach has the following main properties: • The integration is effective. • It does not lead to a computational overhead. • Temporal matching may improve also the discriminating power of model-based systems.

8.8.2

Tracing Rule Chains in Qualitative ConstraintLinked Domain Models

The two most prominent conceptual models for diagnosis are (Bourne, Liu et al., 1991): • Encoding empirical associations between observable behavioural attributes and known causes of a failure. 258

• Comparing observed and predicted behaviour based on qualitative or quantitative simulations of the entity studied. As integrated case-bases grow large (Brown, 1992), exhaustive search of al previous cases seems impractical even for parallel architectures. This is the purpose for which the activation passing retrieval mechanism of the CBR-tool CRASH was developed (Brown, 1992). A network representation is used to store cases and other knowledge. Concepts, cases, and relationships map one to one with nodes in the representation. Memory retrieval is achieved by passing analogue markers, referred to as activation, along the links in the network. Working memories provide several interface facilities like: • Symbolic labels through which internal nodes can be accessed. • Activation distribution „snap-shots“ that record the history of the retrieval mechanism. Similarity is measured as the correspondence between two descriptions. Context selection can either be made via predetermined search paths or in response to the dynamic state of the activation distribution in the working memory. That is why this approach may be used to build a causal domain to produce a goal-directed retrieval. Pruning of the traversable relations is realised by weighting relations. However, no application has been implemented with this CBRtool. There is a lot of further work to be done like to be seen in Chadha, Mazlack and Pick (1991). External control, the construction of knowledge-bases and the similarity context's selection needs to be automated to achieve optimal ergonomic properties of a system for practical use.

8.8.3

Dempster-Shafer Guided Rule-Chain-Tracing in Cause-Effect Hierarchies

Users of diagnostic systems need to understand the causes of faults that are proposed by a diagnostic reasoning system. Further, a diagnostic system should be able to refine its behaviour by experience (Bourne, Liu et al., 1991). It should also have mechanisms for understanding and observing interactions between failure alternatives. This was the main perspective when creating the CEH system. Empirical observations often shortcut chains of causal links and create a kind of „compiled mental model“ of the knowledge. Therefore, the theory of belief strengths about failures by Dempster-Shafer (Shafer, 1976) can be used to diagnose in a network of associations. Belief ( A) = ∑ m( B) with B⊂ A

∑ m( A ) = 1 and

m(∅)=0

A

The belief propagation of two belief functions is done by Dempster's rule (Shafer & Logan, 1987) which represents the combined effect.

∑ m ( A ) m (B 1

(m1⊕ m2) (C) =

2

i

j

)

Ai ∩ B j =C

1−

∑ m ( A ) m (B

Ai ∩ B j =∅

259

1

i

2

j

)

A method for the organisation of beliefs about diagnostic problems combines explicit belief organisation with the implicit organisation of knowledge about physical device characteristics, functionality and behaviour. Beliefs about causes of fault are organised in hierarchical models.

tools

personnel too_little

too_loose

too_high

tool2 experience

tool1 too_tight

too_low

out of control condition

tool1_error judgement

novice trainee experienced

tool2_error inspection

Fig. 8.14 – Example of a cause effect hierarchy. {personnel, tools, inspection} is the set of main causes (features). Here, problems about these causes are represented until the out of control condition occurs.

The lessons learned by the researchers, according to this project, were mainly: • The applicability of a concept presented to various types of problems requires organising and understanding beliefs about diagnostic problems. • The presented method captures some properties of rule-based and model-driven diagnosis. On the one hand, rule chains are traced, on the other hand, the knowledge is organised in a constraint-linked, model-like approach. • It is difficult to create a complete knowledge representation with disjunctive concepts as demanded for an effective use of the Dempster-Shafer theory. • Because of the lack of mechanisms for dealing with interrelationships between symptoms and events (successions of test selections), it is difficult to effectively link multiple effect hierarchies. • The incremental refinement strategy of the system is useful. • The diagnosis result of providing an ordered collection of beliefs may prove to be useful when there are multiple faults. • Further exploration is needed for the optimal handling of events linked to multiple causes, techniques for changing weights and about the perspective handling among the topics.

260

8.8.4.

Semantic Nets with Rules

Most expert systems use rule-based reasoning because of the efficiency of compiled heuristic knowledge, but this kind of reasoning restricts a system's ability for acquiring knowledge from the domain experts. Therefore, the dependency on domain experts should be reduced as much as possible. When perfect heuristic knowledge is available, then it may be used to solve the problem. If not, past cases or model knowledge may be used. I.e. to retrieve cases and to modify them by causal knowledge that represents the machine's structure and the properties of its components. Following the „explanation-based learning terminology“ a fault diagnostic case is an example, causal diagnostic knowledge is a domain theory and measurements or testing knowledge is considered operational or strategic knowledge. Therefore, in Kobayashi and Nakamura (1991) fault diagnosis cases and causal models were integrated. This was done by connecting the following structures: • Cases with: faults, symptoms and causes • Rules of the form (faults, symptoms) => cause They propagate straight-forward in the semantic net, which originally was constructed in the other direction, namely: Starting at the causes, some symptoms can be observed which, in turn, result in the respective faults. The directed graph of the causal model represents physical properties of machine components where the arcs represent the causal relations. This is still a very general approach. However, they did further work. They reduced the explanation structure in elementary rules like in the following description: • Derive primitive rules by tracing all causal paths on the explanation structure which means that all faults and symptoms are represented in a conjunctive form. • Derive the disjunctive rules. All adjacent nodes on the same causal path that have no branch on the original explanation structure. • Disjunctively connect pairs of primitive rules which share the same cause and faults and the most of the symptoms. • Insert to the corresponding rules all negative symptoms used to generate the explanation structure. This way it is also possible to derive causal explanations in the model if there is only a case presented to the system. The system was implemented with the application of a cigarette-making machine. It has the following main properties: • A complete causal model is necessary to derive rules from cases. • The rule base is generalised and completed but the number of rules increases hardly by one percent. • It is possible to obtain knowledge at almost the same level of generalisation from multiple experts.

261

8.8.5.

Meta-Level Reasoning by Application-Specific Meta-Level Rules

There is a lack of efficient techniques on knowledge organisation and integration. In Hong, Xiuwen et al. (1993) a knowledge-based information processing system is described called Knowledge-Based System for Engine Diagnosis (KBSED). It was designed for the purpose of the diagnosis of automobile engines without whole-block disassembly. The system integrates fault diagnosis, state detection and recognition with repair consultation. The used knowledge structure is a multi-layer semantic net for hierarchical diagnosis, knowledge exploitation (namely information retrieval) and also for the knowledge control structure. These layers are: • Task planning and decomposition in the planning layer; • Hierarchical causal diagnostic strategy in the strategy layer; • Hierarchical causal mapping associations, path based system description and state knowledge in the domain layer. It is a model-based approach in order to represent the diagnostic knowledge of a complex system in multiple levels of abstraction. It was proposed as a hierarchical causal model based on a structure and function decomposition principle. This was done because the authors believe that making use of only shallow knowledge and a simple reasoning strategy only is not strong enough to model expertise of human experts. A knowledge layer is considered a general separation of expertise according to the control grades of individual knowledge sources in the expertise, where those at a high level layer will control the utilisation of those at a lower-level layer of knowledge.

262

static knowledge

•concepts •facts •technical terms •representation structure

object knowledge

•heuristic experience •causal associations •structure connectivities •object analysis knowledge

strategy knowledge

•hierarchical strategy •meta-level rules •constraint search •integrating strategy

planning knowledge

•task generation •task monitoring •behaviour diagnosis and repair

Fig. 8.15 – A layer separation of diagnostic expertise1

E.g., meta-level rules, hierarchical problem solving strategies and constraint propagation theories are all knowledge sources of the strategy layer. At this layer the basic entities are object knowledge sources and objects are tasks in which various object knowledge sources can be combined and integrated to achieve a particular diagnostic goal in the highest level, the planning level, which knowledge sources are responsible to create general problem solving plans, to control and monitor the performance.

8.8.6

Meta-Level Reasoning within a Meta-Level Architecture

Flexible problem solving consists of the dynamic selection and configuration of problem solving methods for a particular problem and the goal of problem solving. For this purpose, problem solving methods need to be described in a uniform way by an abstract „model“ of components, which together define the functionality of the methods. Such an abstract „model“ can be used for dynamic selection and configuration of problem solving methods like described in ten Teije and van Harmelen (1995). There are three different knowledge types distinguished to be required for so-called flexible problem solving: • Application description knowledge: This is domain knowledge used by the problem solving methods, e.g. case-dependent knowledge.

1

A similar approach of a framework for knowledge modelling and knowledge level analysis is known by KADS, e.g. in Wielinga, Schreiber and Breuker (1992), or in Armengol and Plaza (1994).

263

• Methods description knowledge: This is knowledge that describes the problem solving methods. It can be about: the functionality, algorithmic aspects (e.g., complexity class), requirements of the input problems (e.g., a causal model). • Knowledge for problem solving flexibility: This is knowledge about selecting and configuring the problem solving methods and how a method can approximate the goal of the given problem input data.

meta-meta-theory (= flexible problem solving knowledge) •schema of problem solving methods •relations between methods •how to configure methods

meta-theory (= problem solving methods) •definitions of methods •definitions of method components

object-theory (= application theory) •domain knowledge •data Fig. 8.16 – Structure of the architecture of meta-level reasoning

Further there a described well-defined relations between the different knowledge types. This means mainly that there needs to be an abstraction hierarchy because, e.g., methods know only about the general class of symptoms without referring to specific instances. However, there are also relations between methods which are related to the observed and expected behaviour to possible problem solutions. Or relations which assign a specific role to objects. The roles are based on how these objects are used by the problem solving method. The three knowledge types are represented explicitly and separately. The representation of the knowledge types, which include mainly predicates, allows also the direct definition of the relations between the knowledge types. The connections are realised by so-called reflection rules. A reflection rule is a rule from a formula that holds in the meta-theory to a formula in the object-theory, or vice versa. We can conclude that the architecture for flexible problem solving is a combination of standard meta architecture techniques (ten Teije & van Harmelen, 1995). The relation to the INRECA project is that in INRECA the „seamless integration level“ allows a seamless interaction between parallel processes, which can communicate about relevant problem solving data. The metaknowledge was directly integrated into the control structure, because the efficiency that can gained by this form of realisation, and because the interfaces of the algorithms where already known before due to a systematic analysis of domains and problems that are to be handled.

264

8.9 Working with Vague Relations Detecting failures by scale-driven object matching is a common approach. The scaled value determines a kind of relevance for the inference mechanism or the membership to a symbolic class. Vague relations must be strictly divided from the so-called uncertain relations (Richter, 1989). While uncertain relations are based on the lack of sufficient information about the target concept, vague relations determine the degree of membership to certain symbolic attribute values or the degree of incorrect measurements. The central question which can be handled with vague relations is the stability of the inferred solution. While a system based on binary logic would present only if a system is stabile or not, a system which can handle vague relations would offer the degree of stability in relation of how near the input signals were to their application destinations. This approach can easily be integrated like any other relevances in purely relevance based reasoning with filter-lists like firstly described in Weis (1995a) (also sections 5.2.2, 5.1.7 and 5.2.1.5). This is the case especially in complex analogue sensor-driven environments which need to be handled by symbolic objects because of their complexity.

8.9.1

Vague Relations for Production Rules

In Döring (1994) an inference system can be found which handles both uncertain relations as well as vague relations. The system works with so-called production rules stored in an external knowledge-base, which can be downloaded from a database at runtime. Like shown in Döring and Burkhardt (1995) uncertainty and vagueness need to be treated differently. Vague rules base on vague numbers. These where derived by (in this context) so-called precise numbers with membership functions. In the core of the vague number the sharp number is to be found with the membership degree of 1.0 like to be shown in the following example (fig. 8.17).

1

0 Core Fig. 8.17 – Membership function of a vague number

If a number x must be compared for equality to a vague number y with the core y0 and environment d the following formula can be used:

265

0 ⎧ ⎪ x − y0 + d ⎪⎪ d equal _ y( x ) = ⎨ x − y0 + d ⎪ d ⎪ ⎪⎩ 0

for

x ≤ y0 − d

for

y 0 − d < x ≤ y0

for

y0 < x < y0 + d

for

y0 + d ≤ x

The resulting value is contained in the interval [0,1]. Analogously also membership functions for total or partial orderings may be defined like the following:

smaller than 1

0

smaller or equal 1

Core

0

0

1

Core

1

Core

0

greater or equal

greater than 1

equal

0

Core not equal

1

Core

0

Core

Fig. 8.18 – Membership functions of total or partial orderings

8.9.2

Scale-Driven Object Matching for Model-Based Reasoning

The underlying foundations of model-based or knowledge-intensive reasoning, especially encapsulation, inheritance, and polymorphism favourite system decomposition and provide unprecedented abstraction mechanisms. One of the strongest appeal is the ability to capture systems behaviour from the functional specification of system components. In Vargas and Bourne (1993) a mechanism is introduced that takes advantage from this global perspective, called scale-driven object matching. They use four types of commonly known value ranges: nominal, ordinal, interval, and fuzzy. As the other types are well known from the INRECA project here the fuzzy scales will be shown only. A fuzzy scale extends the binary nature of nominal matches by creating a 2dimensional plane in which one dimension corresponds to a membership function, as defined in Fuzzy Set Theory (Kandell, 1982; Kruse, Gebhardt & Klawonn, 1993; Zimmermann, 1993) and 266

the second dimension corresponds to the proximities. The semantics of each region in the plane may be explicitly well defined. A fuzzy scale may look like the following example.

Degree of 1 member0.8 ship 0.6

very low

normal

low

high

very high

0.4 0.2 0

Proximity 0

0.2

0.4

0.6

0.8

1

Fig. 8.20 – A fuzzy scale

Within the system the scale acted as a skeleton that supports a set of procedures for object comparison. It balanced the need for flexibility during the knowledge acquisition and the need for data accuracy. Further, it provided the possibility to implement a qualitative form of constraint propagation.

8.9.3

Real-Time Fuzzy Control

There are many control mechanisms which provide regulation values based on sensor data. The main problem is to satisfy the Nyquist criterion, which says that the system begins to swing exponentially if the frequency band of the controller and the distance become incompatible in the way that the guiding frequency of the controller becomes too large. Therefore, one has to adapt the controller for the distance in the way that it leaves the margin of the phase where the acceleration becomes too strong. This can be a time-critical task in real-time systems which are highly dynamic. Therefore, Zadeh (1972) developed the fuzzy control approach for real time monitoring systems. This vague rule-based approach usually has a higher complexity than usual controllers but it does not need any parameter extraction (unprepared linear adaptation problems) in real-time if the environment, namely the distance relative to the control variables, changes unexpectedly. This task has a main event loop with a cycle time Δt with 1/Δt > 2ϖmax (Claude Shannon's theorem 1948; von Puttkamer, 1992). The aim was to describe the control process by a set of local rules based on vague variables. If done in hardware the following algorithm can be implemented with a cycle time of less than 60 μs (fig. 8.21).

267

get sensor data fuzzyfication inference on rules defuzzyfication extract output data Fig. 8.21 – Fuzzy controller‘s main event loop

Fuzzyfication is done by multiple membership functions like to be seen below (fig. 8.22).

Degree of 1 member0.8 ship 0.6

very low

normal

low

high

very high

0.4 0.2 0 0

This value is 65% very low and 35% low. Fig. 8.22 – Fuzzyfication

The relation between input parameters and output values is determined by explicit rules. These may be composed by disjunctive or conjunctive clauses and implications. The evidence of these rules may be calculated for the AND-function by the MINIMUM-function and for the OR-function by the MAXIMUM-function of the memberships of the input parameters. For the defuzzyfication the point of maximal gravity of the resulting fuzzy output parameters is calculated. This can be done relatively to the points of maximal gravity of single triangular or trapezium functions and the remaining plane relatively to the membership rules. From INRECA project point of view, vague relations could be a method for further integration into the Inreca-tree. 268

8.10

Integration

A very popular word in current research projects is the notion of „integration“. A closer look to such projects shows that there are many different techniques that are all called „integration“. The variety of approaches cannot be covered as a whole at this place. From our evaluation point of view we would like to mention integration in synthetic problem domains (Bartsch-Spörl, 1995b), as a contrast to analytic problems handled by INRECA (section 9.3). Aamodt and Nygård (1995) wrote on the aspect of integrating data, information, and knowledge, which points out the strong relations of CBR to information systems, and vice versa. An architecture of integrated information systems is described in Scheer (1995; 1992). Other interesting integration issues are the integration of languages and tools (Janning, 1992) or the a-posteriori integration within already existing CIM-environments (Nagl, Eversheim et al., 1995). In the next section we describe two software engineering projects with a strong emphasis on integration and compare their kind of „integration“ with the INRECA approach. We also cover some other forms of integration: the integration of model-based diagnosis in an engineering system, the integration of a help-desk system in a real enviroment, the integration of planning tasks in a multiprocessor model, as well as the integration of rules and neural nets.

8.10.1

Integration for Software Development Project Support

Nagl (1993; 1990b) describes the IPSEN project (Integrated Software Project Support ENvironment) at the University of Technology at Aachen. In this project, the integration consists of linking all the different documents of a software project into one complex structure. These links are hypertext-like pointers from one document to a specific part of another document. This integration of the data is also supported by the different tools that handle the different types of documents. Each time when data containing a link are changed, the document pointed to is opened so that it can be changed in order to maintain consistency. As often as possible this change is done automatically, or at least semi-automatically. The DAIDA project (Development Assistance for Integrated Database Applications), also at the University of Technology at Aachen, described in Jarke (1992; 1991), uses another kind of integration for its administration of different types of data. The software developed here consists of relational database applications. These programmes are developed in three steps of conceptual modelling which use three different types of representation. These steps are: a semantic network which models the world of the application to build, as well as its environment, then an object-oriented representation of the software structure and finally the relational database programme itself. The main aspect of this project is that the data of these three modelling stages is stored in one information system. This information system does not only store the actual state of the software, but also the development history over its whole lifetime. All other tools developed in this project, such as design assistants and mapping assistants from one representation to another, are grouped around this central information system and use the data stored there. 269

Thus, the integrating aspect of this project is the central data storage system which holds the information for all components of the software development environment. The kind of integration used in the INRECA project is different from the projects mentioned above. In the mentioned software engineering projects, integration consisted of building a system which handles different types of data and relationships among those. This is not a major problem in INRECA where most of the data are stored in the cases, for which one type of representation is sufficient. Integration in INRECA concerns algorithms. The two basic algorithms used in INRECA, induction and CBR both have their advantages and disadvantages. The integration consists of finding a combination of both algorithms that takes benefit from the advantages of both. At critical points, this new algorithm is supposed to apply parts of the basic algorithm which is likely to be more efficient than the other one in the given situation. In INRECA, an integrating element has been introduced in the form of a common data structure, namely the Inreca-tree (chapter 7), to allow the required deep kind of integration in an efficient and transparent way.

8.10.2

Integrated Model-Based Diagnosis

For the model-based diagnosis of the space shuttle main engine a knowledge-based system called EDIS (Engine Data Interpretation System) was developed at the NASA Marshall Space Flight Center. EDIS (Hofmann, Cost & Whitley, 1992) incorporates a combination of diagnostic paradigms because neither heuristic rules nor quantitative models alone represented the scope of knowledge adequately that human experts apply to the task of Space Shuttle Main Engine (SSME) data analysis and diagnosis. The architecture of EDIS has been formulated to enable cooperation of several heterogeneous knowledge sources in order to emulate the diversity of human expert reasoning. The integration perspective of this approach is also the algorithms point of view. While in the INRECA project especially the integration of different approaches for solving analytic problems was stressed, here we stress the integration of different simulation techniques. The system addresses the general problem of diagnosing malfunctions in a complex thermodynamic system based on a fixed set of incomplete data. The plant model is constructed by instantiating component models and linking them according to the plant structure. Behaviour is modelled by incremental qualitative constraints on the system's medium valued parameters. Constraints are qualitative because they map a parameter's values on three different categories: low, normal, and high. Then EDIS tries to find the explanation of the indicated behaviour. This is the point of integration in the EDIS project. A computer quantitative simulation of the SSME power balance provides an additional model of the SSME's behaviour. When the inference method remains the same the method to extract additional domain knowledge is integrated. This is a totally different approach of integration as it was selected for the INRECA project, where integration of reasoning approaches was stressed and not the integration of quantitative domain knowledge.

270

8.10.3

Hybrid Integration of Rules and Neural Nets

While the use of connectionist techniques as a reasoning mechanism to replace the execution of symbolic rules within expert systems has attracted much attention recently, less attention has been paid to the integration of the paradigms of connectionist and symbolic techniques within one system (Senjen, de Beler et al., 1993; also section 1.3.7). Integrating the two approaches can take many different shapes. A problem could be represented as a mixed connectionist and symbolic knowledge base with modes of reasoning selected as appropriate for the current task or as one mode of reasoning embedded in the system where the overall flow of control is determined by the other mode. Both possibilities require a knowledge representation accessible for both modes. Knowledge acquisition is done in the following way: The existence of examples of the solution of the problem or the ability to empirically classify problem instances allows the training of a connectionist network (assuming sufficient training examples). The resultant network can be used to solve future problem examples by its incorporation into a hybrid system or by generating classification rules for their use in a symbolic reasoning system. The connectionist network is used as a filter, to capture abnormal network events, and to provide an initial categorisation of possible faults. The second phase is a rule-based expert system that performs detailed fault diagnosis (fig. 8.23).

neural net

h y p o t h e s e s

rule-based expert system

f a u l t s

Fig 8.23 – Partitioned hybrid system for diagnosis and monitoring

This approach must be seen in the context of the idea of the MAC/FAC model (Many Are Called / Few Are Chosen) in as described in Gentner and Forbus (1991) (section 1.2.1/fig. 1.2). In INRECA the idea underlying the MAC/FAC model is realised differently. The detailed problem solving is done in a case-based fashion, whereas inductively generated rules can work as a filter to improve the similarity assessment and the performance of the overall system.

8.10.4

Integrated Help Desks

Help desk automation focuses on collecting information about callers and their problems to quickly diagnose and help. The system described here is an integrated case-based help desk for 271

service and support. It was mainly developed by Kriegsman from Interleaf and Barletta from Case Data Solutions (Kriegsman & Barletta, 1993). It integrates the paradigms of text-based, rule-based, and case-based systems in a hybrid approach. In the text-based system the user finds problem solutions directly by keywords from past cases or by a directed text search in a database. The rule-based approach supports automatic extraction of relevances (Weis, 1995b). Cases are stored in a decision tree which is created by the rules. The retrieval task in the decision tree is supported by nearest neighbour links. In the integrated algorithm the decision tree works mainly like a preselection of relevant hypotheses. The solution is found by a similarity measure. Here we find several similarities to the INRECA project. E.g., the possibility to integrate rules into the decision tree or the possibility to work as a help desk application. However, INRECA is the further elaborated approach. Here also induction was used to automate rule extraction (chapter 7) as well as numeric similarity measures to optimise the access (Wess, 1995). The Inreca-tree as an integrating element offers a more efficient and a more transparent approach. There is also a certain relationship to the rule-based MOLTKE approach (section 1.3) which was developed earlier. Here the retrieval by context refinement was stressed which would be the analogue approach to the integrated rules into the decision tree (also section 1.3.5 on S3+/INRECA where INRECA was integrated with tecInno's MOLTKE derivative, the S3 service support system).

8.10.5

Integration of Planning Tasks in a Multiprocessor Model

Reasoning from prior cases or abstractions requires that a system can identify relevant similarities between the current situation and the objects represented in the working memory. Often relevance depends upon abstract, thematic, costly-to-infer properties of the situation. Because of the costs of inferring the system needs to learn which descriptions are worth inferring and how costly that inference would be (Owens, 1993). For a case-based reasoner to make effective use of recalled prior experiences, it must be able to judge which of its cases is applicable to the current situation. Computationally this means that a case-based reasoner needs: • A vocabulary of features used to describe a situation. • A case library of data structures describing prior cases each represented using features from the system's vocabulary. • An inference mechanism for strategic, explanation, or repair purposes. • A retrieval mechanism for finding prior cases that share relevant features with the current situation. The ANON programme provides a mechanism for integrating feature extraction and memory search, and for explicitly reasoning about the costs and benefits of individual features. The integrated planner is mainly used for the following purposes: • Extraction of indexes and the actions needed to be taken into account and cost analysis. • Way planning in the context of the planner-navigator-pilot hierarchy (von Puttkamer, 1993) • Explanation of failures. 272

• Recovery from failures. This means planning some activity that will lead to the original goals, taking into account the changed world resulting from the failure. Or, if achieving the original goal now looks too expensive, it should work on some other goal. • Repair of plans that resulted from the failure. If the explanation assigns blame to some condition internal to the planner, e.g., failure to look for certain contradictions before commencing a particular activity modifying the plan so that the future uses of the same plan will not result in the same failure. Critical planning situations may be if, e.g., an action does not have the intended effect, the prediction of failures, or the exploitation of opportunities. For doing that the system can use the PFXP package: Causal description of plan failures with recognition criteria and repair and recovery strategy (Owens, 1990). The planner may also take use of induction-like bottom-up or top-down approaches derived by ID3. The filtering uses a scan/reduce model of computation (Owens, 1993). In addition to the processors used to represent the links between labelling terms and knowledge structures, there is a processor for each labelling term and one for each knowledge structure. When all relevant labelling terms are selected during a given iteration each labelling term broadcasts information to all processors representing links between itself and the knowledge structures to which it points, so that each processor corresponding to a knowledge structure now has information about which of its labels were selected. The system then ranks the knowledge structures in parallel according to how closely their labelling terms match the labelling terms selected for the current iteration of the retrieval process. Specific weights are directly accessible to each processor. The relation to the INRECA project is that here also integrated knowledge structures are used for the case retrieval. The clustering mechanism is similar. Certainly, INRECA has an optimised retrieval architecture for one processor systems. What is missing in the INRECA system is the planning component. This the case because the INRECA project stressed mainly the efficiency and effectiveness of the retrieval task and not the repair of faults that may occur during the retrieval phases. Such planning tasks must be covered by the user who, e.g., could use the exclusion rules to reduce the number of cases and, by this, to focus on relevant aspects.

8.11

Summary

We compared INRECA to related work that has been collected from more than 300 research papers with a special focus on the period from 1991 to 1995. However, also interesting contributions from 1996 and the time before 1991 have been considered. The research papers have been systematically selected according to a number of topics, which are of special interest within the scope of evaluating the INRECA system. This includes different kinds of applications as well as approaches to clustering, databases, decision support, diagnosis, machine learning, information retrieval, integration, meta-level reasoning, model-based diagnosis, rule-based reasoning, real-time, repair, and technical diagnosis. All these research papers have been classified into one (out of four possible) categories, depending on the fact whether the respective paper was of no further interest for the evaluation task (it was not considered at all), whether a reference to the paper appeared to be helpful in at least one aspect (the paper was included in the 273

reference list), whether a number of aspects of the paper have been used during our evaluation task (we considered an abstract of the paper), or whether the contents of the paper has been of high importance for our evaluation task (we considered the complete content of the paper).

274

CHAPTER 9: TOWARDS AN ANALYTIC FRAMEWORK FOR EVALUATING AND DEVELOPING CBR SYSTEMS We would like to begin this chapter with the following two citations that help to position our approach:

„A scientific challenge is to understand the characteristics of tasks, knowledge types, and methods, and on such a basis develop a broad-covering theoretical framework for the integration of case-specific and general knowledge, and case-based and generalisation-based reasoning“ (Aamodt, 1994c). „We need a flexible knowledge acquisition support which automates as much of the knowledge base development process as reasonable“ (Althoff, Maurer et al., 1992). From a software or knowledge engineering perspective, we try to use non-functional system properties as a systematic means for (CBR) system development. This is a similar goal as it is – at least partially – underlying in other software/knowledge engineering projects (e.g., SFB-501, 1994; Aamodt, Benus et al., 1992). By contrast, we focus on CBR systems and, thus, can define more precise criteria than we could for software systems in general. Additionally, we combine these system oriented, more technical criteria with an analysis of application domains where we have experience with. On the one hand, feedback from applications, as for instance described in chapter 4, can be systematically transformed in an evaluation based on domain and application task criteria. On the other hand, methods extracted from CBR-related systems and tools can be labeled with the results of the application of such criteria, as e.g. done in sections 5.1 and 5.2. We will discuss some approaches to evaluation that appear to be a good starting-point for introducing our approach. We will then describe our approach by heavily basing on Althoff and Aamodt (1996a+b). This description will refer to various parts within this document to show the explicit relationship of the framework and the methodology we are aiming at to the evaluation work described throughout this document. To make the description more compact, we will also repeat some figures and text when appropriate. Again this explicates the above mentioned relationships. We will present a structuring of analytic tasks (also Althoff & Bartsch-Spörl, 1996) – the kind of tasks INRECA has been developed for – into three sub-tasks (classification, diagnosis, and decision support). This structuring is an extension of the decomposition of the diagnosis task into a classification and a test selection sub-task underlying the MOLTKE approach (section 1.3). It is based on the methods and the knowledge representation that are necessary for each sub-task and, thus, it is of more general interest. The structuring of analytic tasks shall be viewed as a supplementation of the framework and the methodology described before that allows for a further decomposition of the overall problem.

275

9.1

References and Roots

How evaluation of knowledge-based systems can be viewed, from the perspective of current social and economical science, is described in Kirchhoff (1993). Here a system is specified as an ordered set of elements that are connected to one another. Internal relations are called structure of the system, the external relations are called behaviour. According to Kirchhoff (1993), the evaluation needs to begin with the knowledge-base, which represents the interpretation of the knowledge engineer of the real world expert knowledge. For that purpose a kind of so-called content validation is proposed. Then empirical evaluation should continue with the directly implemented expert knowledge. The validity of all the system directly depends on two items they call objectivity and reliability, which means that results should be independent of the examiner and that a certain degree of exactness should be measured. Further Kirchhoff (1993) states that the developed evaluation methodology, which bases on classical expert systems, may be – from the point of view of real expert problem solving methodology – used also by developers of CBR or induction-based systems. On the level the methodology is reported in Kirchhoff (1993) this suggestion is somehow understandable. However, we do not think that this is the right way of generalising. A methodology needs to be more focused and specialised (e.g., Cohen & Howe, 1988a; section 2.1). In INRECA we focused on CBR and related methods – not on knowledge-based systems in general. In addition, we restricted the scope of our applications to analytic tasks (also section 9.3). We believe that this was of central importance and allowed us to identify commonalties and differences with other approaches in the INRECA and MOLTKE system families or that clearly relate to INRECA because of the used methods. The method of Slagle and Wick (1988) for evaluating candidate expert system applications works such that a weighted sum of the desired attributes of a programme valued the final usefulness of the problem solving process as well as other system capabilities. By this method the authors wanted to come up with interesting applications for expert system technology. The result of such an evaluation is a context-dependent number that can be greater or smaller than the number of another application. Thus, for a given expert system tool the best-suited application could be selected. Our view is the other way round. How to select the best-suited tool for the problem at hand. However, if a company already bought a software tool also the view of Slagle and Wick makes sense. Nevertheless, it has certain drawbacks: • Every interesting characteristic is generalised into an attribute value and a corresponding weight. • There is only differentiated between essential and desirable features. • All the other information is „coded“ into the application of this application framework. It is neither easy to update nor to be extended for different purposes. Our approach (also Althoff & Bartsch-Spörl, 1996; Althoff & Aamodt, 1996a+b; Meissonnier, 1996) aims at building a case-based information system that uses all the capabilities of INRECA for case representation and similarity assessment to model the selection problem in sufficient detail. As pointed out above, it is reasonable to restrict such an approach because the knowledge 276

engineering task for both the case representation and the similarity assessment requires sufficient competence, knowledge, and information to come up with a valuable case/knowledge base.

9.2

The Framework and the Methodology

Our approach tries to close the gap between high level characterisations on the one side and concrete systems on the other side. Knowledge level analysis and CBR are merged to come up with a methodological framework. Technical/ergonomic criteria are combined with domain/task criteria. Therefore, both kinds of criteria need to be detailed to the method level. The sections 5.1 on extracted methods and explicit knowledge representation and 5.2 on implicit knowledge modelling describe a first step into this direction. The task-method decomposition model (fig. 6.3) is used to structure the framework. We developed a set of algorithms – and integrated them into the task-method decomposition hierarchy – that base on the extracted methods, the explicit knowledge representation as well as the implicit knowledge modelling. Our current focus is on analytic tasks as described in section 9.3. In Althoff and Bartsch-Spörl (1996) it has also been contrasted, on a level of high abstraction, with synthetic tasks. Experiences with synthetics tasks like design and planning, which normally require a high amount of case adaptation, are for instance described in Bartsch-Spörl (1995a), Oxmann and Voß (1996), Voß, Bartsch-Spörl and Oxman (1996), Bergmann, Muñoz-A. and Veloso (1996), Smyth and Keane (1995), and Hanney, Keane et al. (1995). A particular strength of CBR over most other methods is its inherent combination of problem solving with sustained learning through problem solving experience. This is therefore a particularly important topic of study, and an issue that has now become mature enough to be addressed in a more systematic way. To enable such an analysis of problem solving and learning, in the following we describe first steps towards the development of an analytic framework for studying CBR methods. It provides an explicit ontology of basic CBR task types, domain characterisations, and types of problem solving and learning methods. Further, it incorporates within this framework a methodology for combining a knowledge-level, top-down analysis with a bottomup, case-driven one. In this section, we present the underlying view and the basic approach being taken, the main components of the framework and accompanying methodology, examples from the evaluation work described throughout this document and how they relate to the framework.

9.2.1.

Introduction

Over the last few years substantial progress has been made within the CBR field. The problems we are facing have become more clearly identified, research results have led to improved methods for case retrieval as well as improved approaches to the harder problems of adaptation and learning (see, e.g., the collection of recent papers in ICCBR-95: Veloso & Aamodt, 1995). In the course of this development it has also become clear that a particular strength of CBR over most other methods is its inherent combination of problem solving with sustained learning through problem solving experience. This is therefore a particularly important topic of study, and an issue that has now become mature enough to be addressed in a more systematic way. To 277

enable such an analysis of problem solving and learning, a unified framework for describing, analysing, and comparing various types and aspects of CBR methods is needed. Integration of learning and problem solving may in general start out from different goals, and be viewed from different perspectives. One example is „concept formation“ as a goal, and the formation and utilisation of operationalisation criteria related to the problem solving task, as the perspective. Another example is „improved performance“ as a goal, and the improvement of total problem solving speed - for computer and human together - as the perspective. A third example is „sustained learning“, i.e. continuous learning through problem solving experience, as a goal, and the impact of the application problem task on the learning method as a perspective. Many more examples may be given, and for each of them a particular area of overlap, an „intersection space“ between machine learning (ML) and problem solving (PS) methods can be identified. Within this space, dependencies and other relations between specific ML and PS methods may be described and analysed in a systematic way, provided we have a suitable means to structure the space. In the following we describe first steps towards the development of a framework and a methodology which defines and makes use of such a structure. Since we are studying CBR methods, the natural focus is on the third of the above goals: Sustained and (continuous) learning from each problem solving experience as a natural part of a problem solver's behaviour. Within a broader perspective of integrated learning and problem solving, it is also natural to start a study of learning as close as possible to the source of learning, namely a concrete experience. Our work is related to some earlier suggestions for analytic CBR frameworks, such as the similarity-focused framework by Richter and Wess (Richter & Wess, 1991), Althoff's analysis of systems for technical diagnosis (Althoff, 1992a), Aamodt's comparison of knowledge-intensive CBR methods (Aamodt, 1991), Armengol's and Plaza's analysis of CBR system architectures (Armengol & Plaza, 1993), and the framework for systems comparison and evaluation as described in Auriol, Manago et. al (1995). Our framework extends these previous suggestions in several respects. It provides an explicit ontology of basic CBR task types, domain characterisations, and types of problem solving and learning methods. Further, it incorporates within this framework a methodology for combining a knowledge-level, top-down analysis with a bottom-up, case-driven one. In this section, we present the underlying view and the basic approach being taken, the main components of the framework and accompanying methodology, and examples of the evaluation work done throughout this document and how they relate to the framework.

9.2.2.

Basic Approach

9.2.2.1. Knowledge-Level Analysis A potentially useful way to describe problem solving and learning behaviour is in terms of the goals to be achieved, the tasks that need to be solved, the methods that will accomplish those tasks, and the knowledge of the application domain that those methods need (section 6.1). A description of a system along these lines is often referred to as a knowledge level description, 278

and more recent research in knowledge acquisition (Steels, 1990; Wielinga, Van de Velde et al., 1993) has clearly demonstrated the usability of this approach. The original knowledge-level idea has undergone some modifications over the years, from Newell's highly intentional, purpose-oriented way of describing a system (Newell, 1982), to a more structured and usable type of description. An incorporation of the CBR perspective into knowledge-level modelling is discussed in Aamodt (1995). Adopting a knowledge-level perspective to the analysis of integrated PS-ML systems enables the description of methods and systems both from a general (intensional) and case-specific (extensional) perspective. A general description relates a method to descriptive terms and relationships within a general model of descriptive concepts - i.e. an ontology of task types, domain characteristics, and method types. Through a case-driven description, methods can be understood by relating them to already known methods within already described/implemented systems (e.g., CBR is combined with rule- and model-based reasoning in the same way as in CREEK (Aamodt, 1994a); a decision tree is generated as in INRECA (Manago, Althoff et al., 1993); the similarity measure is adapted as in PATDEX (Wess, 1993); partial determination rules are generated and used like the so-called „shortcut rules“ in MOLTKE (Althoff 1992a); etc.). After having developed/described a certain number of systems, we will be able to select/instantiate a system description at the knowledge level by a combined use of general and case-specific descriptors. What we are aiming at is an effective combination of top-down and bottom-up analysis and modelling methods, based on an integration of these two perspectives. From an engineering point of view, this will enable a particular symbol-level architecture to be chosen, and/or a chosen architecture to be instantiated, based on a thorough understanding of the real world application task and its domain characteristics. However, a knowledge-level description in itself will not provide a language detailed enough to describe or analyse system designs, or to arrive at a symbol-level architecture specification. Our approach therefore incorporates a focusing perspective and an analytic „tool“ to help in the more detailed description that guides the architectural specification based on a knowledge-level model.

9.2.2.2. Similarity as a Focusing Mechanism The focus provided by this mechanism leads to a view of - in principle - all CBR methods as operations related to similarity, in one sense or another. That is, problem solving can be described as a process of initial assessment of similarity (case retrieval) followed by a more deliberate assessment of similarity (case adaptation), and learning (case extraction, case indexing, and possibly updates of general knowledge) can be described by relating it to later similarity assessment - i.e. to a pragmatic learning goal. Along with Richter and Wess (1991) we view similarity as an a posteriori criterion, and any attempt to assess similarity before a retrieved case has been found useful will only result in a hypothesised similarity assessment. Our retrieval methods should of course try to minimise the difference between the hypothesised similarity measure and the actual similarity determined after the attempt has been made to use the case. A way to describe the role of general domain knowledge, for example, is then as a means to reduce this uncertainty of the initial similarity assessment with respect to the final similarity assessment

279

made after having evaluated the success of the (possibly modified) case in finding a solution to the input problem.

9.2.2.3. Tasks and Domain Characterisations The types of application domains we address cover a wide spectrum, ranging from strong-theory domains with rather well-defined relationships between domain concepts (e.g., diagnosis of purely technical systems), to weak-theory and open domains with uncertain domain relationships (e.g., medical diagnosis). This is an important feature of the framework, since we particularly want to relate characteristics of the task (the what-to-do) and the knowledge on which performance of the task is based, to the methods (how-to-do's) that enable the problem solver to accomplish the task by use of the knowledge. The starting point is always the real world setting in which the system is to operate. Medical diagnosis, for example, in a real world setting, is far from a pure classification task (section 9.3). If a system shall cover the major tasks involved in practical diagnosis, it will have to include planning tasks (e.g., setting up and continuously revising an examination protocol), as well as prediction tasks (assessing the consequences of a treatment). The next section outlines the core components of the framework, with a focus on the knowledgelevel description and analysis and the combined top-down and bottom-up oriented methodology. The incorporation of the similarity assessment mechanism is part of ongoing research.

9.2.3.

Framework and Methodological Issues

9.2.3.1. Basic Framework Components At the highest level of generality, a general CBR cycle may be described by four tasks (Aamodt & Plaza, 1994; section 6.1.1.2): Retrieve the most similar case or cases, reuse the information and knowledge in that case to solve the problem, revise the proposed solution, and retain the parts of this experience likely to be useful for future problem solving. See figure 9.1. Note that the tasks referred to here are internal reasoning tasks, and different from the application problems tasks (diagnosis, planning, etc.) referred to earlier. The four CBR tasks each involve a number of more specific sub-tasks. An initial description of a problem (top of fig. 9.1) defines a new case. In the retrieve task this new case is used to find a matching case from the collection of previous cases. The retrieved case is combined with the input case - in the reuse task - into a solved case, i.e. a proposed solution to the initial problem. The revise task tests this solution for success, e.g. by applying it to the real-life environment or have it evaluated by a teacher, and repaired if failed. This task is important for learning, since the system needs a feedback of how successful its proposed solution actually was. Retain is the main learning task, where useful experience is retained for future reuse, by updating the case base and possibly also the general domain knowledge. As indicated in the figure, general knowledge usually plays a part in this cycle, by supporting the CBR processes. This support may range from very weak to very strong, depending on the type of CBR method. By general knowledge we here mean 280

general domain-dependent knowledge, as opposed to the specific domain knowledge embodied by cases. For example, in diagnosing a patient by retrieving and reusing the case of a previous patient, a model of anatomy together with causal relationships between pathological and other physiological states may constitute the general knowledge used by a CBR system. A set of rules may have the same role. Problem

RET RIE VE

New Case

Learned Case Retrieved New Case Case

RETAIN

Previous Cases

REU SE

General Knowledge Tested/ Repaired Case

REV ISE

Solved Case

Suggested Solution

Confirmed Solution

Fig. 9.1 – The CBR cycle (fig. 6.2)

Knowledge-level analysis, as previously described, is a general approach to systems analysis. It is therefore applicable to the analysis of application tasks and domains - as manifested in the knowledge acquisition methodologies referred to earlier - as well as internal reasoning tasks of a problem solver and learner. In our framework we therefore take a „task – method – domain knowledge“ approach both to the analysis of real-world application tasks, and to the analysis of the CBR reasoning tasks themselves. The mapping between the two is as follows: a method from the application analysis (how to solve a technical diagnosis problem, or how to determine the next test to be done) either decomposes an application task into sub-tasks, or it solves the task directly. In both cases these methods set up tasks at the reasoning level (problem solving and learning from experience). In the following, we concentrate on the reasoning tasks. The tasks from figure 9.1 are further decomposed in figure 9.2. The tasks have node names in bold letters, while methods are written in plain text. The links between task nodes (bold lines) are task decompositions, i.e. part-of relations. The links between tasks and methods (stippled lines) identify alternative methods applicable for solving a task. The top-level task is problem solving and learning from experience and the method to accomplish the task is a case-based reasoning method. This splits the top-level task into the four major CBR tasks corresponding to figure 9.1. All the four tasks are necessary in order to perform the top-level task. The retrieve task is, in turn, partitioned in the same manner (by a retrieval method) into the tasks identify features, search (to find a set of past cases), initially match (the relevant descriptors to past cases), and select (the most similar case). All task partitions in the figure are considered complete, i.e. the set of 281

sub-tasks of a task are intended to be sufficient to accomplish the task, at this level of description. The figure does not show any control structure over the sub-tasks. The actual control is specified as part of the problem-solving method. The actual retrieval method, for example (not explicitly indicated in the figure), specifies the sequence and loop-backs for the sub-tasks of retrieve. A method specifies the algorithm that identifies and controls the execution of sub-tasks, or solves the task directly, while accessing and utilising the domain knowledge needed to do this. The methods shown in the figure are high level method classes, from which one or more specific methods should be chosen. The method set as shown is incomplete, i.e. one of the methods indicated may be sufficient to solve the task, several methods may be combined, or there may be other methods that have not been mentioned. problem solving and learning from experience

case-based reasoning

retrieve

identify features

copy

evaluate solution adapt

search

retain

extract

repair fault

index

interpret problem follow direct indexes search index structure

evaluate by teacher

copy solution

calculate similarity

search general knowledge

use selection criteria

explain similarity

extract relevant descriptors

integrate

initially match select

collect descriptors

infer descriptors

revise

reuse

extract solutions selfrepair

adjust

evaluate copy solution method

elaborate explanations

modify solution method

in real world

modify solution

userevaluate repair in model

indexes

update general knowledge

determine indexes

generalize indexes

extract justifications extract solution method

rerun problem

Fig. 9.2 – Task-method decomposition of CBR (fig. 6.3)

The above structure provides the basis for the analytic framework. It needs to be elaborated and described in more detail, characterisations of domain knowledge types need to be added, and dependencies between the various knowledge types need to be identified (section 5.1.9).

9.2.3.2. Methodology As previously stated, the basic methodological approach is to combine a top-down oriented analysis of application tasks, domain knowledge descriptions, and methods with a bottom-up, case-driven method of studying existing systems. The aim is to arrive at a coherent framework and description language that specialises from the high-level analysis and generalises from the example systems studied. The baseline of the approach is as follows. We describe CBR systems as well as domains and application tasks using two different kinds of criteria, namely criteria characterising the domain and task at hand (domain/task criteria; sections 2.4.3, 2.4.4, and 5.1) and criteria describing the 282

abilities and limitations of existing systems and system components (technical/ergonomic criteria; sections 2.3 and 5.2, as well as chapters 3, 4, and 6). Examples for domain/task criteria are size1, theory strength2, openness3, change over time4, and complexity5, while case and knowledge representation, similarity assessment, validation, and data acquisition and maintenance exemplify important technical/ergonomic criteria. We analyse the underlying methods and domain/task characteristics by relating domain/task criteria with technical/ergonomic criteria. Figure 9.3 gives an example of how CBR systems can be labelled with domain criteria. In section 5.1 we already described several CBR-related methods that have also been labelled with domain criteria. By contrast, chapter 4 includes several domain and application task descriptions that have been structured using technical/ergonomic criteria. Combining these kinds of descriptions with a more general kind of analysis based on the subtask structure of figure 9.2 was already shown to be useful in chapter 6.

Domain Knowledge

Weak

Creek Inreca Protos

Strong

Patdex-1 Patdex-2 Moltke

Environmental Dependency

Size

Theory Strength

Large

Creek Inreca Moltke

Small

Open

Closed

Patdex-1 Patdex-2 Protos

Creek Inreca Protos

Casey Patdex-1 Patdex-2 Moltke

Fig. 9.3 – An example of domain criteria related to existing systems (fig. 2.6)

From a software or knowledge engineering perspective, we try to arrive at non-functional system properties as a systematic means for (CBR) system development. Since we focus on CBR systems we can define more precise criteria than we could for software systems in general. Additionally, we combine these system-oriented, more technical criteria with an analysis of application domains in which we have experience. On the one hand, feedback from applications can be systematically transformed in an evaluation based on domain and application task criteria. On the other hand, methods extracted from CBR-related systems and tools can be labelled with the 1 2 3 4 5

The size of a domain is characterised by the amount of different items representing the explicit knowledge. The theory strength of a domain depends on the degree of certainty of the involved relationships. The openness of a domain depends on its environmental dependencies. The change over time of a domain means that a domain is called static if there are no expected changes and it is called dynamic if it is clear that changes will appear in continuation. The complexity of a domain means the amount of different taxonomic relations.

283

results of the application of such criteria. Again, the aim is to close the gap between high level characterisations, on the one side, and concrete systems on the other side. General knowledge level analysis and case-driven analysis are merged in order to come up with application frameworks for particular types of systems, based on a common terminology and a unified model. Here technical/ergonomic criteria are combined with domain/task criteria. The intended use of this framework both for analysis and development of integrated problem solving and learning systems, can be described as providing answers to the following questions related to the evaluation of AI research (Cohen, 1989; chapter 2): • How is the CBR method to be evaluated an improvement, an alternative, or a complement to other methods? Does it account for more situations, or produce a wider variety of desired behaviour, or is it more efficient in time or space, or does it model human behaviour in a more useful way/detail? • What are the underlying architectural assumptions? Are all design decisions justified? Does the method rely on other integrated methods? Does it subsume other methods? • What is the scope of the method? How extendible is it? Will it scale up? • Why does the method work? Under which conditions would it not work? • What is the relationship between the class of task, of which the current task is an example, and the method used? Can this relationship be stated clearly enough to support a claim that the method is general to the class of task?

9.2.4.

Examples

We will point to two small examples mentioned earlier which are now interpreted within the framework as specified so far. They both illustrate a bottom-up approach (a case-oriented approach) to the study of CBR methods, undertaken within the general, high level, analytic framework. The first example is a description of the learning method (for the retain task), at a suitable level of abstraction, which was derived by abstracting from a set of existing CBR-related research prototype systems (section 6.5.2, algorithm 6.28). The second example is related to the top-level task of case-based problem solving and learning (fig. 9.2), and methods for integrating case-based and generalisation-based reasoning (section 5.1.9). A set of criteria for analysing and evaluating CBR methods was defined and applied within the study of industrial CBR tools described in chapter 3. These criteria were in turn generalised and extended to CBR-related research prototype systems (chapters 5 and 6). Here the task-method decomposition hierarchy (fig. 9.2) was extended with methods that were abstracted from a set of specific CBR algorithms. One abstracted version of these algorithms is shown in figure 9.4, for the CBR task retain (alg. 6.28).

284

IF no_similar_past_case(current_case) THEN construct_new_case; ELSE lazy_generalise(old_case); IF current_case_successful THEN integrate_into_successful_cases; ELSE integrate_into_total_problem_cases; DO adaptation UNTIL system_behaves_as_wanted Fig. 9.4 – Abstracted method for the retain task. Sources of the algorithm: CREEK, MOLTKE, and PROTOS

The THEN and ELSE sentences point to task decomposition methods of the retain task, i.e. methods that in turn will lead to a particular selection and control over the integrate, index, and extract tasks (fig. 9.2). The other example starts out from the top level CBR with the aim of studying methods that integrate different reasoning strategies, in this case purely case-based reasoning and reasoning from general domain knowledge. In table 9.1 a set of methods, corresponding to some specific research systems, are grouped into levels of integration according to the „tightness“ of integration. The different levels of integration are as follows (section 5.1.9). BOLERO

toolbox level co-operative level workbench level seamless level

• • •

CASEY

• •

CcC+

• •

CREEK

• • •

INRECA

• • • •

MOLTKE PROTOS S3+/INRECA

• • •

•

• • • •

Table 9.1 – Task: Problem solving and learning from experience; Method: Combining case-specific and general knowledge (table 5.11)

At the toolbox level the integration is restricted to the common use of parts of the knowledge base. A reasoning method is initially chosen, and that method is then used as a single method. Exchange of reasoning results between methods is not done. This is the weakest level of integration. At the co-operative level different reasoning methods can exchange intermediate results through a common representation formalism. At the workbench level different reasoning methods are aggregated into a single combined method. For instance, a diagnostic system can carry out its classification sub-task using a rule-based module, and the test selection sub-task based on a CBR component. The seamless level is the highest level of integration. Different reasoning methods are integrated within a single algorithm, basing on an integrating element (like the Inreca-tree data structure: chapter 7), and are not viewed as separate methods. Thus, the change of reasoning methods during reasoning is hidden for the user. While CASEY applies model-based causal reasoning speeded up by CBR (Koton, 1989), BOLERO combines case-based test selection and rule-based diagnostic reasoning (López & Plaza, 1991). In CREEK case-based reasoning is performed within a full-fledged semantic network knowledge model integrating cases as well as general domain knowledge, which enables initial goal-driven context focusing as well as knowledge-intensive, explanation-supported methods for the various CBR tasks (Aamodt, 1994a). MOLTKE focuses on rule-based diagnostic reasoning using CBR for exception handling (Althoff, 1992a). While INRECA combines CBR with integrated inductive k-d trees for case-filtering, preference learning, and consultation (Althoff, Bergmann et al., 1995a; Wess, 1995; chapter 7), S3+/INRECA integrates INRECA with a service support system basing on 285

general knowledge (section 1.3.5). CcC+ is a case-based classifier that can cooperate with other sub-systems of the D3 toolbox for diagnostic reasoning (Goos, 1995). However, PROTOS is a stand-alone CBR system for classification, knowledge acquisition, and learning (Bareiss, 1989).

9.2.5.

Conclusion and Further Research

The long term goal of the research reported here is twofold. The first goal is related to AI as an experimental science: To establish a descriptive and analytic framework for improved understanding of the PS-ML integration problem, starting out from a focus on CBR methods. An improved understanding of the „CBR space“ will be the basis for extensions, such as into learning methods for more eager generalisation. The resulting framework will serve as a „language“ and a set of principles for analytic and comparative studies of various systems across different „traditions“ and research groups. The second goal is related to the needs of improved methods for knowledge-based systems engineering: To specify ML-PS integration as an important task to be understood and handled by the knowledge engineer, and to provide methods for dealing with this problem. We have reasons to believe that we are on the right track, so far, even if only a high-level and general version of the framework to some extent has been tested. At the core of the framework is the combination of a top-level analysis, based on a task-method decomposition, and a bottom-up analysis of specific systems. It is our intention that the systems descriptions resulting from this analysis can be used by the ongoing efforts of creating a case-based information system for CBR research and development purposes (Althoff & Bartsch-Spörl, 1996; 1995). An intelligent information system should be able to give advice based on a set of cases describing CBR systems and applications, embedded in a generic methodology for CBR systems decomposition, analysis, and construction. Another use of the framework which is being explored is its use as a modelling ground for integration of knowledge-based systems into information systems and data base systems in general, based on the different roles these systems (or rather: sub-systems) have in a total decision-support system (Aamodt & Nygård, 1995). One of the major refinements of the framework will be to incorporate relevant parts of the similarity model briefly described in section 9.2.2.2.

9.3

Case-Based Reasoning for Analytic Tasks

Analytic problems are a very important sub-set of real-life problems. They include object classification and identification, monitoring, technical and medical diagnosis, therapy selection and repair, data analysis and prediction, advice giving, help desk, service support, information retrieval etc. According to their underlying complexity we want to structure analytic tasks into three groups of problems (fig. 9.5).

286

Analytic Tasks

Classification

Diagnosis

Decision Support

Fig. 9.5 – Structuring analytic tasks (adapted from Wess, 1995)

9.3.1.1. Classification The basic classification process maps objects or situations to a given set of classes. We assume that all necessary information is available in order to carry out the mapping process (fig. 9.6; fig. 9.7). Examples of classification tasks are risk assessment of cars, determination of biological objects, and analysis of client data. For basic classification tasks CBR is in competition with many other approaches. While inductive machine learning approaches normally reason with generalised knowledge and, by contrast, databases store all the case data, CBR is more in flexible in the sense that it distributes its competence on both the cases and the similarity measure (Wess & Globig, 1994; Richter, 1995; also fig. 1.4 and fig. 10.8). Depending on the specific requirements of an application a simple or a well-informed measure can be used. The better the similarity measure is suited for a given task, the less cases must be stored. In principle, the CBR approach offers the flexibility to decide this at a very late stage in system development. Always when this flexibility is wanted, CBR is a good choice. Comparing CBR with approaches based on generalised knowledge results in similar advantages and disadvantages like those associated with (case) interpretation and (case) compilation.

?

Problem characteristics

Problem solutions

Fig. 9.6 – The classification sub-task CBR systems like PATDEX (Althoff & Wess, 1991; Wess, 1993; section 5.2.2.2) and INRECA (section 5.2.2.3) use at least one similarity measure per class which can be of high value for certain classification problems. Additionally, both systems can learn the relevances of certain attributes for the respective classes (e.g., Richter, 1992; section 6.4, alg. 6.16). Aamodt and Nygård (1995) pointed out that depending on their role in decision making and learning by expe287

rience, cases may be viewed as data, information, or knowledge and, thus, the CBR approach maybe helpful for certain domains where the integration of such issues is important. Current situation

Searching for most similar case

Case found?

no

STOP (No solution)

yes

Hypothesis found?

yes

Solution correct?

no

STOP

yes

(Solution found)

no

Modifying the case base and/or the similarity measure

Fig. 9.7 – Basic algorithm for case-based classification

9.3.1.2. Diagnosis Diagnostic processes differentiate from the above mentioned basic classification processes by the problem of incomplete information, i.e. before carrying out the classification process normally (much) more information must be acquired (fig. 9.8). Thus, the diagnosis task includes, in addition to the classification task, the task of selecting the best test(s) to acquire the missing information as cheap, fast, and secure as possible. Examples are fault diagnosis of a car motor, a CNC machine, or an aircraft engine.

?

Problem characteristics

Problem solutions

Fig. 9.8 – The diagnosis sub-task (adapted from Puppe, 1990)

288

The additional test selection sub-task makes the diagnosis task a very specific one (fig. 9.9). Often general domain knowledge is required to reasonably guide the selection process (section 6.2.1). For some application tasks (e.g., fault diagnoses of CNC machines) the degree of incomplete information can be very high (>90%). Therefore, approaches based on induction have serious difficulties in coping with such problems in an efficient way (also table 3.16). This is again due to the interpretation/compilation contrast we mentioned in the section before. CBR can also offer a solution to the test selection problem itself by using an additional kind of cases (section 1.3.1). In principle, general knowledge about the biological or the engineering system can be used to adapt the diagnostic class in the case description (sections 1.3.6 and 6.3.2). Current situation

Searching for most similar case

Case found?

no

STOP (No solution)

yes

Hypothesis found?

yes

Solution correct?

no

yes

STOP (Solution found)

no

Ascertaining more symptom values

Fig. 9.9 – Basic algorithm for case-based diagnosis (adapted from Stadler & Wess, 1989)

9.3.1.3. Decision Support Decision support processes differentiate from the mentioned basic classification processes by the necessity of representing more general knowledge and allowing user-computer interaction to a very high degree. While the classification goal is clearly defined for the classification and the diagnosis processes (static target definition), decision support processes must cope with (more explorative) problems where the classification goal is defined during the problem solving process (dynamic target definition; fig. 9.10). Examples are looking for a house to buy, looking for a last-minute trip without clearly defining the location and the kind of travel, and looking for a well-suited used or new truck for a company.

289

Solution space 1

?

Solution space N

Problem characteristics Fig. 9.10 – The decision support sub-task In general the difference between diagnosis and decision support problems as defined here is very important. While PATDEX is a well-suited CBR system for fault diagnosis of engineering systems, INRECA as a more general decision support system, and a successor system of PATDEX, must include many additional features. While an attribute-value-based case representation was sufficient in PATDEX, the scope of INRECA requires an object-oriented representation. The algorithm shown in figure 9.11 is supported in PATDEX with a relational indexing scheme for case retrieval (fig. 3.3). In INRECA additional retrieval strategies like sequential and integrated k-d tree retrieval had to be included. One main restriction in PATDEX is its inability to efficiently support local similarity measures (section 5.2.1.2). It is well-suited only for applications with symbolic value ranges. In INRECA an extension of the k-d tree retrieval structure had been introduced such that local similarity measures can be efficiently used (chapter 7). On the one side numeric value ranges can be handled efficiently, on the other side the treatment of symbolic value ranges had to be introduced in k-d trees such that both kinds of ranges can be treated in one retrieval structure (chapter 7). This results in a much more flexible system which is, of course, also much more complex.

290

First query

Searching for similar cases

Cases found?

no

Changing the query

yes

Interesting information found?

yes

no

Information sufficient? no

STOP

yes

(Solution found)

Improving the query Fig. 9.11 – Basic algorithm for case-based decision support

Beyond case representation and case retrieval the use of general knowledge is required for decision support tasks. In INRECA constraints, completion rules, and adaptation rules had been introduced to cope with such tasks. The ability of PATDEX to learn its attribute-specific relevances for the respective diagnoses could be (reasonably) rebuilt for INRECA only for classification and diagnosis tasks. For decision support tasks additional structuring of the domain is necessary which otherwise is naturally given by intermediate and final diagnoses (fig. 9.11; also next chapter).

9.4

Summary

We described first steps towards an analytic framework for evaluating and developing CBR systems. Our approach tries to close the gap between high level characterisations, on the one side, and concrete systems, on the other side. Knowledge level analysis and CBR are merged to come up with a methodological framework (Althoff & Aamodt, 1996a+b). Technical/ergonomic criteria have been combined with domain/task criteria, as partially shown during the evaluation work within this document. Therefore, both kinds of criteria must be detailed to the method level as a reasonable grainsize. The task-method decomposition model is used to structure our framework. We developed a set of algorithms – and integrated them into the task-method decomposition hierarchy – that base on extracted methods, explicit knowledge representations as well as the implicit knowledge modelling of CBR-related systems and tools. Our current focus is on analytic tasks that have been structured into classification, diagnosis, and decision support sub-tasks. This has also been contrasted with synthetic tasks (Althoff & Bartsch-Spörl, 1996; Bartsch-Spörl, 1995a).

291

CHAPTER 10: CONCLUSIONS ON THE EVALUATION OF THE INRECA SYSTEM Besides the comparison with industrial CBR tools and some well-known CBR research prototype systems, INRECA has been mainly evaluated with respect to approaches within the MOLTKE (and INRECA) system family. However, this covers more than the systems explicitly mentioned here. There are a number of approaches that used the MOLTKE system architecture as a guideline for solving their respective problems (e.g., Weiß, 1993; Macchion, 1994; Faupel, 1992). In addition, basic assumptions underlying MOLTKE have been confirmed by other authors (e.g., Laczkovich, 1990; Horrix, 1991; Verweyen-Frank, 1990). The development of the D3 workbench and system family can also be seen to be analogous to the development of MOLTKE and INRECA (Bamberger, Gappa et al., 1992; Puppe & Goos, 1991; Bamberger & Goos, 1993; Goos, 1995). An alternative approach to MoCAS is, for instance, described in Portinale, Torasso et al. (1994). A very contrasting approach to the MOLTKE/INRECA approach on diagnostic tasks is the CREEK (Aamodt, 1991; 1994a) approach. This was a very strong reason to cover CREEK (and PROTOS) in the chapters 5 and 6. The evaluation of the INRECA system brought light to a number of different characteristics. One important aspect in the analysis of integrated systems is their level of integration. In INRECA we identified four different levels for the integration of induction and CBR (fig. 10.1).

Toolbox level

Co-operative level

Induction

Induction

Development / Execution

Workbench level

S3-Case Case-based reasoning

Results in CASUEL

Case-based reasoning

Induction

Communication between Modules Case-based reasoning

Induction

Case-based reasoning

Development / Execution

Seamless level

Kate

Development / Execution

Fig. 10.1 – Integration of induction and CBR in INRECA

From these levels we can also abstract more general descriptions for the integration of different methods (Priebisch, 1995; also sections 5.1.9 and 9.2.4):

292

Toolbox level: Here the integration is restricted to the common use of parts of the knowledge base. Reasoning is done by only one method. Exchange of reasoning results is not possible. Co-operative level: During the reasoning process different reasoning methods can be used. The exchange of intermediate results is possible using a common representation formalism. Workbench level: Beyond switching between available reasoning methods, they can also be aggregated to a combined reasoning method. For instance, a diagnostic system can carry out its classification sub-task using a rule-based module, and the test selection sub-task based on a CBR component (this is one possible strategy in the MOLTKE workbench). Seamless level: This is the highest level of integration. Different reasoning methods have been integrated within a new algorithm by use of an integrating element (like the Inrecatree, chapter 7). Thus, the change of reasoning methods is hidden for the user. The seamless integration of induction and CBR in INRECA (fig. 10.2) appears to be of extraordinary importance. Here INRECA contributes to the current state-of-technology in CBR (table 3.31; also chapter 7). It has been directly motivated by the evaluation of commercial CBR tools in 1994 (Althoff, Auriol et al., 1994; sections 2.3.1 and 2.3.2). Case-Based Reasoning

N0

N1

Induction

Number of Cases Nj

•Small case-base •Focus on CBR •Small amount of general knowledge

Nn

•Large case-base •Focus on induction •Large amount of generalised knowledge

Compilation of Knowledge Fig. 10.2 – System evolution over time of the seamlessly integrated INRECA system

Another advantage of INRECA over most competitors is its object-oriented case and knowledge representation (CASUEL). While such a representation language allows to cope with natural domains like the marine sponge domain (section 3.1.2.3; table 3.7), it also makes the processes of modelling the cases and the similarity relationships more complex (fig. 10.3).

293

Attribute 2

Attribute 1

Similarity assessment on attribute level Object 2

Object 1

Similarity assessment on object level

Similarity assessment on case level

Fig. 10.3 – Similarity assessment in INRECA

Thus, in Wess (1995) and Althoff, Auriol et al. (1995b) it is suggested to extend INRECA with a context structure similar to the context-graph in MOLTKE (section 1.3.1). Such a means would allow to structure domains like the travel agency domain (section 3.1.2.4). While the automatic update of INRECA's relevance matrix can be applied for classification and diagnostic tasks (section 9.3), for decision support tasks the underlying learning strategy needs a better focus, which in diagnostic and classification tasks is normally given by the underlying classes. A (simplified) exemplary context structure for the travel agency is shown in figure 10.4. Journey

Context

Motivation: private

Persons > 10

Motivation:business Target: town

Rules for Context Change

Holiday

Group

Business

Context

Fig. 10.4 – Exemplary contexts in INRECA (adapted from Wess, 1995)

The use of general knowledge for case retrieval and adaptation (fig. 10.5) underlines INRECA's good position if compared to other industrial CBR tools (table 3.13).

294

• • • • •

A-kind-of relations: describe inheritance relations Is-part-of relations: describe compositional relations Exclusion rules: exclude retrieval results that are not wanted Completion rules: complete the description of the problem Adaptation rules: adapt the found problem solution Adaptation Rules

Used during • Retrieval • Solution adaptation Exclusion Rules

Solution Case

Completion Rules

Solution adaptation in INRECA

Fig. 10.5 – General knowledge in INRECA

Besides the integration of induction and CBR, its object-oriented case representation, its use of general knowledge for retrieval and adaptation (including the model of solution oriented case adaptation), INRECA's main advantages are its efficient and robust retrieval strategies. Some improvements over known approaches implemented in INRECA are listed in figure 10.6 (the numbers mentioned in the right part of the figure mean the respective factor of improvement achieved by INRECA if compared with the implemented algorithms as described by Friedman et al., 1977, and Broder, 1990, where „main memory“ means the factor of improvement achieved if the cases are kept in the main memory etc.; i.e. the improvement is the highest if the cases are kept in an object-oriented database). Implemented extensions:

Improved performance:

• Dynamic bounds tests • Symbolic attributes • Unordered attribute values • Optimised partitioning • Multiple similarity relations • Disjunctive concept definitions • Optimised BWB and BOB tests • Generation of decision trees • Determination of weight vectors

•Main memory: •File system: •OODBMS:

30 - 100 200 - 500 500 - 1000

compared to Friedmann et al. (1977) and Broder (1990)

Fig. 10.6 – Improvements implemented in INRECA

To bring INRECA into the position to become a tool of daily use, the integration into technical environments and business processes is important (cf., e.g., Bacvanski, 1993). From this point of 295

view the following client-server model for INRECA has been developed in Wess (1995) (fig. 10.7). Application Environment

Case Retrieval and Adaptation

Database Server

Application

Application

Case Storage

INRECA-Server

Database Server Database Server

Application Developing Environment Domain Model Fig. 10.7– Client-server model of INRECA

Since INRECA focuses on analytic tasks (as all current commercial CBR tools), no processes had to be integrated into the case representation and a process oriented solution adaptation was not necessary. The INRECA approach allows different kinds of integration and covers a wide variety of applications. Such applications are easy to realise. INRECA is especially well-suited for large, externally stored case-bases. Important problems with existing (successful) diagnostic expert systems are that they are often built in a single-use-manner and need much effort for knowledge acquisition and maintenance. Thus, using existing approaches the overall development cycle is too slow for industrial real-life applications. What's missing in current research, is an approach which is both application-oriented (in the sense of being usable for real-life applications) and research-oriented (in the sense of integrating state-of-the-art methods). Such a system should be able to speed up the development of diagnostic knowledge-based systems to meet the underlying domain requirements.

296

Database Approach

INRECA CBR-Approach

ML and KBS Approach

Concept = Case Base + Similarity Measure • Knowledge coded into the “cases” (all situations) • Simple “similarity measure”

• Knowledge coded into the “similarity measure” (predicates) • No cases

Storing

Generalising Storing and Generalising

Fig. 10.8 – The INRECA-CBR approach to knowledge-based systems development

We hope that INRECA contributes to the solution of the above problem. INRECA outperformed the „1994 commercial CBR tools“ with respect to most of the evaluation criteria (chapter 3). Though the performance of the INRECA system was comparatively slow, this problem could be solved by a thorough optimisation (in addition, AcknoSoft's KATE-4 and later versions as well as tecInno's CBR-Works will solve this problem). INRECA appears to be the only CBR tool, at least if compared with commercial tools, that supports an optimisation of the executable system based on the system's experience. The INRECA system evolves smoothly along application development time, from a pure CBR approach, where each particular case is a piece of knowledge, to a more inductive approach where sub-sets of the cases are generalised into abstract knowledge (fig. 10.8). This process is supported by learning of implicit relevances as well as explicit preference rules. Here the INRECA/CBR approach to knowledge-based system development hopefully offers an additional degree of flexibility, as shown in figure 10.8 (also Richter, 1995), by building a bridge between the widely used database approach and more isolated approaches of machine learning and knowledge-based systems. Another kind of flexibility, which is offered by the INRECA/CBR approach, is that both is possible (Richter, 1996) to solve problems by finding explicit solutions (or adapting found solutions) and to simply retrieve (useful) information, as known from information retrieval (Fuhr, 1996). This flexibility of the INRECA/CBR approach will allow to build more integrated systems and to develop more applications, two main issues where CBR is about (Bartsch-Spörl and Wess, 1996b). While help-desk and call-centre applications are widely in use, CBR applications for technical diagnosis (sections 4.1 and 4.2) will soon increase in number. Later much more applications in the problem areas of planning, configuration, and construction will be developed (Bartsch-Spörl and Wess, 1996c). What is necessary is a good methodological support that allows an efficient and effective CBR application development. Hopefully, the ongoing INRECA follow-up projects (Althoff, Bergmann et al., 1995b; Manago, Althoff et al., 1995) will help here. 297

REFERENCES Aakvik, G., Aamodt, A., and Nordbo, I. (1991). A Knowledge Representation Framework Supporting Knowledge Modeling. In: Linster, M. and Gaines, B.R. (eds.), Proceedings of the European Knowledge Acquisition Workshop, 1-14 Aamodt, A. (1989). Towards Robust Expert Systems that learn from experience - an architectural framework. In: Boose, Gaines and Ganascia (1989), 311-326 Aamodt, A. (1991). A Knowledge-Intensive, Integrated Approach to Problem Solving and Sustained Learning. Ph. D. Thesis, University of Trondheim, Department of Computer Science and Telematics Aamodt, A. (1993a). A Case-Based Answer to Some Problems of Knowledge-based Systems. In: Sandewall, E. and Jansson, G. (eds.), Proceedings Scandinavian Conference on Artificial Intelligence, IOS Press Aamodt, A. (1993b). An Overview on CBR. Tutorial at the First European Workshop on CaseBased Reasoning (EWCBR-93), Kaiserslautern Aamodt, A. (1993c). Knowledge Level Modelling and the Symbol Level Gap. Slide copies of an invited presentation during the Knowledge Level Seminar at the University of Trondheim Aamodt, A. (1994a). Explanation-Based Case-Based Rasoning. In: Wess, Althoff & Richter, (1994), 274-288 Aamodt, A. (1994b). Case-Based Reasoning. In: C. Rouveirol (ed.), Proc. MLnet Summer School on Machine Learning and Knowledge Acquisition, 1-58, Dourdan Aamodt, A. (1994c). A Knowledge Representation System for Integration of General and CaseSpecific Knowledge. In: Proceedings of the IEEE Tools for Artificial Intelligence (TAI) conference, Nov. 1994 Aamodt, A. (1995). Knowledge Acquisition and learning by experience the role of case-specific knowledge. In: Kodratoff and Tecuci (1995), 197-245 Aamodt, A. and Althoff, K.-D. (1993). Problem Solving and Sustained Learning from Experience: Analyzing Methods with respect to Domain Characteristics. In: M. van Someren (ed.), Proc. MLnet Workshop on „Learning and Problem Solving“, Blanes Aamodt, A. and Nygård, M. (1995). Different roles and mutual dependencies of data, information, and knowledge - an AI perspective on their integration. Accepted for publication in Data and Knowledge Engineering Aamodt, A. and Plaza, E. (1994). Case-Based Reasoning: Foundational Issues, Methodological Variations and System Approaches. AI Communications Vol. 7, No. 1, 39-59 Aha, D. W. (1992). Generalizing from Case Studies: A Case Study. In: Sleeman & Edwards (1992), 1-10 Aha, D. W. (ed.) (1994). Proc. AAAI-94 Workshop on Case-Based Reasoning, Seattle, AAAI Press Allemang, D. (1994). Combining Case-based Reasoning and Task Specific Architectures. IEEE Expert Althoff, K.-D. (1992a). Eine fallbasierte Lernkomponente als integrierter Bestandteil der MOLTKE-Werkbank zur Diagnose technischer Systeme. Doctoral Dissertation, University of Kaiserslautern; also: Sankt Augustin, Germany: infix Verlag

298

Althoff, K.-D. (1992b). Machine Learning and Knowledge Acquisition in a Computational Architecture for Fault Diagnosis in Engineering Systems. In: M. Weintraub (ed.), Proc. International Machine Learning Conference, Workshop on „Computational Architectures for Supporting Machine Learning and Knowledge Acquisition“ in Aberdeen Althoff, K.-D. (1993). Lernverfahren in MOLTKE. In: Pfeifer and Richter (1993), 173-200 Althoff, K.-D. (1994a). A Framework for Describing, Analyzing, and Developing Integrated Architectures for Problem Solving and Sustained Learning in Complex Real World Domains. Unpublished manuscript Althoff, K.-D. (1994b). Manuelles und Automatisches Erstellen von Wissensbasierten Systemen: Diagnostisches Problemlösen. Lecture notes, Dept. of Computer Science, University of Kaiserslautern Althoff, K.-D. (1995a). Evaluating Case-Based Reasoning Systems. In: Proc. Workshop on Case-Based Reasoning: A New Force In Advanced Systems Development, 48-61, published by Unicom Seminars Ltd., Brunel Science Park, Cleveland Road, Uxbridge, Middlesex UB8 3P, UK Althoff, K.-D. (1995b). Problemlösemethoden von Expertensystemen: Klassifikation, Diagnose, Entscheidungsunterstützung. Lecture notes, Dept. of Computer Science, University of Kaiserslautern Althoff, K.-D. and Aamodt, A. (1996a). Zur Analyse fallbasierter Problemlöse- und Lernmethoden in Abhängigkeit von Charakteristika gegebener Aufgabenstellungen und Anwendungsdomänen. KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess), 10-15 Althoff, K.-D. and Aamodt, A. (1996b). Relating Case-Based Problem Solving and Learning Methods to Task and Domain Characteristics: Towards an Analytic Framework. Unpublished manuscript Althoff, K.-D. and Bartsch-Spörl, B. (1995). Call for Participation: Decision Support for CaseBased Applications. Initiative for systematically collecting information about existing CBR systems and applications, BSR Consulting (Munich) and Centre for Learning Systems and Applications (LSA) at the University of Kaiserslautern Althoff, K.-D. and Bartsch-Spörl, B. (1996). Decision Support for Case-Based Applications. Wirtschaftsinformatik 1/96, Special Issue on Case-Based Decision Support (edited by D. Ehrenberg), 8-16 Althoff, K.-D. and Weis jun., K.-H. (1996). An Evaluation of the INRECA CBR System for Decision Support and Diagnosis. In: H.-D. Burkhard, B. Bartsch-Spörl, D. Janetzko and S. Wess (eds.), Proc. of the 4th German Workshop on Case-Based Reasoning, (CBR-96) – System Development and Evaluation – Technical Report, Humboldt University, Berlin Althoff, K.-D. and Wess, S. (1991). Case-Based Knowledge Acquisition, Learning and Problem Solving in Diagnostic Real World Tasks. In: D. Smeed (ed.), Proc. of the fifth European Knowledge Acquisition for Knowledge-Based Systems Workshop (EKAW-91), Glasgow & Crieff; also in: M. Linster & B. Gaines (eds.), Proc. of the fifth European Knowledge Acquisition for Knowledge-Based Systems Workshop (EKAW‘91), GMD-Studien Nr. 211, Sept. 1992, 48-67 Althoff, K.-D. and Wess, S. (1992). Case-Based Reasoning and Expert System Development. In: F. Schmalhofer, G. Strube and T. Wetter (eds.), Contemporary Knowledge Engineering and Cognition, Springer Verlag Althoff, K.-D., Auriol, E., Barletta, R. and Manago, M. (1995a). A Review of Industrial CaseBased Reasoning Tools. AI Intelligence, Oxford, UK 299

Althoff, K.-D., Auriol, E., Bergmann, R., Breen, S., Dittrich, S., Johnston, R., Manago, M., Traphöner, R. & Wess, S. (1995b). Case-Based Reasoning for Decision Support and Diagnostic Problem Solving: The INRECA Approach. In: Bartsch-Spörl, Janetzko & Wess (1995), 63-72 Althoff, K.-D., Auriol, E., Holz, H., Manago, M., Meissonnier, A., Priebisch, C., Wess, S. & Wilke, W. (1994). An Evaluation of Commercial Case-Based Reasoning Tools. INRECA (Esprit P6322), Deliverable D5 Althoff, K.-D., Bergmann, R., Globig, C., Wess, S., Auriol, E. & Manago, M. (1995a). The Seamless Integration of Induction and CBR in INRECA. Unpublished manuscript, University of Kaiserslautern Althoff, K.-D., Bergmann, R., Maurer, F., Wess, S., Manago, M., Auriol, E., Conruyt, N., Traphöner, R., Bräuer, M. & Dittrich, S. (1993). Integrating Inductive and Case-Based Technologies for Classification and Diagnostic Reasoning. In: E. Plaza (ed.), Proc. ECML-93 Workshop on „Integrated Learning Architectures“ Althoff, K.-D., Bergmann, R., Maurer, M., Richter, M. M., Traphöner, R. and Wilke, W. (1995b). Wissensmodellierung und -akquisition für Case-Based Learning. Project proposal, Stiftung Rheinland-Pfalz für Innovation Althoff, K.-D., Bergmann, R., Wess, S., Manago, M., Auriol, E., Larichev, O. I., Bolotov, A. & Gurov, S. I. (1996). Integration of Induction and Case-Based Reasoning for Critical Decision Support Tasks in Medical Domains: The INRECA Approach. Centre for Learning Systems and Applications, LSA-REPORT 96-93E, University of Kaiserslautern Althoff, K.-D., Faupel, B., Kockskämper, S., Traphöner, R. and Wernicke, W. (1989). Knowledge Acquisition in the Domain of CNC Machining Centers: the MOLTKE Approach. In: Boose, Gaines & Ganascia (1989), 180-195 Althoff, K.-D., Kockskämper, S., Maurer, F., Stadler, M. and Wess, S. (1989). Ein System zur fallbasierten Wissensverarbeitung in technischen Diagnosesituationen. In: Retti, J. and Leidlmeier, K. (eds.), 5th Austrian Artificial-Intelligence-Conference, 65-70, Springer Verlag Althoff, K.-D., Maurer, F. and Rehbold, R. (1990). Multiple Knowledge Acquisition Strategies in MOLTKE. In: B. Wielinga et al. (eds.), Current Trends in Knowledge Acquisition (Proc. EKAW-90), IOS Press, 21-40 Althoff, K.-D., Maurer, F., Wess, S. & Traphöner, R. (1992). MOLTKE - An Integrated Workbench for Fault Diagnosis in Engineering Systems. In: S. Hashemi, J. P. Marciano & G. Gouaderes (eds.), Proc. 4th International Conference on AI & Expert Systems Applications (EXPERSYS-92), Paris, i.i.t.t. international Althoff, K.-D., Traphöner, R., and Wess, S. (1996). Efficient Integration of Induction and CaseBased Reasoning: The INRECA System. To appear in: Czap/Ohly/Jaenecke (eds.), Wissensorganisation im Wandel/Neue Formen der Wissensorganisation: Neuronale Netze, Fallbasiertes Schließen, Räumliches Schließen Althoff, K.-D., Wess, S. and Traphöner, R. (1995). INRECA - A Seamless Integration of Induction and Case-Based Reasoning, for Decision Support Tasks. In: K. Morik & J. Herrmann (eds.), Proc. 8th German Workshop on Machine Learning, Technical Report 580, University of Dortmund, 116-122 Althoff, K.-D., Wess, S., Bartsch-Spörl, B. and Janetzko, D. (eds.) (1992). Ähnlichkeit von Fällen beim fallbasierten Schließen. Proc. of the1st Meeting of the German Working Group on Case-Based Reasoning, University of Kaiserslautern, June 1992 Althoff, K.-D., Wess, S., Bartsch-Spörl, B., Janetzko, D., Maurer, F. & Voß, A. (1992). Fallbasiertes Schließen in Expertensystemen: Welche Rolle spielen Fälle für wissensbasierte Systeme?. KI 4, 14-21, Baden-Baden: FBO Verlag 300

Althoff, K.-D., Wess, S., Bergmann, R., Maurer, F., Manago, M., Auriol, E., Conruyt, N., Traphöner, R., Bräuer, M. and Dittrich, S. (1994). Induction and Case-Based Reasoning for Classification Tasks. In: Bock, Lenski & Richter (1994), 3-16 Althoff, K.-D., Wess, S., Weis, K.-H. jun., Auriol, E., Bergmann, R., Holz, H., Johnston, R., Manago, M., Meissonnier, A., Priebisch, C., Traphöner, R. and Wilke, W. (1995). An Evaluation of the Final Integrated System. Esprit Project INRECA (P6322), Deliverable D6 Anand, T. and Kahn, G. (1993). Focusing Knowledge-Based Techniques on Market Analysis. IEEE Expert Vol. 8, No. 4, 19-24 Anick, P., Simoudis, E., Croft, B., Mark, W. and Riesbeck, C. (eds.) (1993). Case-based Reasoning and Information Retrieval. Stanford University: AAAI – Spring Symposium Series, American association for Artificial Intelligence Arcos, J. L. and Plaza, E. (1994). A Reflective Architecture for Integrated Memory-Based Learning and Reasoning. In: Wess, Althoff & Richter (1994), 289-300 Armengol, E. and Plaza, E. (1994). A Knowledge Level Model of Case-Based Reasoning. In: Wess, Althoff & Richter (1994), 53-64 Ashley, K. D. (1990). Modeling Legal Argument: Reasoning with Cases and Hypotheticals. Cambridge: MIT Press Auriol, E. (1995). Intégration d'approches symboliques pour le raisonnement à partir d'exemples – L'induction et le raisonnement par cas dans le diagnostic technique. Ph.D. thesis, L'Université Paris-IX Dauphine Auriol, E., Manago, M., Althoff, K.-D., Wess, S. & Dittrich, S. (1995). Integrating Induction and Case-Based Reasoning: Methodological Approach and First Evaluations. In: J.-P. Haton, M. Keane & M. Manago (eds.), Advances in Case-Based Reasoning. Springer Verlag, 18-32 Auriol, E., Manago, M., Traphöner, R., Wess, S. and Conruyt, N. (1993). Report on Integration of Induction and Case-Based Reasoning - Approaches and Validation Methodology. Technical Report, ESPRIT P6322 (INRECA-D7) Auriol, E., Wess, S., Manago, M., Althoff, K.-D. and Traphöner, R. (1995). INRECA: A seamlessly integrated system based on inductive inference and case-based reasoning. In: Veloso & Aamodt (1995), 371-380 Avesani, P., Perini, A., and Ricci, F. (1993). Combining CBR and Constraint Reasoning in Planning Forest Fire Fighting. In: Richter, Wess et al. (1993), 235 - 239 Bablin, S. and Kashyap, R.L. (1992). Generating Fault Hypotheses with a Functional Model in Machine Fault Diagnosis. Applied AI, Vol. 6, No. 3, 353-382 Bacvanski, V. (1993). Expertensysteme in technischen Anwendungen: Strukturierung und Integration. In: O. Spaniol (ed.), Forschungsprojekte des Graduiertenkollegs Informatik und Technik, RWTH Aachen Bailin, S.C. and Henderson, S. (1992). A Case-Based Software Engineering Environment. In: Proc. AID92, 8 - 12 Bamberger, S. K. and Goos, K. (1993). Integration of Case-Based Reasoning and Inductive Learning Methods. In: Richter, Wess et al. (1993), 296 - 300 Bamberger, S., Gappa, U., Goos, K., and Poeck, K. (1993). Teilautomatische Wissenstransformation zur Unterstützung der Wissensakquisition. In: Puppe & Günter (1993) Bamberger, S., Gappa, U., Goos, K., Meinl, A., Poeck, K. and Puppe, F. (1992). Die DiagnostikExpertensystem-Shell D3. Handbuch. Institut für Logik, Komplexität und Deduktionssysteme der Universität Karlsruhe 301

Banatov, D., Nagarur, N., and Nitikhankasem, P. (1993). EXPERT-MM: A Knowledge-Based System for Maintenance Management. AI in Engineering Vol. 8, No. 4, 283-292 Bareiss, R. (1989). Exemplar-Based Knowledge Acquisition. London: Academic Press Barletta, R. (1994). A Hybrid Indexing and Retrieval Strategy for Advisory CBR Systems Built with REMIND. In: Keane, Haton & Manago (1994), 49-58 Bartlett (1932). Bartsch-Spörl, B. (1987). Ansätze zur Behandlung von fallorientiertem Erfahrungswissen in Expertensystemen. KI 1, 4, 32-36 Bartsch-Spörl, B. (1992). Warum es für die Bestimmung der Ähnlichkeit von Fällen beim fallbasierten Schließen noch keine Patentrezepte gibt und auch nicht geben wird.... In: Althoff, Wess et al. (1992) Bartsch-Spörl, B. (1994a). Representing and Using Cases in Visualisation-Oriented Design Domains. In: Keane, Haton & Manago (1994), 253-262 Bartsch-Spörl, B. (1994b). Zur größeren Reichweite fallbasierter Ansätze. In: KI, Juni/95, 75 Bartsch-Spörl, B. (1995a). KI-Methoden für innovative Design-Domänen. In: Richter & Maurer (1995), 137-151 Bartsch-Spörl, B. (1995b). Towards the Integration of Case-Based, Schema-Based and ModelBased Reasoning for Supporting Complex Design Tasks. In: Veloso & Aamodt (1995), 145-156 Bartsch-Spörl, B., Janetzko, D. and Wess, S. (eds.) (1995). Fallbasiertes Schließen - Grundlagen und Anwendungen. Proc. of the 3rd German Workshop on Case-Based Reasoning (at the XPS-95 conference, organised by AK-CBR), LSA-REPORT 95-02, University of Kaiserslautern Bartsch-Spörl, B. and Wess, S. (1996a). Gasteditorial. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Bartsch-Spörl, B. and Wess, S. (1996b). CBR ist Integration und Anwendungen. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Bartsch-Spörl, B. and Wess, S. (1996c). Epilog. To appear in: KI 1/96, Special Issue on CaseBased Reasoning (edited by B. Bartsch-Spörl and S. Wess) Baudin, C., Pell, B., and Kedar, S. (1994). Using Induction to Refine Information Retrieval Strategies. In: Proc. of the Twelth National Conference on Artificial Intelligence, AAAI Press / The MIT Press, 553-559 Baudin, C., Underwood, J. G. and Baya, V. (1993). Using Device Models to Facilitate the Retrieval of Multimedia Design Information. In: R. Bajcsy (ed.), Proc. of the 13th IJCAI, 1237-1243 Benjamin, M., Viana, T., Corbett, K., and Silva, A. (1993). Satisfying Multiple-Rated Constraints in a Knowledge-Based Decision Aid. In: IEEE 9th Conference on Artificial Intelligence for Application, 277-284 Benjamins, R. (1993). Problem solving methods for diagnosis. Ph.D. thesis, University of Amsterdam Bennet, J. S. (1985). ROGET – A knowledge based system for acquiring the conceptual structure of a diagnostic expert system. Journal of automated reasoning 1, 49-74 Bento, C. and Costa, E. (1993). A Similarity Metric for Retrieval of Cases Imperfectly Described and Explained. In: Richter, Wess et al. (1993), 8-13 Bergmann, R. and Eisenecker, U. (1994). Fallbasiertes Schließen zur Unterstützung der Wiederverwendung objektorientierter Software: Eine Fallstudie. In: Richter and Maurer (1995) 302

Bergmann, R., Maurer, F. and Pews, G. (1994). Objektorientierte Wiederverwendung und fallbasiertes Schließen, Teil 3: Merkmale und Ähnlichkeitsmaße. Technical report, University of Kaiserslautern Bergmann, R., Muñoz-A., H. and Veloso, M. (1996). Fallbasiertes Planen: Ausgewählte Methoden und Systeme. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Bergmann, R., Pews, G. and Wilke, W. (1994). Explanation-Based Similarity: A Unifying Approach for Integrating Domain Knowledge into Case-Based Reasoning for Diagnosis and Planning Tasks. In: Wess, Althoff & Richter (1994), 182-196 Bergmann, R. and Wilke, W. (1995). Building and Refining Abstract Planning Cases by Change of Representation Language. Centre for Learning Systems and Applications (LSA), LSA95-07E, University of Kaiserslautern Bergmann, R., Wess, S., Traphöner, R. and Breen, S. (1994). Using Background Knowledge in the Integrated System INRECA. INRECA Deliverable D29, Version 1 Bernard, J.P. and Durocher, D. (1993). LANGAGE: An Expert System for Diagnosis in a RealTime Context. In: IEEE 9th Conference on Artificial Intelligence for Application, 328337 Bichindaritz, I. (1994). A Case-Based Reasoning System Using a Control Case base. In: Cohen, A.G. (eds.), ECAI '94 - 11th European Conference on Artificial Intelligence, John Wiley and Sons Bisson, G. (1991). KGB: A Generator of Knowledge-bases. In: Kodratoff, Y. (ed.), Machine Learning - EWSL '91, Springer-Verlag, 137 Bisson, G. (1992). Conceptual Clustering in a First Order Logic Representation. In: Neumann, B. (eds.), ECAI '92 - European Conference on Artificial Intelligence, ECCAI, John Wiley and Sons, 458-462 Bisson, G. (1992). Learning in FOL with a Similarity Measure. In: AAAI-92: Proc. of the Tenth National Conference on Artificial Intelligence, Amarican Association for Artificial Intelligence, AAAI Press / The MIT Press, 83-87 Bittner, P. (1995). Untersuchungen zur Verwendbarkeit von Interpolation-Based Grid Files für das ähnlichkeitsbasierte Retrieval beim Fallbasierten Schließen. Diploma thesis, University of Kaiserslautern Blachmann, N. M. (1966). Noise and its effect on Communication. Mc Graw-Hill Bock, H. H., Lenski, W. and Richter, M. M. (eds.) (1994). Information Systems and Data Analysis, Prospects-Foundations-Applications. Proc. 17th Annual Conference of the GfKl, University of Kaiserslautern, 1993. Springer Verlag Bonarini, A. and Maniezzo, V. (1991). Integrating Expert systems and Decision-support systems: Principles and practice. Knowledge-Based Systems Vol. 4, No. 3, 172-176 Bonissone, P. P. and Ayub, S. (1992). Similarity Measure for Case-Based Reasoning Systems. In: B. Bouchon-Meunier, L.V.R.R.Y. (eds.), IPMU '92 - Advanced Methods in Artificial Intelligence, Springer-Verlag Bonissone, P. P. and Dutta, S. (1990). Integrating case-based and rule-based reasoning: the possibilistic connection. In: Proc. 6th Conference on Uncertainty in Artificial Intelligence, Cambridge, MA Boose, J., Gaines, B. and Ganascia, J.-G. (eds.) (1989). Proc. EKAW-1989. Bourne, J.R., Liu, H.H., Orogo, C.D., Collins, G.C., Uckun, N.S., and Brodersen, A.J. (1991). Organizing and understanding beliefs in advice-giving diagnostic systems. IEEE Transactions on Knowledge and Data Engineering Vol. 3, No. 3, 269-280 303

Branskat, S. and Linster, M. (1992). Cases as the Bases for Knowledge Acquisition in the PreFormal Phases of Knowledge Acquisition. In: Ohlbach, H.J. (eds.), GWAI-92: Advances in Artificial intelligence, Springer-Verlag, 100-109 Branting, K. and Aha, D. W. (1995). Stratified Case-Based Reasoning: Reusing Hierarchical Problem Solving Episodes. In: C. S. Mellish (ed.), Proc. IJCAI-95, 384-390 Brazdil, P. B. (ed.) (1993). Machine Learning: ECML-93. Springer Verlag Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984). Classification and Regression Trees. Belmont, CA: Wadsworth Breuker, J. A., Wielinga, B. J., van Someren, M., de Hoog, R., Schreiber, A. T., de Greef, P., Bredeweg, B., Wielemaker, J., Billaut, J. P., Davoodi, M. and Hayward, S. A. (1987). Model driven knowledge acquisition: Interpretational models. Esprit project P1098, Deliverable D1 (task A1), University of Amsterdam and STL Ltd. Brinkop, A., Kockskämper, S., Schick, M. and Zimmermann-Sturm, M. (1995). Die Integration verschiedener Modellierungsarten für die Modellbasierte Diagnose. KI 6/95, 46-50 Broder, A. J. (1990). Strategies for efficient nearest neighbour search. Pattern Recognition 23, 171-178 Brown, M. (1992). Incorporating Similarity Measures into Case Retrieval Using Analogue Marker Passing. In: Neumann, B. (eds.), ECAI '92 - European Conference on Artificial Intelligence, ECCAI, John Wiley and Sons, 590-592 Brüninghaus, S. (1994). DANIEL: Integrating Case-Based and Rule-Based Reasoning. In: Proc. of the Twelth National Conference on Artificial Intelligence - AAAI '94, AAAI Press / The MIT Press, 1428 Bunke, H. and Messmer, B.T. (1993). Similarity Measures for Structured Representations. In: Richter, Wess et al. (1993), 26-36 Buntine, W. and Niblett, T. (1992). Technical Note: A Further Comparison of Splitting Rules for Decision-Tree Induction. Machine Learning Vol. 8, No. 1, 75-86 Burkhard, H. D. (1995). Case Retrieval Nets and Cognitive Modelling. In: L. Dreschler-Fischer and S. Pribbenow (ed.), KI-95 Activities: Workshops, Posters, Demos, Gesellschaft für Informatik Burkhard, H. D., Kühnel, R. and Pirk, P. (1994). Case-based diagnosis in a technical domain. In: I. Plander (eds.), Artificial Intelligence and Information-Control Systems of Robots, World Scientific Publishing Bykat, A. (1991). Intelligent Monitoring and Diagnosis Systems: A Survey. Applied AI Vol. 5, No. 4, 339-352 Cain, T., Pazzani, M.J., and Silverstein, G. (1991). Using Domain Knowledge to Influence Similarity Judgements. In: Proc. CBR91, 191 - 199 Cardie, C. (1993). Using decision trees to improve case-based learning. Proc. 10th Int. Conf. on Machine Learning, 25-32 Catlett, J. (1991). On Changing Continuous Attributes into Ordered Discrete Attributes. In: Kodratoff, Y. (eds.), Machine Learning - EWSL '91, Springer-Verlag, 164-178 Catlett, J. (1992). Peepholing: Choosing Attributes Efficiently for Megainduction. In: Sleeman, D. and Edwards, P. (eds.), ML '92 - Proceedings of the Ninth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 49-54 Chadha, S.R., Mazlack, L.J., and Pick, R.A. (1991). Using existing knowledge sources (cases) to build an expert system. Expert Systems Vol. 8, No. 1, 3-12 Chandrasekaran, B. and Johnson, T. (1993). Generic Tasks and Task Structures: History, Critique and New Directions. In: David, Krivine & Simmons (1993), 232-272 304

Chandrasekaran, B. and Mittal, S. (1983). On deep versus compiled approaches to diagnostic problem solving. International Journal of Man Machine Studies 19, 425-436 Chi, R.T.H. and Kiang, M.Y. (1993). Reasoning by coordination: An integration of case-based and rule-based reasoning systems. Knowledge-Based Systems Vol. 6, No. 2, 103-113 Chou, P.A. (1991). Optimal partitioning for classification and regression tree. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 13, No. 4, 340-354 Chu, B.T.B. and Reggia, J.A. (1991). Modeling Diagnosis at Multiple Levels of Abstraction. I. Representing causal relations at multiple levels of abstraction. International Journal of Intelligent Systems Vol. 6, 617-644 Clancey, W. (1985). Heuristic classification. Artificial Intelligence 27-3, 289-350 Clancey, W. and Shortliffe, E. (eds.) (1984). Readings in medical artificial intelligence. Addison-Wesley Cognitive Systems, Inc. (1994). REMIND Users Manual, 220-230 Commercial Street, Boston MA 02109, USA, Jan. 1994 Cohen, P. R. (1989). Evaluation and Case-based Reasoning. In: Hammond (1989b), 168-172 Cohen, P. R. and Howe, A. E. (1988a). How Evaluation Guides AI Research. AI Magazine 9, No. 4, Winter 1988, 35-43 Cohen, P. R. and Howe, A. E. (1988b). Toward AI Research Methodology: case Studies in Evaluation. IEEE Transactions on Systems, Man, and Cybernetics Coleman, K. (1992). The AI Marketplace in the year 2000. AI Expert, Jan., 35-38 Console, L., Portinale, L. and Dupré, D. T. D. (1991). Focusing Abductive Diagnosis. AI Communications Vol. 4, No. 2/3, 88-97 Cost, S. and Salzberg, S. (1993). A Weighed Nearest Neighbor Algorithm for Learning with Symbolic Features. Machine Learning Vol. 10, No. 1, 57-78 Croft, W.B. (1993). Knowledge-Based and Statistical Approaches to Text Retrieval. IEEE Expert Vol. 8, No. 2, 8-12 Cunis, R. and Neumann, B. (1991). Fallbasierte Diagnoseunterstüzung für ein flexibles Fertigungssystem. In: Welche Rolle spielle Fälle für wissensbasierte Systeme? Workshop der GWAI-91, LKI, Uni Hamburg Cunningham, P. (1994). The Future of Adaptation in CBR Research. Slide copies of an invited talk at the University of Kaiserslautern, December 15, 1994 Dasarathy, B. V. (ed.) (1991). Nearest Neighbour (NN) Norms. NN Pattern Classification Techniques. IEEE Computer Society Press David, J.-M., Krivine, J.-P. & Simmons, R. (eds.) (1993).Second Generation Expert Systems. Springer Verlag Davis, R. (1993). Retrospective on 'Diagnostic reasoning based on structure and behavior'. Artificial Intelligence Vol. 59, 149-157 De Kleer, J. D., Mackworth, A. K., and Reiter, R. (1992). Characterizing Diagnoses and Systems. Artificial Intelligence Vol. 56, 197-222 De la Ossa, A. (1992). Knowledge Adaptation: Analogical Transfer of Empirical Knowledge Across Domains for Case-Based Problem Solving. Doctoral Dissertation, University of Kaiserslautern Dearden, A. M. and Gridge, D. G. (1993). Choosing a reasoning style for a knowledge-based system: lessons from supporting a help desk. The Knowledge Engineering Review Vol. 8, No. 1, 261-222

305

Decker, K. S., Durfee, E. H. and Lesser, V. R. (1988). Evaluating Research in Cooperative Distributed Problem Solving. Dept. of Computer and Information Science, TR 8889, University of Masschusetts Derwand, G. (1994). Effizientes Retrieval und Extraktion von Entscheidungswissen mit k-dBäumen. Diploma thesis, University of Kaiserslautern Döring, U. (1994). Inferenzmechanismen unter Berücksichtigung von Unschärfe und Unsicherheit. Diploma thesis, Technical University of Ilmenau, Faculty for Computer Science and Automatisation, Institute for Theoretical and Technical Computer Science Döring, U. and Burkhardt, R. (1995). Überlegungen zu einem Bewertungssystem für objektorientierte Software-Projekte. In: Philippow, Doberkat et al. (1995), 21-30 Drenth, H. & Morris, A. (1992). Prototyping expert solutions: an evaluation of Crystal, Leonardo, GURU and ART-IM. Expert Systems 9, 1, 35-45 Drenty, H. and Morris, A. (1992). Prototyping expert solutions: an evaluation of Crystal, Leonardo, GURU and ART-IM. Expert Systems Vol. 9, No. 1, 35-46 Drosdowski, G., Köster, R., Müller, W., Scholze-Stubenrecht, W. (1963). Etymologie – Herkunftswörterbuch der deutschen Sprache. Mannheim/Wien/Zürich: Bibliographisches Institut Dvorak, D. and Kuipers, B. (1991). Process Monitoring and Diagnosis: A Model-Based Approach. IEEE Expert Vol. 6, No. 3, 67-74 Ehrenberg, D. (1996). Editorial. Wirtschaftsinformatik 1/96, special issue on case-based decision support (edited by D. Ehrenberg) Eshelman, L. (1988). MOLE: A Knowledge-Acquisition Tool for Cover-and-Differentiate Systems. In: Marcus, S. (ed.)Automating Knowledge Acqusition for Expert Systems, Kluwer Academic Publishers, Ch. 3, 37-80 Fanni, A., Diana, P., Giuaa, A., and Perezani, M. (1993). Qualitative Dynamic Diagnosis of Circuits. AI for Engineering Design, Analysis and Manufacturing Vol. 7, No. 1, 53-64 Fattah, Y.E. and O'Rourke, P. (1993). Explanation-Based Learning for Diagnosis. Machine Learning Vol. 13, No. 1, 35-70 Faupel, B. (1992). Ein modellbasiertes Akquisitionssystem für technische Diagnosesysteme. Doctoral Dissertation, RWTH Aachen Fayyad, U. M. (1994). Branching on Attribute Values in Decision Tree Generation. In: Proc. of the Twelth National Conference on Artificial Intelligence - AAAI '94, AAAI Press / The MIT Press, 601-606 Fayyad, U. M. and Irani, K. B. (1993). Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning. In: R. Bajcsy (ed.), Proc. of the 13th IJCAI, 10221027 Feigenbaum, E. A., Friedland, P. E., Johnson, B. B., Nii, H. P., Schorr, H., Shrobe, H., and Engelmore, R. S. (1994). Knowledge-Based Systems Research and Applications in Japan. AI magazine Vol. 15, No. 5, 28-45 Feltovich, P., Johnson, P., Moller, J., Swanson, D. (1984). LCS – The role and development of medical knowledge in diagnostic expertise. In: Clancey and Shortliffe (1984), 275-319 Fensel, D. and Wiese, M. (1993). Refinement of Rule Sets with JoJo. In: Brazdil, P.B. (eds.), Machine Learning: ECML-93, Springer-Verlag, 280-296 Fink, P. and Lusth, J. (1987). Expert systems and diagnostic expertise in the medical and electrical domains. IEEE Transactions on Systems, Man, Cybernetics Vol. SMC-17-3, 340-349, May/June 87

306

Finn, T. (1995). Meister der Diagnose. Sonderdruck aus Top-Business 2/1995, Danet GmbH, Darmstadt Fischer, O., Jr., J.W.S., and Smith, P. (1992). Extending the Indexing Vocabulary of Case-based Reasoning with Task Specific Features. In: IEEE 8th Conference on Artificial Intelligence for Applications, 226-232 Fluck, W. (1992). Anwendungsmöglichkeiten Verteilter Künstlicher Intelligenz in Expertensystemen zur Bestimmung von Laufkäfern. Diploma Thesis, University of Kaiserslautern Fluck, W. (1993). Einsatz klassifizierender Diagnosesysteme für Anwendungen in der Determination von Insekten. Technical Report, University of Kaiserslautern Francis, A.G. and Ram, A. (1993). The Utility Problem in Case-Based Reasoning. In: Leake, D. (ed.) Case-Based Reasoning, AAAI Press Friedman, J. H., Bentley, J. L. & Finkel, R. A. (1977). An algorithm for finding best matches in logarithmic expected time. ACM Trans. Math. Software 3, 209-226 Friedrich, G. (1993). Model-Based Diagnosis and Repair. AI Communications Vol. 6, No. 3/4, 187-206 Fuhr, N. (1996). Information-Retrieval Ansätze im CBR. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Gaines, B.R. and Compton, P. (1993). Induction of meta-knowledge about knowledge discovery. IEEE Transactions on Knowledge and Data Engineering Vol. 5, No. 6, 990-992 Gaschnig, J., Klahr, P., Pople, H., Shortliffe, E. & Terry, A. (1983). Evaluation of Expert Systems: Issues and Case Studies. In: Hayes-Roth, Waterman & Lenat (1983), 241-282 Gascuel, O. and Caraux, G. (1992). Statistical Significance in Inductive Learning. In: Neumann, B. (eds.), ECAI '92 - European Conference on Artificial Intelligence, ECCAI, John Wiley and Sons, 435-439 Geissman, J. R. and Schultz, R. D. (1988). Verification and Validation. AI Expert 3, 2, 26-33 Gentner, D. and Forbus, K.D. (1991). Mac/Fac: A Model of Similarity-Based Retrieval. In: Proceedings of the 13th Annual Conference of the Cognitive Science Society, 504–509 Ginsberg, A.F. (1993). A Unified Approach to Automatic Indexing and Information Retrieval. IEEE Expert Vol. 8, No. 5, 46-56 Giordana, A., Saitta, L., Bergadano, F., Brancadori, F., and Marchi, D.D. (1993). ENIGMA: A system that learns diagnostic knowledge. IEEE Transactions on Knowledge and Data Engineering Vol. 5, No. 1, 15-28 Globig, C. (1995). Fallbasiertes Repräsentieren und Lernen von Begriffsmengen. In: BartschSpörl, Janetzko & Wess (1995) Globig, C. and Althoff, K.-D. (eds.). Proc. 7th German Workshop on Machine Learning. Centre for Learning Systems and Applications (LSA), LSA-95-01, University of Kaiserslautern Globig, C., Kamp, G. and Lange, S. (1998). ... In: M. Lenz, B. Bartsch-Spörl et al. (1998), … Globig, C. and Lange, S. (1994). On Case-Based Representability and Learnability of Languages. In: S. Arikawa & K.-P. Jantke (eds.), Algorithmic Learning Theory, Springer Verlag Globig, C. and Wess, S. (1993). Case-Based and Symbolic Classification Algorithms - A Case Study Using Version Space. In: Richter, Wess et al. (1993), 133-138 Golding, A. R. & Rosenblum, P. S. (1991). Improving Rule-Based Systems Through Case-Based Reasoning. Proc. AAAI Conference 1991

307

Golding, A.R. and Rosenbloom, P.S. (1993). Improving Rule-Based Systems through CaseBased Reasoning. In: Wilkins, B.G.B.D.C. (ed.), Readings in Knowledge Acquisition and Learning, Morgan Kaufmann, 759-764 Goos, K. (1992). Ähnlichkeit in CcC+. In: Althoff, Wess et al. (1992) Goos, K. (1995). Fallbasiertes Klassifizieren – Methoden, Integration und Evaluation. Doctoral Dissertation, University of Würzburg Graner, N. & Sleeman, D. (1993). A Multistrategy Knowledge Refinement and Acquisition Toolbox. In: E. Plaza (ed.), ECML-93 workshop on „Integrated Learning Architectures", Vienna Grelinger, G. and Modizet-Mahoudeaux, P. (1992). A Fully Integrated Real-Time Multi-Tasking Knowledge-Based System: Application to an On-Board Diagnostic System. In: IEEE 8th Conference on Artificial Intelligence for Applications, 310-316 Grogono, P., Batarekh, A., Preece, A., Shinghal, R. and Suen, C. (1991). Expert system evaluation techniques: a selected bibliography. Expert Systems 8, 4, 227-239 Guida, G. and Mauri, G. (1993). Evaluating Performance and quality of knowledge-based systems. IEEE Transactions on Knowledge and Data Engineering Vol. 5, No. 2, 204-224 Guida, G. and Tasso, C. (eds.) (1989).Topics in Expert System Design. North Holland Hammond, K. (1989a). Case-Based Planning. Academic Press Hammond, K. (ed.) (1989b). Proc. of the 2nd Darpa Workshop on Case-Based Reasoning. Morgan Kaufmann Hanney, K., Keane, M., Smyth, B. and Cunningham, P. (1995). Systems, tasks, and adaptation knowledge: Revealing some revealing dependencies. In: Veloso & Aamodt (1995), 461470 Hanson, S. J. (1990). Conceptual Clustering and Categorization: Bridging the Gap between Induction and Causal Models. In: Kodratoff and Michalski (1990), 235-268 Harmon, P. (1992). Overview: Commercial Case-Based Reasoning Products. Intelligent Software Strategies, January 1992. Hart, A. (1984). Experience in the use of an inductive system in knowledge engineering. M. Bramer (ed.), Research and Development Systems, Cambridge University Press, 117-126 Hayes-Roth, F., Waterman, D. A. and Lenat, D. B. (1983). Building Expert Systems. Reading, Massachusetts: Addison Wesley Heath, D., Kasif, S. and Salzberg, S. (1993). Decision Trees in Numerical Attribute Spaces. In: R. Bajcsy (ed.), Proc. of the 13th IJCAI, 1002-1007 Heinrich, S. (1994). Simulationsalgorithmen. Skript, University of Kaiserslautern Heinrich, S. (1995). Stochastische Algorithmen. Skript, University of Kaiserslautern Hekmatpour, A. and Elkan, C. (1993). Categorization-Based Diagnostic Problem Solving in the VLSI Design Domain. In: IEEE 9th Conference on Artificial Intelligence for Applications, 121-127 Hendler, J. (1995). Experimental AI systems. In: Journal of Experimental and Theoretical AI 7, 1-5 Hennessy, D. and Hinkle, D. (1992). Applying Case-Based Reasoning to Autoclave Loading. IEEE Expert Vol. 7, No. 5, 21-35 Henrion, M., Breese, J.S., and Horvitz, E.J. (1991). Decision Analysis and Expert Systems. AI magazine Vol. 12, No. 4, 64-91 Hilal, D. K. and Soltan, H. (1991). A suggested descriptive framework for the comparison of knowledge-based systems methodologies. Expert Systems 8, 2, 107-114 308

Hofmann, M. O., Cost, T. L., and Whitley, M. (1992). Model-Based Diagnosis of the Space Shuttle Main Engine. AI for Engineering Design, Analysis and Manufacturing Vol. 6, No. 3, 131-148 Hollnagel, E. (1989). Evaluation of expert systems. In: Guida & Tasso (1989), 377-418 Holyoak, K. and Thagard, P. (1987). Analogical mapping by constraint satisfaction. Unpublished manuscript Holyoak, K. and Thagard, P. R. (1987). Analogical Mapping by constraint satisfaction. Unpublished manuscript Hong, D., Xiuwen, G., Shuzi, Y., and Shadong, Z. (1993). An Implementation Architecture of Knowledge-Based System for Engine Diagnosis. Applied AI Vol. 7, No. 4, 397-418 Hornby, A. S., Cowie, A. P. and Gimson, A. C. (1983). Oxford Advanced Learner‘s Dictionary of Current English. Cornelsen & Oxford University Press Horrix, C. (1991). Expertensysteme zur Fehlerdiagnose von Werkzeugmaschinen. Werkstatt und Betrieb 124, 2, 101-104 Janetzko, D. and Schult, T. (eds.) (1993). Fälle in hybriden Systemen. Universität Freiburg, Beiträge zum 2. Workshop des AK-CBR Janetzko, D. and Strube, G. (1992). Case-based Reasoning and Model-based Knowledge Acquisition. In: F. Schmalhofer, G. Strube and T. Wetter (eds.), Contemporary Knowledge Engineering and Cognition, Springer Verlag Janetzko, D., Wess, S., and Melis, E. (1992). Goal-Driven Simmilarity Assessment. In: Althoff, Wess et al. (1992) Janning, T. (1992). Requirements Engineering und Programmieren im Großen – Integration von Sprachen und Werkzeugen. Deutscher Universitäts-Verlag Jantke, K. P. (1992). Case-Based Learning in Inductive Inference. In: Proc. COLT-92 Jantke, K. P. and Lange, S. (1989). Algorithmisches Lernen. In: J. Grabowski, K. P. Jantke & H. Thiele (eds.), Grundlagen der Künstlichen Intelligenz, Berlin: Akademie-Verlag, 246277 Jarke, M. (1992). Strategies for Integrating CASE Environments. IEEE Software , 54-61 Jarke, M., Bubenko, J., Rolland, C., Sutcliffe, A. and Vassiliou, Y. (1992). Theories Underlying Requirements Engineering: An Overview of NATURE at Genesis. Aachener InformatikBerichte No. 1992-36, Technical University of Aachen Jarke, M., Jeusfeld, M. A., Peters, P. and Pohl, K. (1996). Coordinating Distributed Organizational Knowledge. In: Maurer (1996), 18 pages John, G.H. (1994). Finding Multivariate Splits in Decision Trees Using Function Optimization. In: Proc. of the Twelth National Conference on Artificial Intelligence - AAAI '94, AAAI Press / The MIT Press, 1428 Johnston, R. (1994). Case-based Reasoning and Institutional Memory. Irish Engineers‘ Journal, May 1994 Jones, W.T., Martin, W.S., McWilliams, M.E., and Nabors, M.V. (1991). Knowledge-Based System for Assessment of Potential Entrepreneurs. Applied AI Vol. 5, No. 3, 253-266 Kandell, A. (1982). Fuzzy Techniques in Pattern Recognition. New York: Wiley Kalkanis, G. and Conroy, G.V. (1992). Inductive Learning of Effective and Efficient Classification Rules. In: Neumann, B. (eds.), ECAI '92 - European Conference on Artificial Intelligence, ECCAI, John Wiley and Sons Keane, M., Haton, J. P. and Manago, M. (eds.) (1994). Proc. Second European Workshop on Case-Based Reasoning (EWCBR-94). AcknoSoft Press, Paris 309

Kehoe, A. and Parker, G.A. (1991). An IKB defect classification system for automated industrial radiographic inspection. Expert Systems Vol. 8, No. 3, 149-158 Keravnou, E. T. and Johnson, L. (1986). Competent expert systems; a case study in fault diagnosis. Kogan Page Ltd. Kibler, D. & Aha, D. W. (1987). Learning representative exemplars of concepts: An initial case study. Proc. of the Fourth International Workshop on Machine Learning, pp. 24-30., Irvine, CA: Morgan Kaufmann Kira, K. and Rendell, L.A. (1992). A Practical Approach to Feature Selection. In: Sleeman, D. and Edwards, P. (eds.), ML '92 - Proceedings of the Ninth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 249-256 Kirby, J. and Meißner, D.K. (1991). Integration of expert systems in advanced fourth generation languages. Expert Systems Vol. 8, No. 4, 241-250 Kirchhoff, S. (1993). Abbildungsqualität von wissensbasierten Systemen – Eine Methodologie zur Evaluierung. Bergisch Gladbach, Köln: Josef Eul Verlag Kirn, S. (1992). Modeling the Competence of Cooperative Problem Solvers-Can a CBR Aproach Solve the Knowledge Representation Problem?. In: Althoff, Wess et al. (1992), 157-162 Kleiter, G.D. (1992). Bayesian diagnosis in expert systems. Artificial Intelligence Vol. 54, 1-32 Kobayashi, S. and Nakamura, K. (1991). Knowledge Compilation and Refinement for Fault Diagnosis. IEEE Expert Vol. 6, No. 5, 39-46 Kockskämper, S. (1996). Vorgangsmodelle – Ein Ansatz zur Repräsentation und Analyse zeitabhängigen Verhaltens bei der Überwachung und Diagnose technischer Systeme. Doctoral Dissertation, University of Hamburg (forthcoming) Kockskämper, S., Neumann, B., Josub, A., and Müller, H. (1993). Die Anwendung modellbasierten Schließens bei der Diagnose schiffstechnischer Anlagen. In: Puppe and Günter (1993), 14-27 Kodratoff, Y. and Michalski, R. (eds.) (1990). Machine Learning: An Artificial Intelligence Approach (Volume III). San Mateo, CA: Morgan Kaufmann Kodratoff, Y. and Tecuci, G. (eds.) (1995). Integration of Knowledge Acquisition and Machine Learning. Kluwer Academic Publishers, forthcoming Kohonen, T. (1989). Self-Organising and Associative Memory. Springer Verlag, 3rd edition Kohonen, T. (1990). The self-organising map. Proceedings of the IEEE, 78(9), 1464-1480 Koks, G. (1992). Entwicklung einer Testumgebung zur Bewertung der Integrationsmöglichkeiten von GenRule, PATDEX und der MOLTKE-Shell. Diploma thesis, University of Kaiserslautern Kolodner, J. L. (1983). Maintaining organization in a dynamic long term memory. Cognitive science, Vol. 7, 243-280. Kolodner, J. L. (1991). Improving Human Decision Making through Case-Based Decision Aiding. AI magazine Vol. 12, No. 2, 52-68 Kolodner, J. L. (1992). An Introduction to Case-Based Reasoning. AI Review Vol. 6, No. 1, 3-34 Kolodner, J. L. (1993). Case-Based Reasoning. Morgan Kaufmann Kolodner, J. L. and Riesbeck, C. (eds.) (1986). Experience, memory and reasoning. Lawrence Erlbaum Ass. Publ. Kononenko, I. (1993). Inductive and Bayesian Learning in Medical Diagnosis. Applied AI Vol. 7, No. 4, 317-338 Koopmans, L. H. (1987). Introduction to Contemporary Statistical Methods. Second Edition, Duxbury, Boston 310

Koton, P. (1988). Using experience in learning and problem solving. Proceedings of AAAI-88, 256-261 Koton, P. (1989). Evaluated case-based problem solving. Ph.D. thesis, Laboratory of computer science, MIT/LCS/TR-441 Koton, P. (1993). Combining Causal Models and Case-Based Reasoning. In: David, Krivine & Simmons (1993), 79-92 Kriegsmann, M. and Barletta, R. (1993). Building a Case-Based Help Desk Application. IEEE Expert Vol. 8, No. 6, 18-26 Krovidy, S. and Wee, W.G. (1993). Wastewater Treatment Systems from Case-Based Reasoning. Machine Learning Vol. 10, No. 3, 341-263 Kruse, R., Gebhardt, J. and Klawonn, F. (1993). Fuzzy-Systeme. Stuttgart: Teubner-Verlag Kubat, M. (1991). Conceptual inductive learning in the case of unreliable teachers. Artificial Intelligence Vol. 52, 169-182 Kuipers, B.J. (1993). Reasoning with qualitative models. Artificial Intelligence Vol. 59, 125-132 Kumara, S.R.T., Ham, I., Ohsuga, S., Tsatsoulis, C., Ramesh, R., Frost, V., and , R.L.K. (1992). Intelligent Computer Integrated Manufacturing (I-CIM). Applied AI Vol. 6, No. 4, 529252 Kurtz, V. (1996). Regelung der Phosphatelimination auf der kommunalen Kläranlage Elschbach mit einer Fuzzy-SPS. Diploma thesis, University of Kaiserslautern Kurz, E., Gräble, A., and Holzbach, W. (1993). Einsatz eines Expertensystems zur qualitätsoptimalen Aufarbeitung von Zwischenprodukten und Produktionsrückläufen in der Reifenindustrie. In: Qualitätssicherung mit wissensbasierten Systemen, VDI-Gesellschaft Mess- und Automatisierungstechnik, VDI-Verlag, 77-86 Lackinger, F. and Nejdl, W. (1993). Diamon: A Model-Based Troubleshooter Based on Qualitative Reasoning. IEEE Expert Vol. 8, No. 2, 33-40 Laczkovich, R. R. (1990). Expertensysteme zur technischen Fehlerdiagnose. Eine betriebswirtschaftlich orientierte Analyse ihrer Leistungs- und Gestaltungspotentiale. Berlin: Erich Schmidt Verlag Langley, P. (1987). Research papers in machine learning. Machine Learning 2, 3, 195-198 Langley, P. and Allen, J. A. (1993). A unified framework for planning and learning. In: S. Minton (ed.), Machine learning methods for planning, 317-350 Lavrac, N. and Wrobel, S. (eds.) (1995). Machine Learning: ECML-95. Springer Verlag Leake, D. (ed.) (1993). Case-Based Reasoning. Proc. AAAI-93 workshop on CBR, AAAI Press Leake, D. (1995). Case-Based Reasoning: Issues, methods, and Technology. Tutorial at the First International Conference on Case-Based Reasoning (ICCBR-95), Sesimbra, Portugal Lebowitz, M. (1990). The Utility of Similarity-Based Learning in a World Needing Explanation. In: R. S. Michalski and Y. Kodratoff (eds.), Machine Learning - An Artificial Intelligence Approach, Morgan Kaufmann Publishers, Inc., Ch. 5, 140-152 Ledke, D.B. (1994). Case-Based Reasoning. The Knowledge Engineering Review Vol. 9, No. 1, 61-64 Lee, M.H. (1993). The Knowledge-Based Factory. AI in Engineering Vol. 8, No. 2, 109-126 Lee, S.C., Ratliff, M., Peterson, C., and Lollar, L.F. (1991). AMPERES: A Real-Time Fault Monitoring and Diagnosis Knowledge-Bases System for Space Power Systems. Applied AI Vol. 5, No. 3, 281-308

311

Leng, B. and Buchanan, B.G. (1991). Constructive Induction on Symbolic Features. In: Collins, L.A.B.G.C. (eds.), ML '91 - Proceedings of the Eighth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 163-167 Leng, Y.W. and Pau, L.F. (1991). Temporal reasoning in blood gas diagnosis. Expert Systems Vol. 8, No. 3, 159-170 Lenz, M. (1993). CABATA - A hybrid CBR system. In: Richter, Wess et al. (1993), 204-209 Lenz, M. (1994). Case-Based Reasoning for Holiday Planning. In: W. Schertler, B. Schmid, A. M. Tjoa and H. Werthner (eds.), Information and Communication Technologies in Tourism, Springer Lenz, M. (1999). … In Proc. XPS-99, Springer Verlag Lenz, M., Auriol, E., Burkhard, H.-D., Manago, M. and Pirk, P. (1996). Fallbasierte Diagnostik und Entscheidungsunterstützung To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Lenz, M., Bartsch-Spörl, B., Burkhard, H.-D. and Wess, S. (eds.) (1998). Case-Based Reasoning Technology - From Foundations to Applications. Springer Verlag Lenz, M. and Burkhard, H.-D. (1995). Retrieval ohne Suche? In: Bartsch-Spörl, Janetzko & Wess (1995), 1-10 Lewis, L. (1993). A Case-Based Reasoning Approach to the Management of Faults in Communication Networks. In: IEEE 9th Conference on Artificial Intelligence for Applications, 114-120 Liu, W.Z. and White, A.P. (1994). The Importance of Attribute Selection Measures in Decision Tree Induction. Machine Learning Vol. 15, No. 1, 25-42 Long, W. J., Naimi, M. G. and Kurzrok, S. (1986). Reasoning about therapy from a physiological model. MEDINFO 86, Proceedings of the Fifth Conference on Medical Informatics, 757-760, Washington, October 1986 López, B. and Plaza, E. (1991). Case-based Learning of strategic Knowledge. In: Y. Kodratoff (ed.), Proc. EWSL-91, 398-411 Lutz, E. and Schlee, M. (1995). Fachexpertengerechte Wartung von wissensbasierten Systemen. KI 6/95, 51-57 Macchion, D. J. (1994). A Hybrid Knowledge-Based System for Technical Diagnosis Learning and Assistance. In: Wess, Althoff and Richter (1994), 301-312 Manago M. (1989). Knowledge Intensive Induction. Proc. of the sixth International Machine Learning workshop, Morgan Kaufmann Manago M. (1990). KATE: A Piece of Computer Aided Knowledge Engineering. Proc. of the 5th AAAI workshop on KA for knowledge-based systems, Gaines B. & Boose J. eds., Banff, AAAI Press Manago, M, Althoff, K.-D. & Wess, S. (1994). Comparison of Induction and Reasoning from Cases. In: Reasoning with Cases: Theory and Practice, Supporting Material for ECAI-94 Tutorial Manago, M. and Auriol, E. (1995). Integrating Induction and Case-Based Reasoning for Troubleshooting CFM-56 Aircraft Engines. In: Bartsch-Spörl, Janetzko & Wess (1995) Manago, M. and Blythe, J. (1989). Learning Disjunctive Concepts. In: Morik, K. (ed.), Knowledge Representation and Organization in Machine Learning, Springer-Verlag, 211-230 Manago, M., Althoff, K.-D., Auriol, E., Bergmann, R., Breen, S., Richter, M. M., Stehr, J., Traphöner, R. and Wilke, W. (1995). INRECA II: Information and Knowledge Reengineering for Reasoning from Cases. Project proposal, Esprit IV 312

Manago, M., Althoff, K.-D., Auriol, E., Traphöner, R., Wess, S., Conruyt, N., Maurer, F. (1993). Induction and Reasoning from Cases. In: Richter, Wess et al. (1993), 313-318 Manago, M., Bergmann, R., Conruyt, N., Traphöner, R. Pasley, J., Le Renard, J., Maurer, F., Wess, S., Althoff, K.-D. & Garry, S. (1994). CASUEL: A Common Case Representation Language. INRECA Deliverable D1, Version 2.01 Manago, M., Conruyt, N. and Le Renard, J. (1992). Acquiring Descriptive Knowledge for Classification and Identification. In: Wetter, Althoff et al. (1992), 392-405 Matheus, C.J. (1991). The Need for Constructive Induction. In: Collins, L.A.B.G.C. (eds.), ML '91 - Proceedings of the Eighth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 173-177 Maurer, F. (1995). Current Status of Case Retrieval in Engineering Domains: An Analysis from the Knowledge Engineering Perspective. Knowledge-Based Systems Journal Maurer, F. (ed.) (1996). 2nd Knowledge Engineering Forum. Technical Report 01/96, SFB 501: Entwicklung großer Systeme mit generischen Methoden, University of Kaiserslautern Maurer, F. and Ruppel, A. (1990). Integration von Neuronalen Netzen in die MOLTKE 3.0 Expertensystemtoolbox. Technical Report, University of Kaiserslautern Meissonnier, A. (1996). Ein fallbasiertes Informationssystem für fallbasierte Anwendungen und Systeme auf der Basis von Inreca. Diploma Thesis, University of Kaiserslautern (forthcoming) Michalski, R. and Tecuci, G. (eds.) (1994). Machine Learning: A Multi-Strategy Approach (Volume IV). San Francisco, CA: Morgan Kaufmann Mingers, J. (1989). An Empirical Comparison of Selection Measures for Decision-Tree Induction & An Empirical Comparison of Pruning Tree Methods for Decision-Tree Induction. Machine Learning 3 (319-342);4 (227-242) Mohr, O. (1991). Entwicklung und Einführung eines wissensbasierten Diagnosesystems für Werkzeugmaschinen. In: Warnecke, H.J. and Bullinger, H.J. (eds.), IAO-Forum Expertensysteme in Produktion und Engineering, Springer-Verlag, 219 Moinard, Y. (1994). Reasoning by Cases without Contraposition in Default Logic. In: Cohen, A.G. (eds.), ECAI '94 - 11th European Conference on Artificial Intelligence, John Wiley and Sons, 381-385 Moore, A. W. (1990). Acquisition of dynamic control knowledge for a robotic manipulator. In: Proc. of the Seventh International Conference on Machine Learning, 242-252. Austin, TX: Morgan Kaufman Morik, K. (1989). Sloppy Modeling. In: K. Morik (ed.), Knowledge Representation and Organization in Machine Learning, Springer, 107-134 Morik, K. (1996). Induktion für alle Fälle. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Morizet-Mahoudeaux, P. (1992). An Approach to Monitoring and Diagnosing Engineering Processes by a Knowledge-Based System. Applied AI Vol. 6, No. 4, 417-442 Muñoz-A., H. and Huellen, J. (1995). Retrieving cases in structured domains by using goal dependencies. In: Veloso & Aamodt (1995), 241-252 Muñoz-A., H., Paulokat, J. and Wess, S. (1994). Controlling a Nonlinear Hierarchical Planner using CBR. In: Keane, Haton & Manago (1994), 195-204 Nagao, M. (1991). Importance of Similarity Measure and Analogical Inference. Applied AI Vol. 5, No. 1, 11-14 Nagl, M. (1990a). Softwaretechnik: Methodisches Programmieren im Großen. Springer Verlag 313

Nagl, M. (1990b). Das Forschungsprojekt IPSEN. Informatik Forschung und Entwicklung 5, 103105 Nagl, M. (1993). Eng integrierte Software-Enwicklungsumgebungen. Informatik Forschung und Entwicklung , No. 8, 105-119 Nagl, M., Eversheim, W., Michaeli, W., Spaniol, O. and Weck, M. (1995). SUKITS – Software und Kommunikationsstrukturen in Technischen Systemen. In: Fachgruppe Informatik der RWTH Aachen (ed.), Jahresbericht 1994, Aachener Informatik-Berichte, 95-1, 143-145 Nakatani, Y. and Israel, D. (1993). Tuning Rules by Cases. In: Proceedings EWCBR93, 319 324 Nakhaeizadeh, G. (1994). Learning Prediction of Time Series. A Theoretical and Empirical Comparison of CBR with some other Approaches. In: Wess, Althoff and Richter (1994), 65-76 Nakhaeizadeh, G. (1996). CBR gleich KNN. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Newell, A. (1982). The knowledge level. Artificial Intelligence, Vol. 18, pp 87-127 Nilson, N. J. (1965). Learning machines - Foundations of Trainable Pattern-Classifying Systems. MacGraw-Hill Nökel, K. and Lamberti, H. (1991). Temporally Distributed Symptoms in a Diagnostic Application. AI in Engineering 6, No. 4, 196-204 Oliveira, A.L. and Sangiovanni-Vincentellli, A. (1992). Constructive Induction Using a NonGreedy Strategy for Feature Selection. In: Sleeman, D. and Edwards, P. (eds.), ML '92 Proceedings of the Ninth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 355-360 Owens, C. (1990). Indexing and retrieving abstract planning knowledge. Ph. D. thesis, Department of Computer Science, Yale University, New Haven Owens, C. (1993). Integrating Feature Extraction and Memory Search. Machine Learning Vol. 10, No. 3, 311-339 Oxman, R. and Voß, A. (1996). Fallgestütztes Entwerfen. To appear in: KI 1/96, Special Issue on Case-Based Reasoning (edited by B. Bartsch-Spörl and S. Wess) Padalkar, S., Kansai, G., Biegl, C., Sztipanovits, J., Okuda, K., and Miyasaka, N. (1991). RealTime Fault Diagnosis. IEEE Expert Vol. 6, No. 3, 75-85 Paulokat, J. (1995). Entscheidungsorientierte Rechtfertigungsverwaltung zur Unterstützung des Konfigurationsprozesses in IDAX. In: Richter & Maurer (1995), 19-36 Pews, G. (1994). MoCAS/2 - Fallbasierte Diagnose in technischen Domänen. Diploma Thesis, University of Kaiserslautern Pews, G. and Weiler, F. (1992). MoCAS - ein Ansatz zur falladaptiven Diagnostik. Project thesis, University of Kaiserslautern Pews, G. and Wess, S. (1993). Combining Case-Based and Model-Based Approaches for Diagnostic Applications in Technical Domains. In: Richter, Wess et al. (1993), 325-328 Pfau-Wagenbauer, M. and Nejdl, W. (1993a). Integrating Model-Based and Heuristic Features in a Real-Time Expert System. IEEE Expert Vol. 8, No. 4, 12-18 Pfau-Wagenbauer, M. and Nejdl, W. (1993b). Model/Heuristic-Based Alarm Processing for Power Systems. AI for Engineering Design, Analysis and Manufacturing Vol. 7, No. 1, 65-78 Pfeifer, T. and Richter, M. M. (eds.) (1993). Diagnose von technischen Systemen - Grundlagen, Methoden und Perspektiven der Fehlerdiagnose. Deutscher Universitäts-Verlag. 314

Pfeifer, T. and Faupel, B. (1993). Die Anwendung von MOLTKE: Diagnose von CNC-Bearbeitungszentren. In: Pfeifer and Richter (1993), 42-67 Pfeifer, T., Grob, R. and Klonaris, P. (1995). Teamgestützte Erfassung von Erfahrungswissen zur Fehleranalyse. In: Richter and Maurer (1995), 37-54 Philippow, I., Doberkat, E.-E., Breutmann, B. and Schnappert, A. (eds.) (1995). Proc. 3rd German Conference on Expert Systems (XPS-95), Workshop „Wissensbasierte Unterstützung des Softwareentwurfs“. Centre for Learning Systems and Applications, LSA-95-06, University of Kaiserslautern Pirk, P. (1992). Ähnlichkeit von Fällen bei technischen Diagnoseaufgaben. In: Ähnlichkeit von Fällen in Systemen des fallbasierten Schließens Plaza, E. and Arcos, J.L. (1993). A Reflective Architecture for Integrated Memory-based Learning and Reasoning. In: Richter, Wess et al. (1993), 329 - 334 Plaza, E. and Lopez de Mantaras, R. (1990). A case-based apprentice that learns from fuzzy examples. In: Z. Ras, M. Temankova and M. L. Emrich (eds.), Methodologies for Intelligent Systems 5, 420-427, North Holland Portinale, L. (1991). Generalization Handling in a Dynamic Case-Memory. In: , 71–81 Portinale, L., Torasso, P., Ortalda, C. and Giardino, A. (1994). Using Case-Based Reasoning to Focus Model-Based Diagnostic Problem Solving. In: Wess, Althoff and Richter (1994), 325-337 Pozzi, S. and Celentano, A. (1993). Knowledge-Based Document Filing. IEEE Expert Vol. 8, No. 5, 34-45 Praasch, R.K. and Agogino, A.M. (1993). A Structured and Behavioral Reasoning System for Diagnosing Large-Scale Systems. IEEE Expert Vol. 8, No. 4, 13-36 Preist, C., Allred, D., and Gupta, A. (1992). An Expert System to Perform Functional Diagnosis of a Bus Sybsystem. In: IEEE 8th Conference on Artificial Intelligence for Applications, 88-97 Priebisch, C. (1995). Integration von fallbasierter und kontextbasierter Diagnose am Beispiel der Systeme INRECA und S3. Diploma Thesis, University of Kaiserslautern Prieto-Diaz, R. (1994). Historical overview. In: Schäfer, W., Prieto-Diaz, R., and Matsumoto, M., Software reusability, chapter 1. Puppe, F. (1987). Diagnostisches Problemlösen mit Expertensystemen. Springer Verlag Puppe, F. (1990). Problemlösungsmethoden in Expertensystemen. Springer Verlag Puppe, F. (1994). Learning from cases for classification problem solving. In: Bock, Lenski & Richter (1994), 44-55 Puppe, F. and Goos, K. (1991). Improving Case-based Classification with Expert Knowledge. In: Christaller, T. (eds.), Proc. GWAI-91: 15. Fachtagung für Künstliche Intelligenz, Springer-Verlag, 196-205 Puppe, F. and Günter, A. (eds.) (1993). Expertensysteme 93. Proc. of the 2nd German Conference on Expert Systems. Springer Verlag Quinlan, J.R. (1990). Probabilistic Decision Trees. In: Michalski, R.S. and Kodratoff, Y. (eds.), Machine Learning - An Artificial Intelligence Approach, Morgan Kaufmann Publishers, Inc., Ch. 5, 140-152 Quinlan, R. (1986). Induction of Decision Trees. Machine Learning 1, 81-106 Quinlan, R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan Kaufmann Rahmel, J. (1995a). Diagnosis with Topological Maps. Centre for Learning Systems and Applications (LSA), LSA-95-09E, University of Kaiserslautern 315

Rahmel, J. (1995b). Similarity-based self-organized clustering. In: Proc. EPIA 95: Workshop Fuzzy Logic and Neural Networks in Engineering, Funchal Rahmel, J. and von Wangenheim, A. (1994). KoDiag: a Connectionist Expert System. International Symposium on Integrating Knowledge and Neural Heuristics, Pensacola Ram, A. (1993). Indexing, Elaboration and Refinement: Incremental Learning of Explanatory Cases. Machine Learning Vol. 10, No. 3, 201-248 Redmond, M. (1991). Improving Case Retrieval Through Observing Expert Problem Solving. In: , 516–521 Rehbold, R. (1991). Integration modellbasierten Wissens in technische Diagnostik-Expertensysteme. Doctoral Dissertation, University of Kaiserslautern Reinders, M., Vinkhuyzen, E., Voß, A., Akkermans, H., Balder, J., Bartsch-Spörl, B., Bredeweg, B., Drouven, U., van Harmelen, F., Karbach, W., Karssen, Z., Schreiber, G., and Wielinga, B. (1991). A Conceptual Modelling Framework for Knowledge-Level Reflection. AI Communications Vol. 4, No. 2/3, 74-87 Reiter, R. (1987). A theory of diagnosis from first principles. Artificial Intelligence 32, 57-95 Rhodes, P.C. and Karakoulas, G.J. (1991). A Probabilistic Model-Based Method for Diagnosis. AI in Engineering Vol. 6, No. 2, 86-99 Richter, M. M. (1989). Prinzipien der Künstlichen Intelligenz. B.G. Teubner, Stuttgart Richter, M. M. (1992). Classification and Learning of Similarity Measures. Proc. 16th Annual Conference of the German Society for Classification, Springer Verlag Richter, M. M. (1994). On the notion of similarity. In: Globig and Althoff (1994), 1-13 Richter, M. M. (1995). Some elementary remarks on case-based reasoning. Unpublished manuscript Richter, M. M. (1996). The flexibility of case-based reasoning technology and its relationship to other information technologies. Personal communication Richter, M. M. and Maurer, F. (eds.) (1995). Expertensysteme 95. Proc. of the 3rd German Conference on Expert Systems. infix Verlag Richter, M. M. and Wendel, O. (1993). Lernende Systeme, Part 1. Lecture notes, Dept. of Computer Science, University of Kaiserslautern Richter, M. M. and Wess, S. (1991). Similarity, Uncertainty and Case-Based Reasoning in Patdex. In: R. S. Boyer (ed.), Automated Reasoning - Essays in Honor of Woody Bledsoe.Kluwer Academic Publishers, 249-265 Richter, M. M., Wess, S., Althoff, K.-D. and Maurer, F. (eds.) (1993). Proc. First European Workshop on Case-Based Reasoning (EWCBR-93). SEKI-REPORT SR-93-12, University of Kaiserslautern Riddle, P., Richard Segal, , and Etzioni, O. (1994). Representation Design and Brute-Force Induction in a Boeing Manufacturing Domain. Applied AI Vol. 8, No. 1, 125-148 Riesbeck, C. K. and Schank, R. C. (1989). Inside Case-Based Reasoning. Hillsdale, NJ: Lawrence Erlbaum Associates Rissland, E. L. and Daniels, J. J. (1995). Using CBR to Drive IR. In: C. S. Mellish (ed.), Proc. IJCAI-95, 400-408 Rissland, E. L. and Skalak, D. B. (1991). CABARET: Rule interpretation in a hybrid architecture. International Journal on Man-Machine Studies 34(1), 839-887 Rissland, E. L., Skalak, D. B. and Friedman, M. T. (1993). Case Retrieval through Multiple Indexing and Heuristic Search. In: R. Bajcsy (ed.), Proc. of the 13th IJCAI, 902-908

316

Robey, B.L., Fink, P.K., Venkatesan, S., Redfield, C.L., and Ferguson, J.W. (1994). DRAIR ADVISER: A Knowledge-Based System for Material Deficiency Analysis. AI magazine Vol. 15, No. 2, 67-82 Rosch, E. and Mervis, C. B. (1975). Family Resemblances: Studies in the internal structure of Categories. Cognitive Psychology 7, 573-605 Rosch, E. and Mervis, C. B. (1975). Family Resemblances: Studies in the internal structure of categories. Cognitive Psychology 7, 573-605 Rothenberg, J. (1989). Expert system tool evaluation. In: Guida & Tasso (1989), 205-229 Rothenberg, J., Paul, J., Kameny, I., Kipps, J. R. and Swenson, M. (1987). Evaluating Expert System Tools: A Framework and Methodology. Technical report, R-3542, DARPA, Rand Corporation Rumelhart, D. E. (ed.) (1988). Parallel Distributed Processing. The MIT-Press Ruppel, A. (1990). Assoziatives Problemlösen mit Hilfe fallbasierter und konnektionistischer Methoden am Beispiel technischer Diagnose. Diploma thesis, University of Kaiserslauern Saitta, L. (ed.) (1995). State of the Art in Machine Learning. MLnet News, Special Issue, September 1995 Sakakibara, Y. (1993). Noise-tolerant Occam Algorithms and their Application to Learning Decision Trees. Machine Learning Vol. 11, No. 1, 37-62 Salzberg, S. (1991). A Nearest Hyperrectangle Learning Method. Machine Learning 6, 277-309 Sano, C. (1992). Hybrid of (ID3 extension + Backpropagation) hybrid and (Case-Based Reasoner + Grossberg Net) hybrid with economics modeling by Genetic Algorithm. In: Biswas, G. (eds.), Applications of Artificial Intelligence X: nowledge-Based Systems, SPIE - The Int. Society for Optical Engineering, SPIE, 180-194 Santamaría, J. C. and Ram, A. (1994). Systematic Evaluation of Design Decisions in CBR Systems. In: Aha (1994) Saxena, S. (1993). Fault Isolation during Semiconductor Manufacturing using Automated Discovery from Wafer Tracking Databases. In: IEEE 9th Conference on Artificial Intelligence for Application, 313-320 Schank, R. (1982). Dynamic memory, a theory of reminding and learning in computers and people. Cambridge University Press Scheer, A.-W. (1992). Architektur integrierter Informationssysteme. Springer Verlag Scheer, A.-W. (1995). Wirtschaftsinformatik – Referenzmodelle für industrielle Geschäftsprozesse. Springer Verlag Schründer, C.P., Galletly, J.E., and Bicheno, J.R. (1994). A fuzzy, knowledge-based decision support tool for the production operations management. Expert Systems Vol. 11, No. 1, 312 Schuch, A. (1992). iMAKE: Inkrementelle Modellierung und Simulation technischer Geräte zur Generierung einer Wissensbasis für MOLTKE 3.0. Diploma thesis, University of Kaiserslautern Schult, T. J. (1992). Fallbasierte Expertensystemshells - Methoden und Werkzeuge. Research Reports, Institute of Psychology, University of Freiburg, No. 89 Schultz, S., Henning, R., Jäckel, O., and Härtig, G. (1993). DIAG_FSB und DIAG_MOD - wissensbasierte Systeme für die Fehlersuche an elektronischen Baugruppen. In: Qualitätssicherung mit wissenbasierten Systemen (Tagung Köln März '93), VDIGesellschaft für Mess- und Automatisierungstechnik, VDI Verlag, 57-66 317

Schweiger, U. (1991). Eine erklärungsorientierte Methode zur Integration von fall- und modellbasierten Schließens. Diploma Thesis, Fakultät für Informatik, Universität Karlsruhe Scott, P.D. and Sage, K.H. (1992). Why Generalize? Hybrid Representation and Instance-Based Learning. In: Neumann, B. (eds.), ECAI '92 - European Conference on Artificial Intelligence, ECCAI, John Wiley and Sons, 484-486 Sebag, M. & Schoenauer, M. (1994). A Rule-Based Similarity Measure. In: Wess, Althoff & Richter (1994), 119-131 Seidelmann, G. (1993). Using Heuristics to Speed up Induction on Continuous-Valued Attributes. In: Brazdil, P.B. (eds.), Machine Learning: ECML-93, Springer-Verlag, 280296 Seifert, C.M., Hammond, K.J., Johnson, H.M., Converse, T.M., McDougal, T.F., and Vanderstoep, S.W. (). . Machine Learning Sembugamoorthy, V. and Chandrasekaran, B. (1986). Functional representation of devices and compilation of diagnostic problem solving systems. In: Kolodner and Riesbeck (1986), 47-73 Senjen, R., de Beler, M., Leckie, C., and Rowles, C. (1993). Hybrid Expert Systems for Monitoring and Fault Diagnosis. In: IEEE 9th Conference on Artificial Intelligence for Application, 235-241 SFB-501 (1994). Entwicklung großer Systeme mit generischen Methoden. Project proposal 19951997, University of Kaiserslautern Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press Shafer, G. and Logan, R. (1987). Implementing Dempster's rule for hierarchical evidence. Artificial Intelligence Vol. 33 no. 3, 271-298 Shama, R.S. and Conrath, D.W. (1993). Evaluating Expert Systems: A Review of Applicable Approaches. AI Review Vol. 7, No. 2, 77-92 Shannon and Weaver (1947). The Mathematical Theory of Computation. University of Illinois Press Sheridan, F. (1991). A survey of techniques for inference under uncertainty. AI Review Vol. 5, No. 1/2, 89-121 Shimazu, H., Kitano, H. and Shibata, A. (1993). Retrieving Cases from Relational Data-Bases: Another Stride Towards Corporate-Wide Case base Systems. In: R. Bajcsy (ed.), Proc. of the 13th IJCAI, 909-914 Shutt, T.S. and Simoudis, E. (1993). COPRA: Computer Operations Problem Resolution Assistant. In: IEEE 9th Conference on Artificial Intelligence for Applications, 107-113 Simoudis, E. (1992). Using Case-Based Retrieval for Customer Technical Support. IEEE Expert Vol. 7, No. 5, 7-13 Simoudis, E. and Miller, J.S. (1991). The Application of CBR to Help Desk Applications. In: Proc. CBR91, 25–36 Slagle, J. R. and Wick, M. R. (1988). A Method for Evaluating Candidate Expert System Applications. AI Magazine 9, No. 4, 44-53 Sleeman, D. and Edwards, P. (eds.) (1992). Machine Learning – Proc. of the Ninth International Workshop. Morgan Kaufmann Smyth, B. and Keane, M. (1995a). Remembering to Forget: A Competence-Preserving Case Deletion Policy for Case-Based Reasoning Systems. In: C. S. Mellish (ed.), Proc. IJCAI95, 377-383 Smyth, B. and Keane, M. (1995b). Experiments on adaptation-guided retrieval in case-based design. In: Veloso and Aamodt (1995), 313-324 318

Smyth, P. and Goodman, R.M. (1992). An information theoretic Approach to rule induction from databases. IEEE Transactions on Knowledge and Data Engineering Vol. 4, No. 4, 301-316 Smyth, P. and Mellstrom, J. (1992). Detecting Novel Classes with Application to Fault Diagnosis. In: Sleeman, D. and Edwards, P. (eds.), ML '92 - Proceedings of the Ninth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 416-425 Sokal, R. R. and Rahlf, F. J. (1981). Biometry. W. H. Freeman and Co., San Francisco Spanoudakis, G. and Constantopoulos, P. (1994). Similarity for Analogical Software Reuse: A Computational Model. In: Cohen, A.G. (eds.), ECAI '94 - 11th European Conference on Artificial Intelligence, John Wiley and Sons Stadler, M. and Wess, S. (1989). PATDEX: Konzept und Implementierung eines fallbasierten, analogieorientierten Inferenzmechanismus zur Diagnose eines CNC-Bearbeitungszentrums. Project thesis, University of Kaiserslautern Steels, L. (1990). Components of expertise. AI Magazine 11, 2, 29-49 Steels, L. (1993a). The componential framework and its role in reusability. In: David, Krivine & Simmons (1993), 273-298 Steels, L. (1993b). Tools for Corporate Knowledge Management. In: European Knowledge Acquisition Workshop, 227-242 Steels, L. and Lepape, B. (eds.) (1992). Enhancing the Knowledge Engineering Process. North Holland Strube, G. & Janetzko, D. (1990): Episodisches Wissen und fallbasiertes Schließen: Aufgaben für die Wissensdiagnostik und die Wissenspsychologie. Schweizerische Zeitschrift für Psychologie, 49 (4), 211-221 Strube, G. (1989). Episodisches Wissen. Arbeitspapiere der GMD, 385, 10-26 Szepesvári, C., Balázs, L. and Lörincz, A. (1993). Topology learning solved by extended objects: a neural network model. Technical report, Department of Photophysics, Institute of Isotopes of the Hungarian Academy of Sciences, Budapest, P.O.Box 77, Hungary, H1525 Tanaka, T. and Sueda, N. (1993). Combining Strict Matching and Similarity Assessment for Retrieving Appropriate Cases Efficiently. In: Anick, Simoudis et al. (1993) ten Teije, A. and van Harmelen, F. (1995). Using reflection techniques for flexible problem solving with examples from diagnosis. Proc. IJCAI-95 workshop on Meta-Level Reasoning and Reflection, Montreal Ting, K. M. (1994). The problem of small disjuncts: Its remedy in decision trees. Proc. of the Tenth Canadian Conference on Artificial Intelligence, 91-97 TopBusiness (1995). Meister der Diagnose. Top-Business – Industriemagazin für innovatives Management 2, 82-86 Torasso, P., Portinale, L., Console, L., and Casassa Mont, M. (1992). Approximate reasoning in a system combining protoypical knowledge with case-based reasoning. In: L. A. Zadeh and J. Kacprzyk (eds.), Fuzzy Logic for the Management of Uncertainty, John Wiley & Sons Traphöner, R. (1993). Das Service-Support-System S3. In: Qualitätssicherung mit wissensbasierten Systemen, VDI-Gesellschaft Meß- und Automatisierungstechnik, VDI-Verlag, 7787

319

Traphöner, R. and Althoff, K.-D. (1993). Recent developments in case-based reasoning and inductive learning and their applicability in automotive industries. In: Proc. 26th International Symposium on Automotive Technology and Automation, Dedicated Conf. on Mechatronics, Aachen, Germany Traphöner, R., Bräuer, M., Dittrich, S., Althoff, K.-D. and Maurer, F. (1995). Documentation of the final integrated Inreca System. Esprit Project INRECA (P6322), Deliverable D16 Traphöner, R., Manago, M., Conruyt, N. and Dittrich, S. (1992). Industrial Criteria for Comparing the Technologies of INRECA. INRECA (Esprit P6322), Deliverable D4 Tsujino, K., Dabija, V.G., and Nishida, S. (1992). Knowledge Acquisition Driven by Constructive and Interactive Induction. In: Current Developments in Knowledge Acquisition - EKAW '92, Springer-Verlag, 153-170 Tversky, A. (1977). Features of Similarity. Psychological Review 84, 327-352 Tzafestas, T., Palios, L., and Cholin, F. (1994). Diagnostic expert system inference engine based on the certainty factors model. Knowledge-Based Systems Vol. 7, No. 1, 17-26 Utgoff, P. (1988). ID5: An incremental ID3. Fifth International Conference on Machine Learning, Morgan Kaufmann, Los Altos Uthurusamy, R., Means, L.G., Godden, K.S., and Lytinen, S.L. (1993). Extracting Knowledge from Diagnostic Databases. IEEE Expert Vol. 8, No. 6, 27-39 Van de Merckt, T. (1992). NFDT: A System that Learns Flexible Concepts based on Decision Trees for Numerical Attributes. In: Sleeman, D. and Edwards, P. (eds.), ML '92 Proceedings of the Ninth International Workshop on Machine Leaning, Morgan Kaufmann Publishers, 322-331 Van de Merckt, T. (1993). Decision Trees in Numerical Attribute Spaces. In: R. Bajcsy (ed.), Proc. of the 13th IJCAI, 1016-1021 van Someren, M., Zheng, L. L. and Post, W. (1990). Cases, models, or compiled knowledge? A cooperative analysis and proposed integration. In: B. Wielinga, J. Boose et al. (eds.), Current Trends in Knowledge Acquisition, IOS Press, 339-355 Vargas, J.E. and Bourne, J.R. (1993). Detecting Device Failures through Scale-Driven Object Matching. Applied AI Vol. 7, No. 2, 165-176 Vargas, J.E. and Raj, S. (1993). Developing maintainable expert systems using case-based reasoning. Expert Systems Vol. 10, No. 4, 219-266 Veloso, M. M. (1992). Learning by Analogy and Reasoning in General Problem Solving. Ph.D. thesis, Carnegie-Mellon-University Veloso, M. M. and Aamodt, A. (eds.) (1995). Case-Based Reasoning Research and Development. Springer Verlag Veloso, M. M. and Carbonell, J. G. (1989). Learning Analogies by Analogy - The Closed Loop of Memory Organisation and Problem Solving. In: Hammond (1989b), 153-158 Venturini, G. (1993). Sia: A Supervised Inductive Algorithm with Genetic search for Learning Attributes based Concepts. In: Brazdil (1993), 280-296 Verweyen-Frank, H. (1990). Wissensakquisition für Fehlerdiagnosesysteme - entscheidend ist der Knowledge Engineer. KI, 2, 53-55 Vietze, T. and Günter, A. (1994). A Generalization Based Approach for Case-Based Configuration in Technical Domains. In: World Congress on Expert Systems Vinson, J.M., Grantham, S.D., and Ungar, L.H. (1992). Automatic Rebuilding of Qualitative Models for Diagnosis. IEEE Expert Vol. 7, No. 4, 23-30 von Puttkamer, E. (1992). Prozeßrechentechnik. Lecture notes, Department of Computer Science, University of Kaiserslautern 320

von Puttkamer, E. (1993). Autonome Mobile Roboter. Lecture notes, Department of Computer Science, University of Kaiserslautern Voß, A. (ed.) (1994). FABEL: Similarity concepts and retrieval methods. GMD, FABEL Report No. 13, Gesellschaft für Mathematik und Datenverarbeitung mbH, Sankt Augustin Voß, A., Bartsch-Spörl, B. and Oxman, R. (1996). A Study of Case Adaptation Systems. To appear in AI in Design 96 Voß, A., Bartsch-Spörl, B., Hovestadt, L., Jantke, K. P., Petersohn, U. & Strube, G. (1996). FABEL. To appear in KI Waleschkowski, N., Schahn, M. and Forchert, T. (1995). Wissensmodellierung und Wissenserwerb am Beispiel der Fahrzeugdiagnose. KI 1/95, 55-61 Wallace, C.S. and Patrick, J.D. (1993). Coding Decision Trees. Machine Learning Vol. 11, No. 1, 7-22 Weber, G. (1993). Fallbasiertes Lernen und Analogien. Habilitationsschrift, Universität Trier; also: Weinheim: Beltz, PsychologieVerlagsUnion 1994 Wei, H. and Tsatsoulis, C. (1993). Uncertain Case-Based Reasoning. In: Proc. AAAI-93 Casebased Reasoning Workshop, 169 Weis, K.-H. jun. (1994). Aufbau topologischer Weltmodelle mit neuronalen Netzen. In: Proc. Seminar „artificial life & real robots“, University of Kaiserslautern Weis, K.-H. jun. (1995a). Ein integrierter Ansatz zum fallbasierten Schließen. Project thesis, University of Kaiserslautern Weis, K.-H. jun. (1995b). Bewertung integrierter fallbasierter Systeme. Diploma Thesis, University of Kaiserslautern Weiß, S. (1993). Modellbasiertes Lernen zur Wissensakquisition für technische Diagnosesysteme. Doctoral Dissertation, Carl-Hanser Verlag Wess, S. (1990). PATDEX/2: ein System zum adaptiven, fallfokussierenden Lernen in technischen Diagnosesituationen. Diploma thesis, University of Kaiserslautern Wess, S. (1993). PATDEX - ein Ansatz zur wissensbasierten und inkrementellen Verbesserung von Ähnlichkeitsbewertungen in der fallbasierten Diagnostik. In: Puppe and Günter (1993), 42-55 Wess, S. (1995). Fallbasiertes Schließen in wissensbasierten Systemen zur Entscheidungsunterstützung und Diagnostik. Doctoral Dissertation, University of Kaiserslautern Wess, S. (1996). Intelligente Systeme für den Customer Support: Fallbasiertes Schließen in Help-Desk und Call-Center Anwendungen. Wirtschaftsinformatik 1/96, special issue on case-based decision support (edited by D. Ehrenberg) Wess, S. and Globig, C. (1994a). A Case Study on Case-Based and Symbolic Learning. In: Aha (1994), 43-47 Wess, S. and Globig, C. (1994b). Case-Based and Symbolic Classification Algorithms - A Case Study Using Version Space In: Wess, Althoff & Richter (1994), 77-91 Wess, S., Althoff, K.-D. and Derwand, G. (1993). Improving the Retrieval Step in Case-Based Reasoning. In: Richter, Wess et al. (1993), 83-88 Wess, S., Althoff, K.-D. and Derwand, G. (1994). Using k-d Trees to Improve the Retrieval Step in Case-Based Reasoning. In: Wess, Althoff & Richter (1994), 167-181 Wess, S., Althoff, K.-D. and Richter, M. M. (eds.) (1994). Topics in Case-Based Reasoning. Springer Verlag Wess, S., Janetzko, D., and Melis, E. (1992). Goal-Driven Similarity Assessment. Technical Report., University of Kaiserslautern (Seki Report) 321

Wetter, Th., Althoff, K.-D., Boose, J., Gaines, B., Linster, M. & Schmalhofer, F. (eds.) (1992). Current Developments in Knowledge Acquisition - EKAW ´92. Springer Verlag Wettschereck, D. (1994a). A Hybrid Nearest-Neighbor and Nearest-Hyperrectangle Algorithm. In: F. Bergadano & L. De Raedt (eds.), Proc. ECML-94, Springer Verlag, 323-335 Wettschereck, D. (1994b). A Study of Distance-Based Machine Learning Algorithms. Ph.D. thesis, Oregon State University Wielinga, B. J., Schreiber, A. T. and Breuker, J. A. (1992). KADS: A modelling approach to knowledge engineering. Knowledge Acquisition 4(1), 5-54 Wielinga, B. J., Van de Velde, W., Schreiber, G. & Akkermans, H. (1993). Towards a unification of knowledge modeling approaches. In: David, Krivine & Simmons (1993), 299335 Woltering, A. (1993). Regelbasierte und fallbasierte Interpretation von Lungenfunktionsdaten erste Ergebnisse eines empirischen Vergleichs. Bericht RAP/6/1993, Technische Universität Berlin Woltering, A. and Wess, S. (1996). Entscheidungsunterstützungssysteme und fallbasiertes Schließen – Eine empirische Untersuchung im deutschsprachigen Raum. Wirtschaftsinformatik 1/96, special issue on case-based decision support (edited by D. Ehrenberg) Wrobel, S. (1994). Concept Formation and Knowledge Revision. Kluwer Academic Publishers Yasdi, R. (1991). Learning classification rules from database in the context of knowledge acquisition and representation. IEEE Transactions on Knowledge and Data Engineering Vol. 3, No. 3, 293-306 Yu, X. and Biswas, G. (1992). A Multi-level Diagnosis Methodology for Complex Systems. In: IEEE 8th conference on Artificial Intelligence for Applications, 81-87 Zadeh, L. A. (1972). A Rationale for Fuzzy Control. Journal of Dynamic Systems, Measurement and Control, Series 6, 94:3-4 Zarri, G.P. (1993). Using a High-Level, Conceptual Knowledge Representation Language for Visualizing Efficiently the Internal Structure of Complex „Cases“. In: Richter, Wess et al. (1993), 89-94 Zeleznikow, J., Hunter, D., and Vossos, G. (1993). Integrating Rule-Based and Case-Based Reasoning with Information Retrieval: The IKBALS Project. In: Richter, Wess et al. (1993), 341-346 Zhang, J. (1990). A method that combines inductive learning with exemplar-based learning. Proc. for Tools for Artificial Intelligence, 31-37. Herndon, VA: IEEE Computer Society Press Zimmermann, H.-J. (1993). Fuzzy Technologien – Prinzipien, Werkzeuge, Potentiale. VDI Verlag Zink, K.-J. (1993). Literaturzusammenstellung Personalführung. Technical report, University of Kaiserslautern.

322

INDEX AMR (architecture for autonomous mobile robots) .............................................................246, 254 Applications ............................................ 22, 23, 92-109, 228-233, 242-247, 253-256, 270, 23, 22 AWACS .................................................................................................................................. 42, 160 BOLERO .......................................................................................................................127, 128, 284 CAPLAN/CBC (computer assisted planning/case-based control).........................126, 171, 174, 176 CARET (case retrieval tool) .........................................................................................................232 CASEY ... 42-43, 45-48, 110, 127-129, 135, 143, 150, 152, 155, 159, 170-171, 173-174, 176, 185, 187-188, 190, 207, 284 CASUEL (INRECA common case representation language) ...........ii, 32, 50, 60, 104, 108, 233, 292 Category model .............................................. 112-114, 122, 128-129, 157, 190, 204-206, 234-235 CBR 2 (see also CBR EXPRESS) ................................................................................................iv, 91 CBR cycle ........................................................................................ 2, 4, 66, 146-147, 208, 279-280 CBR EXPRESS ...................................................................................................... i, iv, 42, 52, 60-91 CBR-WORKS ......................................................................................................................ii, 91, 296 CCC+ ... 42, 45-48, 113, 115, 128-129, 134, 139, 142-143, 150, 152, 155, 157-159, 165-166, 170, 176, 185, 193-195, 207, 284 CHEF ....................................................................................................................................172, 176 Classification............................................................................................................... 286, 285-289 Clustering ................................................................................................................ 8, 115, 233, 234 Context graph ...................................................................................................................... 116-117 CRASH .........................................................................................................................................258 CREEK .......................................ii, 42-43, 45-48, 114, 122, 136, 140, 168, 187, 194, 204, 110-208 CYRUS .........................................................................................................................................110 D3 ..........................................................................................................................................42, 291 DAIDA (development assistance for integrated database applications) ......................................268 DANIEL (distributed architecture for the knowledge-based integration of expert systems in law)..........................................................................................................................................230 323

Databases.............................................................................................................4, 7, 231, 232, 268 Decision support............................................................................... 7, 264-267, 268, 288, 285-290 DEDAL .................................................................................................................................241, 252 Diagnosis........................................................... 10-35, 124, 151-152, 242-247, 253, 287, 285-290 DIAMON (diagnosis and monitoring) ..........................................................................................251 DISARM (diagnosis system wit automated rebuilding of models)...............................................250 Dynamic diagnosis ......................................................................................................................257 EDIS (engine data interpretation system) ....................................................................................269 ESTEEM ...................................................................................................................... i, v, 42, 60-91 Evaluation......................................................................................................37-51, 52-56, 274-290 Experience graph................................................................................................................. 197-200 Explanation-based learning........................................................................... 16, 140, 233-234, 260 FABEL ..........................................................................................................................................150 GenRule.................................................................................................................... 17-18, 201-202 Help desk.....................................................................................................................................270 IAP (intelligent alarm processor) .........................................................................................250, 251 ID3 (see Induction, TDIDT) .................................................................................................238, 240 iMAKE ..................................................................................................................................... 16-17 Indexing...................................................................... 62-63, 110-129, 148-170, 186-207, 209-225 Induction........................................................................................................4-6, 209-225, 235-241 INFERRULE ..................................................................................................................................237 Information retrieval ....................................................................................... 7, 148-170, 241, 252 INRECA .......i, ii, vi, 40-43, 60-91, 111, 132, 141, 163, 172, 175, 179, 110-208, 209-225, 291-296 Inreca-tree ........................................................................................................................... 209-225 Integration ...................................................................... 12-21, 25-31, 193-206, 209-225, 268-271 IPSEN (integrated software project support environment)...........................................................268 KAISER ........................................................................................................................................238 KALES ..................................................................................................................................159, 191 KATE tools........................................................................................................... i, ii, iv, 60-91, 296 KBSED (knowledge-based system for engine diagnosis) ............................................................261 324

k-d tree................................................................... 111, 132, 163-164, 166-167, 181-183, 209-225 Knowledge acquisition..................................................................3, 12, 92-109, 144-147, 276-285 Knowledge representation.........................10-35, 59-67, 92-109, 110-143, 208, 209-225, 285-289 KoDIAG ................................................................................................................................... 31-36 Learning ................................................................... 4-6, 35, 176-177, 186-207, 209-225, 235-241 Legal reasoning ................................................................................................................... 229-230 MAC/FAC model............................................................................................................................23 MDA (manoeuvre decision aid) ...................................................................................................254 Meta-level reasoning...........................................................................................................261, 262 MIMIC ..........................................................................................................................................251 MoCAS .............................................................................................. 26-31, 42, 118, 159, 180, 203 Model-based diagnosis.......................................16, 26-31, 118-121, 135, 248-252, 256, 265, 269 MOLTKE shell ..........................................................................................................................14, 21 MOLTKE workbench..................12, 13, 42-43, 45-48, 114, 116, 124, 140, 191, 201, 203, 110-208 Nearest neighbour search ........................................................................................ 8, 130-142, 194 PARIS ...........................................................................................................................................126 PATDEX ..................................................................................... II, 18, 21, 45-48, 134, 172, 110-208 PATDEX/1 ......................................................................................... 42-43, 140, 197, 200, 110-208 PATDEX/2 ........................................................................... 42-43, 45, 113, 140, 176, 200, 110-208 PRODIGY/ANALOGY ...........................................................................................................126, 174 PROTOS ..........................................................................42-43, 45-48, 122, 136, 168, 204, 110-208 QPE 249 QSIM ....................................................................................................................................249, 251 Qualitative model (see Symbolic model) ....................................................................................257 Real time .............................................................................................................. 253-256, 266-267 REMIND ............................................................................................................. i, iv, 42, 60-91, 120 Repair ..................................................................................................................................178, 252 Retrieval ......................................................................................7, 28, 110-143, 148-170, 209-225 Rule-based reasoning .......................................................................... 14, 21, 23, 25, 257-263, 264 S3, S3+...................................................................................................................................23, 116 325

S3+/INRECA .........................................................................................................v, 25, 42, 166, 181 S3-CASE ..................................................................................................................... i, iv, 42, 60-91 SCADA (supervisory control and data acquisition)......................................................................251 Semantic networks ...................................................................................... 122-123, 168, 204, 229 Service support........................................................................................................................ 23-24 Similarity.......................................4-6, 8, 27, 32-35, 64, 92-109, 130-142, 148-170, 179, 186-207 SUPER ..........................................................................................................................................253 Sustained learning ............................................................................................... 3, 4, 204, 276-285 Symbolic model....16, 26-31, 118-122, 135, 159-162, 173, 180, 191-192, 203-204, 248-252, 256, 257 Task method decomposition model.......................................................144-148, 144-208, 280-281 TDIDT (see Induction, ID3)...................................................................................209-225, 236, 241 Technical diagnosis ...................................................................................10-35, 242-247, 248-252 Test domains ........................................................................................................................... 57-59 WDS (wissensbasiertes Diagnosesystem)....................................................................................244 WheelNet.............................................................................................................................133, 195

326