Extending Ripple-Down Rules Paul Compton School of Computer Science and Engineering, University of New South Wales Sydney 2052, Australia. email:
[email protected] Debbie Richards Department of Computing, Div of Information and Communication Sciences Macquarie University Sydney, Australia email:
[email protected]
Abstract Ripple-Down Rules (RDR) has had considerable success in providing simple incremental knowledge acquisition in classification domains. It has been extended to multiple classification, configuration, search and more recently to resource allocation tasks. Based on the experience of applying RDR to a resource allocation task, this paper proposes a generalisation of RDR to enable it to apply to a wide range of knowledge-based system tasks.
Introduction RDR was developed to deal with the problems found in four years maintenance of the medical expert system GARVAN-ES1(Compton, et al. 1989). One of the key observations in this was that experts never gave a comprehensive explanation for their decision making. Rather they justified that the conclusion was correct and the justification was created for and shaped by the context in which it was given (Compton & Jansen 1990). The starting aim of RDR then was to provide a knowledge acquisition and maintenance strategy that properly dealt with the contextual nature of knowledge. The most common form of maintenance for a KBS is correcting errors of interpretation for particular cases. In such maintenance the justification for a new conclusion will be presented at least in terms of the features of the case that suggest a different interpretation of the data from the one given. That is, the context in which this justification is given is that a particular incorrect conclusion has been made for a specific case. In RDR, a rule consists of the conjunction of features in the case identified by the expert and the rule is added to the KBS so that it will only be used when the same wrong conclusion is reached through the same pathway of rules; i.e. in the same context in which the error was made. RDR provides a further validation step in that the case is stored with the rule, and if a further correction rule is added, the expert must select some features that distinguish the new case from the stored case(s) for which the given interpretation was correct. In RDR, this maintenance strategy is used to build the entire knowledge base.
The critical features of RDR are that: •
the system organises and controls the addition of any new knowledge to the KBS.
•
it validates any knowledge acquisition to ensure that the new knowledge provides an incremental addition to the system knowledge and does not degrade previous knowledge.
•
to add new knowledge, the expert only has to identify features in a case that distinguish it from other cases retrieved by the system to which other conclusions apply.
Taken together these features provide the central difference of RDR from other KA approaches. A guiding principle of human intellectual endeavour is seeking to make things understandable; to make clear what is going on so we understand how and why things happened and what might happen in the future. Reasonably, KA tools, techniques and most modern KA research are motivated by the same idea that since a KA tool is intended to acquire knowledge, acquiring the knowledge should be a process of organising, and making clear how an area of expertise operates. In fact modern KA is moving away from the goal of producing an artefact able to make expert-like decision, to helping people and organisations structure and make clear the relevant issues, information and data in a domain (Akkermans, et al. 1999). The result of all this is that KA tools and techniques are designed to assist the expert and knowledge engineer in structuring and making clear the domain. They are intimately involved in deciding how to organise and structure the knowledge and problem solving. Much of the research is aimed at sets of templates to assist in this. RDR has the completely different goal of hiding the organisation of the knowledge from the expert (and knowledge engineer). RDR distinguishes between an expert as a rational practitioner who is able to justify his or her practical decisions, and an a expert as an authority or consultant in a domain able to expound the principles and theory. We see this scientific activity as very important and something for which RDR KBs may be a valuable resource (Lee & Compton 1996; Richards & Compton 1997), but we prefer to deal with the expert as rational practitioner in building a KB. Hence, the RDR principles above, which taken together require an expert to be a rational practitioner, rather than a scientific theorist. RDR systems have been implemented for a range of application areas and tasks. The first industrial demonstration of this approach was the PEIRS system, which provided clinical interpretations for reports of pathology testing (Edwards, et al. 1993). The approach has also been adapted to a of tasks: multiple classification (Kang, et al. 1995), control (Shiraz & Sammut 1997), heuristic search (Beydoun & Hoffman 1997; Beydoun & Hoffman 1998), document management using multiple classification (Kang, et al. 1997) and configuration (Compton, et al. 1998). There is also resarch on the re-expression and reuse of this knowledge (Richards & Compton 1997). There are a other lines of RDR research integrating RDR with fuzzy reasoning (Martinez-Bejar, et al. 1998) and ontological development (Martinez-Bejar, et al. 1998). The level of evaluation in these studies varies, but they have invariably shown very simple and highly efficient knowledge acquisition. RDR systems for the task of providing interpretative comments for medical Chemical Pathology reports are now available commercially. Results from this experience have not yet been published, but confirm that very large knowledge bases (> 7000 rules) can be built and maintained very easily by pathologists with little computing experience or knowledge (Pacific Knowledge Systems, personal communication).
The other critical feature of the RDR evaluations was that despite the incremental addition of knowledge, knowledge acquisition was very efficient. It might be expected that incremental addition of knowledge where knowledge was only added (as a refinement), but never changed, may result in very large knowledge bases with much repeated knowledge. However, simulation studies have shown that the size of the knowledge bases is comparable to those produced by machine learning (Compton, et al. 1995; Kang, et al. 1998) and there is significant increase in size only a when random choice is used by the expert. (We trust that human experts use other criteria for their choices!). In studies on a human developed MCRDR knowledge base (~3000) rules, only 10% compression could be achieved (Suryanto, et al. 1999). The success of RDR in these studies raises the question of whether RDR can be extended to other types of problems and can be developed into a general problem solving method. We have recently attempted to extend RDR to resource allocation (Akkermans, et al. 1999). This attempt involved yet again developing an RDR method tailored to the particular problem solving required. However, in the process of this we started to see what might be the general principles underlying a general RDR problem solver. This paper outlines these principles.
RDR Background Methodology RDR provides both a methodology and specific technology. The key features of the methodology that has resulted in successful commercial application are as follows: Ø
A task is identified which has the following characteristics: • there is an information system, which outputs case data. (The origin of the case data is not important) • there is a need for expert interpretation of this case data • it is normal or convenient for experts to monitor the output case data, and in doing so can monitor the output of the KBS as part of this activity. (For example in pathology, it is common for pathologists or senior laboratory scientists to monitor reports. The rule addition, to correct errors, that this results in takes 15 minutes or so per day, and does not interfere with the experts’ normal duties.) Ø Sufficient data modelling and software engineering is carried out to enable case data to be passed to the KBS from the information system, processed by the KBS, and to be presented to experts in a suitable way for them to identify features and add rules. Ø An RDR system embodying standard RDR technology as described below, and the further developments required to handle cases in the domain is then put into routine use. • If during monitoring a case is identified where the KBS output is incorrect in some way, the expert enters the correct conclusion for that part of the output and the case is flagged for rule updating. • The KBS is then updated. However, only a single component of the system is output – a single rule conclusion is corrected at a time. • When rules are updated, the expert is shown a display of the case, perhaps also a previously stored case (a cornerstone case) and perhaps also a list of the differences between the two cases which perhaps also highlights possibly important features. However it should be noted that this assistance is not critical as the expert has already identified features in the case in deciding that the initial conclusion was incorrect.
•
• • •
The expert selects sufficient features to construct a rule to eliminate relevant cornerstone cases. (The expert also has the option of assigning the conclusion to cornerstone cases, if appropriate. This can occur when a system is developed piecemeal, and it has become appropriate to make a more complete response to a case.) The system adds the rule to the KB, so that it will be evaluated in a fixed sequence. The options are described below. The input case is added into the cornerstone case database and linked to the new rule. The case is rerun and more components of the conclusion may be changed by a repetition of the knowledge acquisition process.
RDR Technology A variety of user and expert interfaces have been developed. These have significant differences, particularly in the way cases are presented to experts for rule addition. However, these differences improve the useability of the system without changing the basic structure. More critical are the difference inference mechanisms that have been developed. Single Classification The first approach to an incremental maintenance system was based on a binary tree with a rule at each node, with a child rule replacing the conclusion given by a parent rule. If a case satisfied a rule, it was evaluated against its child rule. The last rule satisfied provided the conclusion given. If conclusion was incorrect, a new rule was added as a child rule. However, if a parent rule was satisfied and the child rule was not, then the new child rule was added to the false branch of the child rule. This preserved the rule evaluation sequence when any case was re-run. Validation required consideration of only one case, the one associated with the parent rule. This case was excluded from satisfying the child rule. This system could only give a single conclusion for a case. Cases were described in terms of attribute-value pairs. This was later extended to temporal sequences of data. For single classification and in following, we assume that there is no default, and that if no rule fires, then no conclusion is given. Multiple Classification To extend RDR to multiple conclusions (MCRDR), an n-ary tree was used. A case that satisfied a rule was then tested against all its child rules. The children of any satisfied child rules were further evaluated against the case and so on. A conclusion is given by each satisfied rule which is not a parent of another satisfied rule. A conclusion from a parent rule is given only if none of its children are satisfied. The cost for the multiple conclusions this allowed for a case was that more cases had to be considered in validation. If a rule was added at the top level, then all past cases needed to be validated against this rule. However, generally a maximum of two or three cases had to be considered before a sufficiently precise rule was provided (Kang, et al. 1998). Again cases were described in terms of attribute-value pairs, and also temporal sequences of such data. Configuration (Intermediate conclusion RDR) The two methods above are useful for classification tasks. The principal extension required for configuration tasks is that there is little distinction between input data and conclusions. Both conclusions and input data are attribute-value pairs. Although some attribute-value pairs are only ever input data, and are never the conclusion of rule, there
is no difference in how these are handled. In some cases some attributes will be given, whilst in others these will result from the inferencing. Some cases may be almost fully determined whilst for others very little or even none of the final configuration may be given. To allow for this data structure, multiple inference passes were allowed, but on each pass attribute-value pairs where the value was unambiguous were added to the case. By unambiguous, we mean that either a single rule assigned a value to the attribute, or if multiple rules assigned values to this attribute, the same value was assigned by all rules. This is essentially a very simple conflict-resolution strategy. Once an attributevalue pair was added to the case, it could not be removed by later inference cycles – it became equivalent to the input data. Inference stopped when values had been assigned to all attributes in the case, or when no further unambiguous values were available. If the case was incomplete, the user could be asked to make a selection between the possible values reached for an attribute and further inference cycles continued. Although important in a practical sense, this is not intrinsic to the inference method. This method can of course also be used for classification and becomes in effect heuristic classification RDR, where some of the attributes are intermediate conclusions. Knowledge acquisition for configuration is identical to multiple classification. What this meant in practice was that inference was stopped after each cycle, when the expert was free to correct or add any conclusion they liked using the method noted above. The case stored was the case available at the time of knowledge acquisition; i.e. a case with the original input data and whatever attributes whose values had been decided on previous cycles. The new conclusions were then added to the case and further cycles processed one by one. Experimental results suggest that when the KB is starting out, such cycles in knowledge acquisition will be fairly common, but this is rapidly reduced to one or two (Compton, et al. 1998). The most common problem solving method for configuration tasks is propose and revise where some sort of best-first search proceeds until constraints are broken, when the search is moved to a new part of the space. With RDR we have a very conservative search strategy where only attributes with unambiguous values are irreversibly added to the case. However, rather than a revision step in reasoning there is a revision step in knowledge acquisition allowing the addition of further knowledge to make the search strategy work for that case. Results, albeit very partial, suggest that this rapidly improves the performance of the very simple search (Compton, et al. 1998). Search (Nested or Backward chaining RDR) Nested RDR (Beydoun & Hoffman 1999) provides a series of single classification trees. The conclusions are binary in that a single classification is either true or false. The conditions this tree’s rules may be provided by input data or by another tree. Conditions in its rules may again be provided as the conclusions of further rules. The trees are recursively processed in a similar way to conventional backward chaining. Although cycles are possible with such a structure, the expert is prevented during knowledge acquisition from adding rules that may result in cycles. If Nested RDR allowed for conclusion attributes with multiple rather than binary conclusions, it would appear to be comparable to configuration RDR. There are a number of levels of conflict resolution with NRDR. Firstly NRDR is aimed at narrowing search and is used in conjunction with some kind of kind of search. This search has it is own conflict resolution strategy. Secondly, at the RDR level, there is no need for conflict resolution since there is only a single tree for each attribute, which makes a single conclusion. Further the potential conflict resolution in each SCRDR tree is managed by the implicit prioritising of rules, according to when they are added. Although NRDR are used in a backward chaining mode, they could potentially be used in a forward chaining mode. In this case the problem of cycles could be resolved by the
mechanism used in MCRDR where once a fact is added to working memory it is not removed. Knowledge acquisition would have a similar process of stopping and correcting errors on each inference cycle. With these changes NRDR would be very similar but not identical to configuration RDR. The single classification rule ordering may produce slightly different evolution from the configuration RDR strategy of only adding unambiguous conclusions. An interesting feature of NRDR, is the way, in which it keeps and uses the inference history. NRDR was developed for search tasks, in particular for playing chess. This means that the output of each inference is what to do at the particular step the search is up to now. However in adding rules, the expert may wish to base a rule on past moves. The most important piece of information about what to do at a current move may be what happened in a previous move. RDR has no notion of strategy, as strategy is not part of the data available. However, a particular sequence of moves may be indication of a strategy under way. The expert in basing his move on past moves, is basing it on the data by which he or she recognised the strategy being followed. Resource allocation RDR The issues that arose in developing RDR for resource allocation motivated us to develop the generalisation proposed in this paper. It is therefore described in greater detail than the above. Full details are given in (Richards & Compton 1999) Sisyphus I was a problem developed by the KA community to provide a basis for comparing KA approaches (JHCS 1994). It was a room allocation problem where the material provided included relevant information about a set of offices to be occupied, information about the staff who were to be assigned to offices and a protocol describing how an ‘expert’, Siggi D, carried out the allocation. The problem was small, with only 16 people to be allocated to offices. In keeping with an RDR approach this data was used as case data to assign people to rooms with the rules used being based on the protocol of how Siggi D carried out the allocation. The result then is an allocation of the given staff to the given offices according to Siggi D’s rules. However, we have no information on whether this would work on another set of data, extensions to the present set etc. The same problem applies to other Sisyphus I solutions: that the system developed is not proven to work on other data sets. The key features of the initial solution were: Ø The data was divided into people cases and room cases with a ‘case’ for each person and a ‘case’ for each room in the data set. Ø Since a rule is a justification for a choice, rules consisted of features (attribute-value pairs) of the person and features of the room occupied, that justify that the person is appropriate for the room. The rule conclusion is that the person and room match. Such a rule base could be used to check room allocations, however the rules here also have a rule action of allocating the person to the room. A refinement rule represents a decision that a different allocation was more appropriate.
Ø Single pass MCRDR was used for inference (and knowledge acquisition). However, in given the data is a set of people cases and room cases, this does not identify which data to apply to the rule. There are two data selection algorithms •
The user selects a person to allocate to a room. As each rule is processed, room cases are retrieved to find a room or rooms which match the rule. Implicit in this, is a check on people who are already occupants of the room, neighbours etc., to make sure that the rule conditions are satisfied. An incorrect allocation results in a new rule being added by the expert.
•
The system simply selects the next unallocated person. This is followed by the same steps as above, however because the people are processed in random order, there is a greater possibility of inappropriate allocation. However, processing is again one person at a time and at any stage the user can decide to de-allocate a previously allocated person, and re-allocate the room, or simply add a rule to carry out a better allocation than the one proposed.
•
If no room was found for the person under consideration, the user selects from a list of the unoccupied rooms and adds a new rule for this selection.
Ø Some extra functionality was required: •
Room and person retrieval to check against the rule was the major new feature.
•
When a person was allocated to a room, the cases were updated where appropriate. The occupant slot for the relevant room was updated, as well as the room slot for the relevant person, as well as other slots such as close_to.
•
The graphic interface was modified and some support functions such as showing all the unoccupied rooms, were added
Sisyphus I Results With the user selecting the order of the cases, 16 people were allocated rooms and 7 rules were added. Of these only 1 was an exception rule. The 3rd, 7th , 9th and 11th to 16th cases processed used rules added earlier in the KA. The increase in latter cases using pre-existing rules suggests that there is some convergence in covering the domain; however, the data set is far too small to assert this with any confidence. With random ordering of staff to be allocated offices, 7 rules were again added, with 1 exception rule. However, there were also 2 de-allocations carried out, 1 of an office with a single occupant the other of an office with 2 occupants. The staff who were deallocated, did not require new rules to be added. They were simply rerun immediately after their office had been re-allocated to someone else and successfully allocated another office. The suggestion of convergence was somewhat different this time, with the 2nd, 3rd, 6th (1st rerun), 7th (2rd rerun), 9th (3rd rerun), 10th to 14th, 18th and 19th cases all using pre-existing rules. Our previous paper contains a detailed discussion of this study as well as a comparison to the other Sisyphus I solutions. Some of these were also based on various types of classification systems, but none attempted an incremental refinement of the knowledge base.
Generalising RDR The most important feature of this generalisation is discussed below in the section on knowledge acquisition and inference, but some discussion of the notion of case, rule action and rule conditions is also required. Case A case is all the information available to the expert when they make a decision. The proper case then for the room assignment problem is not just the individual who is being assigned to a room, but all the individuals and all the rooms, as far as these are known. This would require a representation system able to deal with object-attribute-value triples, sets, relationships e.g. next_to in the Sisyphus I problem. It probably also requires some notion of modifiers. E.g. A room allocation problem may have some of the rooms already occupied. These may need to be flagged as unchangeable allocations etc. There are of course functions to abstract data into more significant features etc. Early RDR dealt only with attribute-value pairs, for the more general notion of a case outlined here, some sort of frame system is probably appropriate, as is common in other KA and ontology research (Grosso, et al. 1999). However, a significant difference would be that there would be no requirement for reasoning via inheritance as well as via rules as occurs in most frame systems. The frames would be strictly data templates for assembling a case. The role of demons (or widgets) will be outlined below. We might note that with the chess-playing problem in NRDR, the case is not simply the current board positions, but the whole history of the game. The issue is not a special type of reasoning, but an adequate representation so that everything the expert deals with can be represented in the ‘case’. Obviously, to deal with scheduling, planning, design problems etc, spatial and temporal localisation will be needed as well as information on steps taken. Clancey’s notion of a Situation Specific Model should be noted here (Clancey 1992). Essentially an SSM is the data that was reasoned about plus everything further that was learned about the data as part of the reasoning process. In the following section we describe how a rule action adds to a case corresponding to Clancey’s notion. Breuker has a similar notion of a case (Breuker 1994). Rule actions It seems clear to us that the action of a rule even in a classification system is not to simply assign a classification to the data being processed, but to assign a value to the classification attribute of the case. It is very clear in configuration problems that rules assign values to various attributes. In room assignment, the rule action is to assign a value of the type person to the occupier slot of room object and to assign a room value to the occupant_of slot for the relevant person. In a classification system the justification provided by the expert justifies their action is assigning a particular classification to the case. In a room allocation problem, the expert justifies their action of assigning a particular person to a particular room. This raises the question of the role of demons in the frame representation proposed. In the approach proposed, certain types of rule actions could be defined, so that all the necessary consequences of assigning a person to a room were defined. Alternatively demons could be used, so that when for example a particular person was assigned to an occupier slot of a room, the occupier slot demon would update the person’s occupant_of slot. In terms of RDR this is potentially unsafe, as we would want demons only to carry out the implicit rule action. However, no mechanism has been
defined for ensuring that demons would only be able to carry out such actions. Implicit in this is the notion that a rule has a single action1 . If a rule carried out multiple actions, then it may be unclear which particular action is being replaced by a refinement rule. What we are concerned with in the room allocation is a single action, but which appears to have multiple components because of the way the case is represented. This requires further development. However, the central notion is that a rule makes a single change to the case. It normally assign a value to something previously unknown. Rule conditions Rules conditions refer to the data in the case. However, the inference engine will need to be able to deal with rules of the form: If
(the case contains an object of type A, with attribute B having value C) and (the case contains an object of type X, with attribute Y and value Z)
Then carry out some single action on this particular A type object and X type object Although RDR systems have not dealt with such rule conditions before, there is no intrinsic difficulty of handling such rules. It can be noted in passing that Claude Sammut has developed version of UNSW Prolog that incorporates RDR (single classification only at this stage) and a frame system (C. Sammut personal communication). This would support such rules and data representation. Rules can also query relational information about the case. E.g. if there is vacant room next_to a manager etc. It could be speculated that reasoning in rules could become very complex. We do not believe this will the case. RDR allows the expert to provide fairly simple justifications for their decisions because these can be refined when errors are found. It should be noted also that the reasoning remains propositional: rules are only concerned about whether certain things are true about a case. Knowledge acquisition and inference structure Knowledge acquisition and inference are inextricably linked in RDR, so some of the knowledge acquisition steps depend on the inference structure described below and vice versa. The key feature of knowledge acquisition is that we do not want the expert to provide us with any problem-solving knowledge. We want the expert simply to justify their individual decisions in terms of the case data to hand. An RDR refinement of problemsolving knowledge may be possible, but we believe it is more appropriate simply to ask the expert to justify their decision in the light of case data. Their core expertise is in making decisions in the light of data. Being able to justify a decision by identifying the features of the data that support the decision, is not essential to making a decision. However, it is not too far removed from it and in any decision where some level of rationality is involved we would expect experts to be able to identify relevant features in the case. Experience with RDR shows that they do not give all the significant reasons on the first pass, but normally not more than two or three refinements are required. On the other hand the ability to explain decisions in terms of problem solving strategies is considerably more removed from more actual decision making. Secondly any justification of problem solving strategy would go beyond the present case to the expert's overall experience. If one were attempt to obtain a justification for a problemsolving rule simply in terms of the case data at hand we believe the justification would 1
Ghassan Beydoun has proposed that the problem of multiple actions for rule may be to have the action or actions proposed and their consequcnes as part of the conditions of a rule, with the conclusion of the rule simply assigning some class to these changes. This would allow any number of actions associated with a rule. This remains to be investigated further.
become a series of sequential justifications for individual discussions. This is what we propose to capture in any case in dealing with justifications for individual decisions. It seems to us that in construction tasks like resource allocation, or planning, the expert will attempt to set up a sequence of steps that lead to an appropriate solution. Each step represents a decision as to what that step will be. The expert will attempt to carry out the decisions, or set up the sequence in such a way that no backtracking and undoing of past decisions is required. In practice, often the expert may make a mistake and some backtracking may be required. However, if an RDR system can be built for such a task, the rules would be refined so that the next time the appropriate sequence was followed. It seems clear to us that for any problem involving a sequence of steps it must be possible to put the steps in such an order that no backtracking is involved. Our aim then is an RDR structure, which will allow individual decisions to be corrected, but in doing so to also correct the order in which the decisions are made. Inference structure SCRDR and MCRDR act on the case (assign on or more classifications) on the completion of inference. Configuration RDR is MCRDR but with multiple inference cycles and a conflict resolution strategy to determine the changes to be made to the case at the end of each inference cycle – after all the rules have been evaluated. We have hypothesised how NRDR might be used under similar circumstances; at one level the SCRDR used in NRDR means that no conflict resolution would be required. We assume that at the next level NRDR would adopt the configuration strategy of once a change was made to case, this would not be undone. Our aim is to have some way of capturing the order in which decisions are made without asking the expert about ordering rules, but also to be able to refine this order. We propose: • • • • • • • •
When an error is made a rule is added which is a refinement of the incorrect rule. The conclusion from any particular refinement path, is the conclusion from the last rule in the path, which was evaluated as true. (Both of these are standard MCRDR) The order in which rules are added to the system is recorded. However, rule evaluation is depth-first, so that eldest child of a rule is evaluated first and then its eldest child until a child rule fails to fire. The children of a rule that fails to fire are not evaluated. The siblings of the last rule that fired are evaluated in the order in which they were added to the KB, with any refinements of these rules evaluated in the same depth-first fashion The whole tree is recursively evaluated in this way The conclusion from the last true rule in a refinement pathway is added to the case before the next rule is evaluated
Summarising this algorithm: 1) inference starts with the first rule added to the knowledge base, which is evaluated in the following fashion. Before inferemce starts the tentative conclusion is null. 2) evaluation: step A if X fires then X’s conclusion replaces the tentative conclusion. and X’s eldest child (Y) is evaluated (recursively repeating step 2) if
Y fails to fire then Y’s next eldest sibling is evaluated (recursively repeating step 2)
step B when there are no more siblings to be evaluated (or there were no children of X) the tentative conclusion from X is acted on resulting in an addition to the case. A new null tentative conclusion is formed. step C the next eldest sibling of X is evaluated (recursively repeating step 2). After all the siblings of X are evaluated, control moves back to the parent of X and the process repeated. 3) The entire process is repeated starting with the first rule, until no more changes are made to the case. (This step is not strictly necessary, but may reduce the requirement for rule addition, particularly for problems where the order and expert carries out the task is not so important) 4) Two possible variants of step 3 are that after each conclusion is acted on and a new null tentative conclusion is formed: a) inference returns to the parent of the rule whose conclusion was acted on. This would still require that the whole inference process be repeated b) inference returns to the first rule in the KB. There would be no need to for further inference cycles. This process starts with the first rule added to the KB. The tentative conclusion the system starts with is the null conclusion, i.e. the case is not acted on and changed. The actions allowed are additive, i.e. the specify values for attributes that were previously unknown. A more abstract way of characterising this process is to define a KBS as something that takes in data (a case) and on the basis of that data, outputs further data to be added to the case. In our framework KBS can be combined in only two ways. There is either a simple sequence whereby the case data plus any additions made to it are passed to the next KBS, or the case is passed to a correction KBS which is authorised to override the additions to the case proposed by the previous KBS and instead make different additions. The priorities in these links are that a refinement KBS will be traversed before a sequential KBS. A KBS is composed of other KBS with only sequential or correction links allowed. In the system we have described here, at the bottom level a KBS is a single rule.
We might also note in passing, that although this algorithm does not allow changes to the case made during inference to be undone by later inference steps, it would be possible to flag input data as fixed or changeable. Changeable allocations would not be automatically changed during inference, but rules could refer specifically to changeable data to allow this. Knowledge Acquisition Knowledge acquisition here is based on the same principle as configuration RDR. That is when an error is made the expert runs the case, but stops inference at the point the error is made. A refinement rule is then added, so this will become the last satisfied rule on that path and its conclusion will be acted upon rather than the conclusion of the previous rule. The expert then steps through the remainder of the inference, possibly needing to add more rules to overcome changes resulting from the addition of the previous rule. This may occur because rules can depend on features in the case added on previous inference steps. The effect of this is that if in the room allocation problem a secretary was assigned to the wrong room, because she was assigned before the manager she was working for, then the rule assigning the secretary would have a refinement rule which would fire if the manager was unassigned, with the manager assignment over-riding the secretary assignment. If the secretary rule was re-processed, it may well fire as the refinement rule assigning the mangager would no longer fire (as the manager was already assigned). With step 4) above, the secretary rule would not be re-evaluated until a second pass through the knowledge base. With option 5a) it would be reprocessed immediately, and with option 4b) all previous rules would be re-evaluated before the secretary rule, but none of the subsequent rules would come before the secretary rule. If this particular secretary rule no longer fired, a later rule may now fire, appropriately assigning the secretary, or it may be necessary to add a new rule, or new correction. In option 3) the new rule would be added at the end of all previous rules, as adding it earlier would not result in earlier evaluation. With options 4a) and 4b) the rule can be added at any inference step. It would be added as the sibling to the rule that gave the conclusion. As well as changing the priority of rule conclusions by refinement, the order in which rules were evaluated and acted on, would reflect the order in which the expert dealt with different problem cases. Errors in ordering would be corrected by the mechanisms above. It should be noted that this is not just a matter of simply repeatedly changing the order in the hope that there would be convergence. Each change in the order is caused by the addition of a refinement rule. That is the action of a rule, which itself only fires in certain circumstances, is replaced by the action of another rule, only if this rule also fires for this case. As with other RDR methods, all cases that resulted in the addition of rules would be kept. If the application of the new rule, would result in a change to these previous cases, then the expert would be required to make the rule more precise. Obviously there are significant interface problems in dealing with these cases, which are beyond the scope of this paper to address. All RDR systems implicity use the order of rule addition in evaluating refinement rules. SCRDR also uses the order of rule addition to prioritise the order of rule evaluation, so that once a rule fires, only it’s children are evaluated, but no later siblings. However, SCRDR gives only a single conclusion, rather than a gradually assembled solution. MCRDR and NRDR may allow some control of the order in which a solution is added, but they do not directly capture the order in which the expert assembles a solution. The more generalised RDR proposed here, while able to handle previous RDR problems now also captures the order in which an expert develops a solution.
In terms of problem solving, the central idea that we are proposing is that one can keep correcting knowledge in a rule base until it manages to find appropriate solutions for different cases without any backtracking. At this stage we have no proof that this structure will work. However, methods like propose and revise, which explicitly allow revisions or backtracking have been shown to be poor at finding a solution unless the search is given a kick-start by using a case-based reasoning approach to find an appropriate starting point in the space (Motta and Zdrahal). However, such methods do not allow incremental improvement to the knowledge base, or incremental improvements in how the direction of the search is controlled. Our hypothesis, is that these incremental changes, can be used to gradually build knowledge bases that enable solutions to be found, simply by a best first search ( depite their being no actual search), without any revision. Another way of putting this is that rather than having propose and revise as part of the inference, we have moved the revise step to knowledge acquisition. Our best evidence that this will work is the ion chromatography configuration studies (Compton et al), where extremely rapid convergence was found. However, this work needs to be repeated with a larger and perhaps more randomised data set. More importantly we need to evaluate the approach on a large-scale resource allocation problem. Given that the method outlined above maintains the basic incremental RDR structure, and this has been highly successful elsewhere, we are confident that it will provide a more general incremental KA structure. We see the major issue in such a system is not so much the convergence, but the difficulties of representation and providing an appropriate interface for the expert. However, these are problems faced by any KBS for such domains.
Acknowledgements This research has been partly funded by the Australian Research Council. The authors thank Ghassan Beydoun for his helpful comments.
References Akkermans, H., Speel, P. and Ratcliffe, A., Problems, Opportunity and Feasability Analysis for Knowledge Management - An Industrial Case Study in B Gaines, R Kremer & M Musen, 12th Banff Knowledge Acqusitiont for Knowledge-Based Systems Workshop, 1999 (Banff, SRDG Publications, University of Calgary, 1999), pp 2-1.1 - 2-1.22. Beydoun, G. and Hoffman, A., Acquisition of search knowledge in E Plaza & R Benjamins, Knowledge Acqusition, Modeling and Management, (Berlin, Springer, 1997) , 1-16. Beydoun, G. and Hoffman, A., Building problem solvers based on search control knowledge in B Gaines & M Musen, 11th Banff knowledge acqusition for knowledgebased systems workshop , 1998 (Banff, SRDG Publications, University of Calgary, 1998), pp SHARE 3, 1-18. Beydoun, G. and Hoffman, A., Hierachical Incremental Knowledge Acquisition in B Gaines, R Kremer & M Musen, 12th Banff Knowledge Acqusitiont for KnowledgeBased Systems Workshop, 1999 (Banff, SRDG Publications, University of Calgary, 1999), pp 7-2.1 - 7-2.20. Breuker, J., Components of Problem Solving and Types of Problems in L Steels, G Schreiber & W Van de Velde, A Future for Knowledge Acqusition: Proceedings of EKAW'94 , (Berlin, Springer-Verlag, 1994) , 118-136.
Clancey, W.J., Model construction operators, Artificial Intelligence 53 (1-115) (1992). Compton, P., Horn, R., Quinlan, R. and Lazarus, L., Maintaining an expert system in J R Quinlan, Applications of Expert Systems, (London, Addison Wesley, 1989) , 366385. Compton, P., Preston, P. and Kang, B., The Use of Simulated Experts in Evaluating Knowledge Acquisition in B Gaines & M Musen, Proceedings of the 9th AAAISponsored Banff Knowledge Acquisition for Knowledge-Based Systems Workshop, 1995 (Banff, Canada, University of Calgary, 1995), pp 12.1-12.18. Compton, P., Ramadan, Z., Preston, P., Le-Gia, T., Chellen, V. and Mullholland, M., A trade-off between domain knowledge and problem-solving method power in B Gaines & M Musen, 11th Banff knowledge acqusition for knowledge-based systems workshop , 1998 (Banff, SRDG Publications, University of Calgary, 1998), pp SHARE 17,1-19. Compton, P.J. and Jansen, R., A philosophical basis for knowledge acquisition, Knowledge Acquisition 2 :241-257 (1990). Edwards, G., Compton, P., Malor, R., Srinivasan, A. and Lazarus, L., PEIRS: a pathologist maintained expert system for the interpretation of chemical pathology reports, Pathology 25 :27-34 (1993). Grosso, W., Eriksson, H., Ferguson, R., Gennari, J., Samson, W. and Musen, M., Knowledge Modelling at the Millenium (the Design and Evolution of Protege-2000) in B Gaines, R Kremer & M Musen, 12th Banff Knowledge Acqusitiont for KnowledgeBased Systems Workshop, 1999 (Banff, SRDG Publications, University of Calgary, 1999), pp 7-4.1 - 7-4.36. JHCS, (1994).
Sisyphus I Special Issue,
Journal of Human-Computer Studies 40 (2)
Kang, B., Compton, P. and Preston, P., Multiple Classification Ripple Down Rules : Evaluation and Possibilities in B Gaines & M Musen, Proceedings of the 9th AAAISponsored Banff Knowledge Acquisition for Knowledge-Based Systems Workshop, 1995 (Banff, Canada, University of Calgary, 1995), pp 17.1-17.20. Kang, B., Compton, P. and Preston, P., Simulated Expert Evaluation of Multiple Classification Ripple Down Rules in B Gaines & M Musen, 11th Banff knowledge acqusition for knowledge-based systems workshop , 1998 (Banff, SRDG Publications, University of Calgary, 1998), pp EVAL 4, 1-19. Kang, B., Yoshida, K., Motoda, H. and Compton, P., A help desk system with intelligent interface, Applied Artificial Intelligence 11 ((7-8)) :611-631 (1997). Lee, M. and Compton, P., From heuristics to causality in J Lee, J Liebowitz & Y Chae, Proceedings of the third world congress on expert systems, 1996 (Seoul, Cognizant Communications, 1996), pp 946-953. Martinez-Bejar, R., Benjamins, R., Compton, P., Preston, P. and Martin-Rubio, F., A formal framework to build domain knowledge ontologies for ripple-down rulesbased systems in B Gaines & M Musen, 11th Banff knowledge acqusition for knowledge-based systems workshop , 1998 (Banff, SRDG Publications, University of Calgary, 1998), pp SHARE-13,1 -20.
Martinez-Bejar, R., Shiraz, H. and Compton, P., Using ripple-down rule-based systems for acquiring fuzzy domain knowledge in B Gaines & M Musen, 11th Banff knowledge acqusition for knowledge-based systems workshop , 1998 (Banff, SRDG Publications, University of Calgary, 1998), pp KAT-2.1 - KAT-2.20. Richards, D. and Compton, P., Uncovering the conceptual models in ripple-down rules in D Lukose, H Delugach, M Keeler, L Searle & J Sowa, Conceptual Structures: Fulfilling Peirce's Dream, Proceedings of the Fifth International Conference on Conceptual Structures (ICCS'97) Lecture Notes in Artificial Intelligence, No 1257, (Berlin, Springer Verlag, 1997) , 198-212. Richards, D. and Compton, P., Revisiting Sisyphus I - an Incremental Approach to Resource Allocation Asing Ripple-Down Rules in B Gaines, R Kremer & M Musen, 12th Banff Knowledge Acqusitiont for Knowledge-Based Systems Workshop, 1999 (Banff, SRDG Publications, University of Calgary, 1999), pp 7-7.1 - 7-7.20. Shiraz, G. and Sammut, C., Combining Knowledge Acquisition and Machine Learning to Control Dynamic Systems in 15th International Joint Conferences on Atrificial Intelligence (IJCAI'97) 1997. , 1997 (Nagoya, Japan, Morgan Kaufmann, 1997), pp 908-913. Suryanto, H., Richards, D. and Compton, P., The Automatic Compression of Multiple Classification Ripple Down Rule Knowledge Base Systems: Preliminary Experiments in L Jain, Proceedings of the Third International Conference on Knowledge-Based intelligent Information Engineering Systems (IEEE Cat. No. 99TH8410), 1999 (Adelaide, 1999), pp 203-206.
