United States and Europe are developing prototype associate systems, evidence of a growing interest in this technology.This is particularly promising to the ...
Learning Approach to Knowledge Compilation A Pilot’s Associate Appkation Keith R. Levi, Maharishi International University David 1. Perschbacher, Alliant Techsystems Mark A. Hoffman, ISX Christopher A. Miller and Barry B. Druhan, Honeywell Systems and Research Center Valerie 1. Shalin, State University of New York, Buffalo
United States and Europe are developing prototype associate systems, evidence of a growing interest i n this technology.This is particularly promising to the future automation of complex systems such as spacecraft, aircraft, and other advanced vehicles. In these contexts, an automated associate helps a human pilot perform a mission by obtaining and compiling information, recommending actions and, in some cases, carrying out certain mission tasks. Associate systems generally require multiple, large, interrelated knowledge bases. The high-technology environments in which such systems operate require frequent alterations and updates to the knowledge bases. Thus, automating portions of this knowledge-engineering problem is critical. We are using explanation-based learning*-2as part of a larger knowledge compilation system for automating the development and maintenance of associate knowledge bases. Knowledge compilation’ restructures an existing system or knowledge base to increase efficiency, usability, and consistency. The process typically changes the representation level, reduces or summarizes the reasoning steps, and decreases the explicitness or transparency
W E ARE APPLYZNG EXPLANATION-BASED LEAWING TO THE PROBLEM OF DEVELOPZNG AND MAINTAINZNG THE KNOWLEDGE BASES OF INTELLIGENT AUTOMATED ASSOCIATE SYSTEMS. SULZNGUP TO COVER AN ENTZRE DOMAZN REMAZNS AN IMPORTANT OPEN QUESTION.
of the knowledge. Our system exhibits all these defining characteristics. Figure I shows the underlying components of a knowledge compilation system. A domain expert provides an initial representation whose form often cannot be interpreted effectively by a target performance system. The knowledge compilation system translates the representation into one that is more compatible with the target system’s computational requirements. The knowledge acquisition community has recognized the problem of representation mismatch: that is, the representations desirable for acquiring knowledge from a human expert often differ from those desired for processing by a knowledge-based artificial-intelligence ~ y s t e m Much .~ current work focuses on bridging this gap. This article shows how a knowledge
compilation approach resolves this knowledge acquisition problem.
Knowledge compilation for Pilot‘s Associate Not all AI systems need a knowledge compiler. In relatively simple systems (such as Mycin-like rule-based systems), an expert might be able to give rules that the expert system can use directly. Another approach is to build expert-system inference engines that closely match the task requirements. This interpreter-modification approach3 is advocated by researchers in model-based reasoning and generic-task architecture^.^ When feasible, such solutions to the representation mismatch problem are preferable because they are simpler:
, I
I
They avoid the need for the additional software of a knowledge compiler. However, such solutions are not feasible in many cases. Intelligent associate systems, for example, must support real-time planning and decision making i n dynamic and evolving situations. Pilot’s Associate6 is one of the largest and most thoroughly developed associate systems. It is a coordinated suite of five expert systems (see Figure 2 ) that help the pilot of an advanced tactical fighter aircraft. The Tactics Planner receives most of its input from the Situation Assessment subsystem, which analyzes and describes the state of the world outside the aircraft. The System Status subsystem fills a similar function for the world within the aircraft. The Mission Planning subsystem provides mission routes and suggests tactical routes with low observability. All Tactics Planner suggestions are routed to the pilot through the Pilot Vehicle Interface, which also interprets pilot intent. Finally, all interactions take place through the Mission Manager, which keeps a complete record of subsystem interactions and current context. There are three principal reasons why Pilot’s Associate requires a knowledge compilation approach as opposed to an interpreter-modification approach. First, Pilot’s Associate addresses a very complex problem that requires a combination of planning, monitoring, assessment, and diagnostic capabilities and that cannot be solved by a single generic task such as diagnosis or design. Second, the system’s real-time requirements are challenging. Pilot’s Associate must perform large amounts of processing over distributed yet cooperating systems in a matter of seconds, so the expert systems tend to be highly procedural and have very specialized representations. The knowledge i n the runtime subsystems must be highly compiled if the system is to achieve the very fast response times required. The first two reasons why PA requires a knowledge compilation approach lead to the third, which involves knowledge acquisition. The performance requirements for PA are so challenging that the representation required for PA cannot be tailored to be compatible with the way pilots express their knowledge. Similarly, despite repeated efforts, pilots have not been able to change the way they express their ~ _ _ _ _ _ _
JUNE 1992
compilation
~
~~
~~
Figure 1. Knowledge compilation approach to representation mismatch (adapted from Goel et al.%
Aircraft sensors
1 Sensor data
j Sensor data
Pilot Vehicle Interface
~~~
Mission
~
Figure 2. Conceptual architecture of Pilot’s Associate.
knowledge to be more compatible with the PA representation. Traditionally, such mismatches have been bridged by human knowledge engineers who, largely by hand, translated expert knowledge into the representations necessary for knowledge-based systems. This approach is less than optimal for large, distributed systems like Pilot’s Associate for several reasons: The knowledge bases would be immense. The system would be subject to continual modifications in response to technology changes, and updates would have to be made very quickly. Verification and validation would be required. (This would be facilitated by automated knowledge-engineering
systems that impose consistency and formalism.) The system would have to be maintained by Air Force personnel, who are not the original developers.
Our approach Our program, Learning Systems for Pilot Aiding (LSPA), is automating portions of the off-line process of incorporating new information into Pilot’s Associate knowledge bases and then propagating pertinent changes to other Pilot’s Associate module^.^-'^ We describe only implemented systems here (except where otherwise noted).
~~
45
I
Figure 3. Knowledge compilation in LSPA.
An initial study'.' recommended expla- also be represented and reasoned about. nation-based learning for knowledge ac- The second step is to transform the macro quisition in our domain. This approach into a plan representation. This stage rerequires a domain theory of facts and rela- quired that we develop both semantic and tionships in the target domain, as well as syntactic principles for distinguishing difa learning instance (a new case not previ- ferent kinds of information in the macro. ously understood by the system).' In LSPA, Finally, a knowledge engineer performs the learning instance can come from records the final transformations to obtain a fully of pilot behavior in a flight simulator. The executable and tested plan. domain theory is a set of relatively unLSPA has several novel aspects. Typical changing facts about the aircraft's physical explanation-based-learning applications capabilities and the world it inhabits. When generate macros to increase the target systhese facts and relationships do change, tem's processing speed." In such applicasuch as when new technologies are incor- tions the target system uses the same knowlporated, the domain theory must be up- edge representation as the domain theory. dated. When a pilot uses a tactic for which In essence, the domain theory is compiled the Tactics Planner does not have a corre- for speed. In contrast, our approach uses sponding plan, LSPA can automatically the domain theory to interpret the scenario generate such a plan for the Tactics Plan- trace and create a new plan in a representaner by using its domain theory and a record tion very different from the explanationof the flight in which the tactics occurred. based-learning domain theory-that is, the LSPA builds an explanation ofhow a known representation of the Tactics Planner. Since goal was accomplished via its understand- knowledge compilation happens off-line, ing of the world; then it generalizes this we are not greatly concerned with the speed explanation to produce a new plan to be at which the system compiles tactical plans incorporated into the system's knowledge , from the domain theory. We do not even bases. (In this article, a plan is a special- put the macro back into the domain theory. ized, complex data structure developed by , This distinction is important: Most explanation-based-learning systems slow the Tactics Planner.) Figure 3 shows this process based on the down after acquiring many macros, due to general knowledge compilation framework the increased computational cost of matchin Figure 1. In this case, the expert is a pilot ing the patterns of the macros i n addition to who produces a domain theory of facts and matching the original rules in the knowlrelationships about aircraft tactical ma- edge base. We do not add our macros back neuvers. LSPA interprets and compiles this into the domain theory, so this so-called domain theory into a plan that is close to utility problem" is not a problem for us. being executable. There are two kcy steps The utility problem also does not arise for in this translation process. First, LSPA the target system (the Tactics Planner), creates a macro-rule that generalizes and since we are not trying to totally automate summarizes the explanation ofthe learning the knowledge-engineering process. It is therefore quite reasonable for the knowlexperience. (A macro-rule, which we'll call a macro from now on, is a rule formed edge engineer or pilot to identify the learnby taking the leaves of an explanation tree ingexperience and goal. This humandirecas the antecedents and the root ofthe tree as tion will help ensure that only useful plans the consequent. If the antecedents are prov- are acquired. A key characteristic of Pilot's Associate ably true, we can assume the consequents are true also.) This process is similar to is its distributed nature. Since the various typical explanation-based-learning appli- modules are highly interrelated, adding or cations. except that temporal relations must modifying information in one typically ~
46
requires updates to the others. For example, each plan in the Tactics Planner has a corresponding data structure in the Pilot Vehicle Interface that represents the information the pilot needs to perform the plan. To address this aspect of Pilot's Associate, the LSPA system has two subsystems: the Learning System for Tactics Planning (LSTP) acquires plans for the Tactics Planner, while the Learning System for Information Requirements acquires information requirements data structures associated with plans acquired by LSTP."' Here we focus on LSTP.
The learning algorithm Figure 4 illustrates the general stages of the LSTP translation process. Before using our system. the pilot and a knowledge engineer must identify a learning opportunity, that is, the missing plan. To initiate the learning process, the pilot and knowledge engineer develop a trace of a tactical scenario that contains an example of a tactic for which they want to learn a plan. One way to generate this trace is to have a pilot fly the learning scenario on a flight simulator. The simulator maintains a time-stamped record of all state changes in the form of message packets. which include the pilot'scontrol actions; the plane's position. velocity. and azimuth; and enemy aircraft, missiles, and ground threats such as surface-to-air missile (SAM) sites. Our explanation-based-learning shell has two parts, an explanation generator and an explanation generalizer. The generator requires three inputs: a scenario trace, the pilot's goal. and a predefined knowledge base (domain theory). The pilot and the knowledge engineer select the goal, which serves as the target concept to be learned, and generate the domain theory using a traditional knowledge-engineering process. Acquiring this domain theory is a critical issue for any application of explanationbased learning. ~
IEEE EXPERT
1 ~
L--------
~
~~~
~
~
~~~
1
Figure 4. High-level system architecture of LSTP.
The generator constructs an explanation of how the pilot achieved the goal in terms of the domain theory of general knowledge about tactical plans and maneuvers. The explanation i s quite similar to a logicalproof tree. The root corresponds to the pilot’s goal, while the branches correspond to rules in the domain theory linked via a simple postcondition-achieves-precondition relationship. The leaves relate the learning scenario to the domain theory. The explanation generator also includes a parser submodule that preprocesses the simulator packets for the explanation generator. Normally, we would make the parser separate from the explanation generator; however, since we had to modify the standard explanation-based-learning algorithms to handle the temporal nature of this domain, the explanation generator had to be able to use the parser repeatedly. The explanation generator sends its output to the explanation generalizer. This module, which i s based on the EGGS algorithm (Explanation Generalization using Global Substitution),” syntactically generalizes the explanation data structurc by pruning scenario-specific details from the explanation and hence from the plan to be learned. The output is an explanationbased-learning macro representing the gcnera1 features and constraints of the plan that enabled the pilot to achieve the goal. Finally, the macro is input to a macro translator that partitions and translates the macro into an executable plan.
Modifying the learning algorithm
’
0t her mac h in e- 1earn in g s y ste i n s I 3have learned tactics from simulator tracer. but their applications involve flight ~
~~
JUNE 1992
simulations in which the planes are autonomous. This means they can ignore both the complexities of having a person in the loop and the temporal nature of the domain. Their planners are best described as reactive, and their primary learning algorithms are not explanation-based, but rather are similarity-based (such as genetic algorithms and AQI I ) . Although novel, our application of standard explanation-based-learning techniques is reasonably straightforward-with one important exception. The original proccdures. as outlined by earlier researchers,’-’ apply to a broad range of problems but are not sufficient for Pilot‘s Associate. Specifically. they cannot deal with temporal i n formation or incorporate temporal relations into the explanation of a learning instance. Sincc almost all plans i n the Pilot’s A s o c i a t e Tactics Planner have a critical temporal component, we were forced to develop new techniques for representing temporal components i n the learning instance and temporal relations in the rules of the domain theory.
Parsing temporal intervals. Simulator runs can easily produce several hundred time-stamped messages a minute. Even for scenarios that la\t only several minutes. this is far too many states to reason about with a traditional AI planning approach. Further. the simulator only produces messages when some object changes its value. For example, if the radar is on from time point I O to time point 530, the simulator would produce one message at time I O saying the radar is on, and another at 531 saying the radar i s off. The fact that thc radar i s also on from time 1 1 to 530 (and off froin 0 to 9 and 532 to the end) must be inferred. Our wlution is to parsc message traffic into ncw messages with an
associated temporal interval. For example, we might parse message traffic about the radar into new messages such as (Radxstatus Off 0 9), (Radar-status On 10 530), and (Radar-status Off 531 3000).
Interval aggregation. Although we solved a number of problems by parsing time stamps into temporal intervals, we cncountered a further complication in trying to reason with this representation. We needed to ensure that all the temporal intervals we reasoned about were “maximal”; that i s . if two or more intervals for some variable had the same value, they must be aggregated into a singlc interval.lh For example, if the plane’s heading was 80 degrees from time S to time 9 and also 80 degrees from time 10 to time 20, then we could assert that the heading was 80 degrees from time 5 to time 20. Although the initial parse of the simulator trace could easily create maximal intervals for all the simulator inputs, the explanation process might assert new facts into the database that don’t follow this rule. Nonmaximal intervals can cause problems in generating explanations. To handle this problem in the context of explanation-based learning, we use an interval aggregation approach based on repeatedly parsing data. Whenever the explanation generator uses a rule that might possibly generate nonaggregated intervals. the system stops processing the explanation. invokes the parser to collect all the adjacent intervals for which aprcdicate has been asserted. and reformulates them as a single intcrval. Control returns to the explanation generator, and the processing of the explanation continues. This ensures that the explanation generator i s always working with maximally aggregated intervals.
~~
47
name 'Within-elevation
add these macros to the parser, fire all rules and macros in the parser on the initial state, and try again to achieve the original goal with normal backward chaining.
(el-thresh (sensor-id 7sensorl) (elevation-threshold ?el-thresh) (interval 7intl) (flir (flir-elevation 7sensor-el)(interval 7int2) (relative-track-position (track-id ?track-objl) (elevation strack-el) (interval ?int3)) #f(abs-diff-It ?track-el ?sensor-el ?el-thresh) #f(intersect 7 i n t l 71nt27int3 71nt4))))
I
- __ __ Figure 5. Domain theory rule for the Flir scenario. ~~
~
~~~~
~~
~~
This time the goal might be achieved by a fact that was asserted by a new macro in the parser.
~
~~~~~
:name 'Safe-from-Sam-site-macro :type 'non-max '(safe-from-Sam-site (target ?target) (samIAnd , (track-class (track-id ?Sam-id) (object-type 1) (interval ?class-interval)) (track-status (track-id %am-id) (radar-mode track) (interval ?status-interval)) (relative-track-position (track-id ?track-id) (azimuth ?track-az) (interval ?track-az-interval)) (Lisp (abs-value ?track-az ?abs-track-az)) (Lisp (abs-diff-It sabs-track-az 90 7 5))) (Lisp (intersect ?status-interval ?class-interval ?intersect-interval)))) _
_
_
_
_
_
_
_
~
~
Figure 6. Macro associated with the Doppler notch maneuver.
nonmaximal if its antecedents contain vari- pear in the antecedents but not i n the conables (other than interval variables) that do sequent, possibly producing adjacent innot appear i n the consequent. I f a variable tervals that require aggregation. F o r appcars only in the antecedent. it could example, i f the target happens to be at two split the maximal interval for which the different elevations (say I O degrees for the i n t e n d 0-40 and 15 degrees for the interrule's consequent will be true. For example, the Within-elevation rule val 40- I 00), both of' which are within the in Figure 5 is nonmaximal. The first clause threshold distance (say 20 degrees), then in this rule i s the consequent. and the ante- the predicate will be asserted for two adjacedent conditions follow the And conjunc- cent (nonmaximal) intervals. That is, the tion predicate. The goal ofthe rule is to say target will be Within-elevation for the inthat some object in the simulator environ- terval 0-40 and for the interval 40-100. ment ( a "track object") is within elevation instead of for the maximal interval 0-100. of a sensor for some time interval. .?int4. This could cause a problem if there were a The first antecedent condition wys that the query for the target Within-elevation durelevation threshold of some sensor (?xeri- ing the interval 30-50. This query would .so/-/) i s ?i,/-rliri~.s/zfor the interval ?in[/. fail, siuce the subintervals do not individThe next condition says that the elevation ually include the entire interval 30-50. of the forward-looking infrared setisor is .?srri.sor-c.l during ."itif2. Thc third condiI n tCrv(1I tr,q,yregntiori in r.rplanotion get1 eratiori. The explanation generator uses a tion contains the elevation of the track object. Thc fourth condition i\ a call to high-level algorithm to aggregate intervals: Lisp that evaluates the elevations of the ( 1 ) Receive the top-level goal as input and track object and the sensor to see if the perform backward chaining to find rules absolute value of their difference i s within that satisfy subgoals. Do not treat inthe sensor's elevation threshold. Finally, tervals in a special way. the last antecedent condition, another call ( 2 ) I f a nonmaximal rule is found: to a Lisp function, finds the temporal interstop backward chaining, section ofthe first three time intervals and find all explanations that satisfy the returns t h i 3 value as ?itit4. which is the 5ubgoal. generalize and create a macro from time that the consequent holds. This rule is nonmaximal because each explanation.
The r m i m The macro in Figure 6 states that the pilot will be Safe-from-sam-site when there is a track identification (TrackID) o f a SAM missile site (Object-type I ) , the SAM site is in track mode, and the pilot's relative azimuth to the SAM site is close to 90 degrees. The initial learning instance consisted of apilot who used a Doppler notch maneuver the _ to defeat _ ~ SAM ~site. The domain theory was able to explain this maneuver because i t knew that this type of SAM site uses a type ofradar that tracks an object by analyzing changes in its relative distance. The domain theory also knea, that flying a constant distance from an object eliminates any changes in its relative distance, and that a pilot can achieve a constant distance by having arelative azimuthvalue close to 90 degrees (that is, by flying in a circle with the SAM site at its center). W e obtained the macro in Figure 6 by removing the intermediate reasoning steps from the explanation. Only the initial conditions. actions that were performed, and the conclusion remain. Our repeated-parsing control structure required only one modification to EGGS I?: We sometimes combine macros created for nonmaximal rules into macros created for maximal rules.
Creating a plan from the macro The macro in Figure 6 has most of the information the Tactics Planner needs, but its format is far from correct. Implementation of the Tactics Planner has been driven hy the complexities, uncertaintiex, and performance requirements of the domain of tactical planning." The Tactics Planner is built on the Kadet system. a versatile, hyhrid. hierarchical, skeletal-planning system whose conceptual roots go back to Molgen, a system designed for chemical analysis. Kadet has been adapted to ~~~
48
IEEE EXPERT
support reactive planning by continuously reinstantiating skeletal plans and evaluating their applicability. The representation of world knowledge in this framework is highly procedural and involves many implicit assumptions about the planner’s special control structure. Plans are continuously visited using a context-dependent tree-search algorithm. Each plan component has a life cycle that is traversed based on the plan’s applicability to the current context and on the pilot’s approval or guidance. Phases of the plan element life-cycle include creation, specialization, decomposition into subgoals, suggestion (if appropriate), execution (if approved), transition suggestion (if the plan succeeds or fails), and termination (removal from the active planning tree). Thus, the Tactics Planner uses different types of information in different ways. For example, selection rules check for conditions that must be true for a plan to be viable. Execution rules monitor conditions that can change during the life of the plan and recommend that the pilot take actions to achieve adesired state. Figure 7 contains examples of selection and execution rules for a simple Doppler notch plan. The selection rule states that this plan should be selected if there is a SAM site whose radar mode is set to “track.” The execution rule says that if the plane is not flying close to a perpendicular heading from the SAM, a new corrected heading should be suggested to the pilot. To create a plan from the macro, we first partition the macro’s conditions according to the type of rule they will be used in, and then we translate the partitioned conditions into the required syntax.
Partitioning conditions into rule categories. We categorize the macro’s conditions in two ways. First, we categorize them according to whether they represent something that a pilot can control easily. For example, a pilot can easily turn a sensor on or off, but he might not be easily able to lock a sensor onto a track object or make a malfunctioning sensor operational. We also categorize conditions according to whether their arguments are constants or variables. A constant argument is class defining, because its value must hold for any instantiation of the plan. A variable argument must be distinguished by whether it is present in the macro’s
(defrule simple-doppler-notch-selection doc “selection rule documentation for simple-doppler-notch’ local mvar-2-id (second (first (first (assertions ‘(mvar-2 t))))) mvar-2-id-object (find-object-from-sa-object-id mvar-2-id) temps (object-type radar-mode) if (setq radar-mode (prop radar-mode mvar-2-id-object)) (setq object-type (type-of mvar-2-id-object)) (equal object-type ’Sam-site) (equal radar-mode ’track) then (assert (list ’invocation-request (class) parents) goal ((simple-doppler-notch-selection-goal)) thresh 0 0 conf ( 1 0 1 0) calc ‘independent) (defrule simple-doppler-notch-execution doc ”execution rule documentation for simple-doppler-notch’ local mvar-2-id (second (first (first (assertions ’(mvar-2 t))))) mvar-2-id-object (find-object-from-sa-object-id mvar-2-id) temps (927 926 heading) if (setq 926 (nose-off-angle *lead-plane* mar-2-id-object)) (setq 927 (abs (926)) (or (greaterp 927 (t 90 7 5)) (lessp 927 ( - 90 7 5))) then (remove-assert ‘(parameter heading t)) (setq heading (round (perpendicular-heading ’lead-plane’ rnvar-2-id-object))) (assert (list parameter heading heading)) (new- suggest) goal ((simple-doppler-notch-execution-goal)) thresh 0 0 conf ( 1 0 1 0) calc ‘independent) ~
~~
~~
Figure 7. Tacticsmnner rules: (a) a selection rule; (branexecution rule.
postcondition. If so, it is instance defining, because once it is instantiated with a binding, it retains that binding for the life of the plan. If it is not bound in the postcondition, it can be updated during plan execution. We call such a variable updatable. Using these categories, we can automatically sort the macro’s conditions into rule types. For example, execution rules come from conditions that the pilot can easily control and update. Selection rules come from conditions that are not easily controllable, so the plan should be selected only if these conditions are satisfied. If the selection conditions are not satisfied, the plan will have preconditions that are difficult for the pilot to achieve. Updatable or instance-defining variables must have a test condition, that is, a predicate condition, if the condition is to be useful as execution or selection knowledge. Any constant argument will suffice, since it acts as a test condition. Figure 8 shows the partitioning of the Doppler notch macro from Figure 6.
Changing precondition syntax. Once the preconditions have been partitioned into plan indices, LSPA invokes another
-__-
algorithm to rewrite each condition into the syntax used by the Tactics Planner. This is mostly a nontheoretical process of changing syntactic conventions in uninteresting ways. It mainly requires understanding (and in some cases proposing) consistent coding conventions for Tactics Planner rule syntax. For example, our conditions use matching and unification to acquire variable bindings, while the Tactics Planner uses Lisp assignment operators. Thus, we must generate a Lisp assignment statement for each variable in our conditions. There are several exceptions that require more than simple syntactic changes. This is why the output of the LSPA macrotranslator in Figure 3 is a nearly complete plan. In our experience, however, knowledge engineers have found it straightforward to fill in the final nonsyntactic details. The selection and execution rules in Figure 7 were generated by the macro translator from the conditions in Figure 8. These rules have been successfully tested on a flight simulator with the Tactics Planner and the rest of the Pilot’s Associate modules. Because it automatically writes plans, the translator algorithm avoids many of the 49
1
Execution (relative-track-position (track-id ?track-id) (azimuth 7noa)) (fctn (abs-value 71108 7abs-noa)) (pred (abs-difference-less-than sabs-noa 90 7 5))
Transition absence of one of (track-class (track-id ?track-id) (object-type 1)) (track-status (track-id ?track-id) (radar-mode track)) (relative-track-position (track-id ?track-id) (azimuth 7noa)) __________ ~ _ _ __ _ Figure 8. Partitioned conditions from the Doppler notch macro. ~~
common syntax and semantic errors that knowledge engineers typically make during plan development. including siniple syntactic programming errors. partitioning errors by novice knowledge engineers unfamiliar with the plan’s domain theory. and syntactic and semantic style differences between know ledge engineers.
Scope of the domain theory A critical issue for any application of explanation-based learning is the scope and nature of the domain theory required for generating explanations. Our biggest concern during our program‘s first year was the granularity (that is, the depth of reasoning) required to explain an observed tactic. It was not obvious whether we would have to reason at a general level, at levels requiring deep models ofphysics. p5ychology. and physiology. or at some intermediate level. We now believe that the general level i s adequate, and is similar to thc level at which the Pilot’s Associate pilots and Tactics Planner knowledge engineers act ual 1y co mmun ica te . Our domain theory rules cover only a small part o f t h e Pilot‘s Associate domain. However, they do cover much of the relatively self-contained domain for defeating SAM missiles. We believe our domain theory is successful as a feasibility dcmonstration. but scaling our system up to cover the entire Pilot’s Associate domain remains an important open question. If the required domain theory is sufficiently large, then i t might be inore work to 50
~
-
build the domain theory than to build all the tactical plans directly, even if each individual rule is easicr to build. However, this will probably not be the case. since the domain theory is essentially a model of the primitive functionality of the aircraft and its environment. Tactical plans, in contrast, model all possible behaviors arising from the functionality ofthe aircraft and its environment. This i s analogous to a set of axioms and the set of all possible theorems that can be derived from the axioms. The former is typically a finite \et, and the latter is typically infinite. Thuf, it will not only be easier to create an initial Pilot’:, Associate system using our approach: i t will also be much easier to modify and adapt such a system. since we should be able to recompile the same underlying explicit domain theory in different ways to create new plans.
the knowledge bases for a functional Pilot’s Associate will be a critical challenge to thi\ or any similar associate system. Our knowledge compilation solution demonstrates the feasibility of learning leaf plan5 (relatively simple plans that do not reason about plan interactions), which can be executed by the Tactics Planner. We are working on the machine learning of parent plans that could coordinate at least some of the interactions between leaf plans. This will largely involve improving the ability of our macro translator to reason about temporal information i n our domain theory.
We plan to use the Time Map Manager approach for this reasoning.” We are also investigating alternative approaches for acquiring the scenario trace, whose major purpose in an explanationbased-learning framework i s to guide the construction of an explanation. Most of our work has directly processed our flight simulator’s message traffic. But since demonstrating complex flight maneuvers in a simulator can be difficult, expensive, and time-consuming, we are investigating having an interactive planner-a pilot, or apilot and a knowledge engineer-construct explanations directly from the domain theory. We have focused here on one of two major LSPA systems. The other system, the Learning System for Information Requirements, learns the Pilot Vehicle Interface’s information requirements for each plan. This system illustrates the notion that learning for one module of an associate system must be accompanied by learning associated information for the other modules. Future programs must investigate similar sorts of linked learning for the Situation Assessment, Situation Status, and Mission Planning modules, and other parts of the Pilot Vehicle Interface module. Our general approach should be relevant and easily generalizable to other intelligent associate systems, such as a submarine commander’s associate and a helicopter pilot’s associate. For substantially different systems, explanation-based learning should be generally valid as a front end for knowledge compilation. That is, we should be able to use a domain theory and explanation-based learning to identify and generalize the important conditions and features of a learning experience. This will be particularly appropriate in domains where we cannot obtain the large numbers of learning examples required for similarity-basedlearning methods. The macro translator portion o f o u r system is inore tied to this particular application. However. the general approach of using aset of syntactic and semantic principles for partitioning and indexing the conditions in an explanation-based-learning macro might be valid in many contexts. The ability to generalize any indexing procedure should be facilitated by continued work in defining generic tasks, methods, and architectures. These definitions should naturally lead to generic indexing principles.
Acknowledgments This work was supported in part by the Learning Systems Pilot Aiding contract F33615-88C- 1739from the Wright Research and Development Center. We are pleased to acknowledge the support of our contract’s technical monitor, Gurdial Saini. Jerry DeJong of the University of Illinois contributed a great deal to the technical direction of this effort. We are also indebted to the following members of Lockheed’s Pilot’s Associate team: Gary Edwards of ISX, David Smith of Lockheed, Norm Geddes of Applied Systems Intelligence, and Belinda Hoshstrasser of Search Technology.
References 1 . G. DeJong and R. Mooney, “Explanation-
Based Learning: An Alternative View,” Machine Learning, Vol. I , No. 2, 1986, pp. 145-176. 2. T.M. Mitchell, R.M. Keller, and S.T. Kedar-Cabelli, “Explanation-Based Generalization: A Unifying View,”Machine Learning, Vol. I , No. I , 1986, pp. 47-80. 3. A.K. Goel et al., “KnowledgeCompilation: A Symposium,” IEEE Expert, Vol. 6, No. 2, Apr. 1991, pp. 71-93. 4. S. Marcus, ed., Machine Learning, special issue on knowledge acquisition, Vol. 4, No. 3/4, 1990. 5. B. Chandrasekaran, “Generic Tasks in Knowledge-Based Reasoning: High-Level Building Blocks for Expert-System Design,” IEEEExpert, Vol. I , No. 3, 1986, pp. 23-30.
1 1 . S . Minton, “Quantitative Results Concern-
ing the Utility of Explanation-Based Learning,”Artifcial Intelligence, Vol. 42, 1990, pp. 363-392. 12. R.J. Mooney and S.W. Bennett, “A Domain-Independent Explanation-Based Generalizer,” Tech. Report UILU-ENG-862216, AI Research Group, Coordinated Science Laboratory, Univ. of Illinois at Urbana-Champaign, 1986. 13. D.S. Gordon and J.J. Greffenstette, “Expla-
nations of Empirically Derived Reactive Plans,” Proc. Seventh Int’l Con5 Machine Learning, Morgan Kaufmann, San Mateo, Calif., 1990, pp. 198-203. 14. M. Erickson and J . Zytkow, “Utilizing Experience for Improving the Tactical Manager,” Proc. Fifth Int ‘1 Con$ Machine Learning, Morgan Kaufmann, San Mateo, Calif., 1988, pp. 444-450. 15. C.M. Lewis, Identification of Rule-Based Models, doctoral dissertation, Georgia Inst.
of Technology, Atlanta, Ga., 1986. 16. B.C. Williams, “Doing Time: Putting Qualitative Reasoning on Firmer Ground,” P roc. Nat’l Assoc. Art(ficia1 Intelligence (AAAI ’86), MIT Press, Cambridge, Mass., 1986, pp. 105-112. 17. G. Edwards and M. Hoffman, “The Kadet Planning Framework,” DARPA KnowledgeBased-Planning Workshop, 1987, pp. 162 1 , Available from the authors at ISX Corp., 4353 Park Terrace Dr., Westlake Village, CA 91361. 18. T.L. Dean and D.V. McDermott, “Temporal Database Management,” Artifkial Intelligence, Vol. 36, 1987, pp. 1-55.
6. S.B. Banks andC.S. Lizza, “Pilot’s Associ-
ate: A Cooperative, Knowledge-Based System Application,”IEEE Expert, Vol. 6, No. 3, June 1991, pp. 18-29. 7. K.R. Levi et al., “An Analysis of Machine-
Learning Applications for Pilot-Aiding Expert Systems,” Air Force Wright Aeronautical Laboratory Tech. Report TR-871147, Wright Patterson Air Force Base, Ohio, 1987. 8. K.R. Levi, V.L. Shalin, and D.L. Perschbacher, “Automating Acquisition of Plans for an Intelligent Assistant by Observing User Behavior,” Int ’1 J . Man-Machine Studies, Vol. 33, 1990, pp. 489-503. 9. V.L. Shalin et al., “A Formal Analysis of Machine Learning for Knowledge Acquisition,” Int’l J . Man-Machine Studies, Vol. 29, 1988, pp. 429-446. 10. V.L. Shalin et al., “Towards a Theory of
Pilot Information Requirements During Plan Development and Execution,” Proc. Third Int ’1 Conf Human-MachineInteraction and Artificial Intelligence in Aerospace Systems, 1990. Available from the author at
the Dept. of Industrial Eng., State Univ. of New York, Buffalo. JUNE 1992
Keith R. Levi is an assistant professor of computer science at Maharishi International University. Previously, he worked at Honeywell’s Systems and Research Center, where he was the principal investigator for the LSPA program. Levi received his MA in statistics, MS in computer science, and PhD in mathematical psychology from the University of Michigan. His research focuses on machine learning, knowledge acquisition, intelligent tutoring systems, and intelligent interfaces. He is a member of AAAI and ACM. David 1. Perschbacher is a senior research scientist at Alliant Techsystems’ Research and Technology Center. Prior to join Alliant Techsystems, he worked for Honeywell researching machine-learning and planning technolo-
cy. He received his BS in computer science from :he University of Minnesota in 1984. His research interests include machine learning, multiagent coordination, and embedded signal processing. Mark A. Hoffman is a technical program manager at ISX Corp. His research interests include knowledge acquisition and dissemination, reasoning with uncertainty, machine learning, and planning. He received his BS in computer science and analytical mathematics from the University of Washington and his MS in computer science from Rensselaer Polytechnic Institute. Christopher A. Miller is a senior research sci-
entist at Honeywell’s Systems and Research Center. His research focuses on explanation and argument, knowledge acquisition and representation, information management and interface design, learning, and training. He received his MA and PhD in cognition and communication from the University of Chicago. Barry B. Druhan is an associate research scientist at Honeywell’s Systems and Research Center, where he is working as an intern. He is also an MS candidate in systems science and a PhD candidate in cognitive psychology at Louisiana State University. His research is in classifier systems, genetic algorithms, and models of knowledge representation and acquisition.
Valerie 1. S h a h is an asustant profeswr of industrial engineering at the State University of New York at Buffalo She obtained her BS in psychobiology at the University of California at Los Angeles and her MS and PhD in learning, developmental, and cognitive psychology from the University of Pittsburgh. Her research focuses on knowledge elicitation, technical training, and intelligentaiding. She is a member of IEEE, AAAI, the Cognitive Science Society, and the Human Factors Society. The authors can be reached through Keith R. Levi, Computer Science Dept., 1000 N. Fourth St., Maharishi International Univ., Fairfield, IA 52557-1 143. 51
