A Framework for Learning Implicit Expert Knowledge ... - CiteSeerX

A Framework for Learning Implicit Expert Knowledge through Observation

Taha A. Sidani Avelino J. Gonzalez University of Central Florida Electrical and Computer Engineering Dept. Orlando, FL 32816

Abstract There are a number of limiting factors presently constraining the development of a truly intelligent and autonomous machine. The most significant of these is that acquiring expert knowledge continues to be a difficult and time-consuming process. Automated knowledge acquisition techniques have been partially successful in reducing the effort involved in acquiring knowledge from an expert and representing it in a form that can be used by the computer. Most of these techniques, however, focus on the gathering and representation of explicit knowledge. This type of expertise, which includes facts, formulas and rules, makes up most of the expert’s knowledge and is relatively easy to articulate. There is a second type of expertise called implicit knowledge, which includes the more abstract forms of intuition and judgment and is much more difficult to articulate and represent. One way humans learn implicit knowledge is by observing others handle real-life situations and by adapting what we have observed to handle new situations. Most current approaches only address explicit knowledge via query sessions and ignore the implicit expertise altogether. Humans, on the other hand, continually learn and apply both types of knowledge.

Yet, intelligent autonomous systems that can operate in an unfamiliar

environment require the representation of both types of knowledge. Having machines that can reason and behave in a manner similar to a human expert requires formulating an approach for capturing and representing the implicit knowledge that is commonly applied by experts.

This paper describes a framework for automatically learning implicit knowledge by non-intrusionally observing the behavior of a human expert in a simulation of the tasks to be reproduced by the autonomous system. This framework is implemented as a prototype, is rigorously tested and evaluated, and conclusions are derived from the work.

Page 1

1. Introduction

The most commonly used knowledge acquisition approach is to employ a competent knowledge engineer as an intermediate between the expert and the system [1]. A query process is repeated until the amount, the form, and the quality of the knowledge acquired is satisfactory for the current problem. This is a tedious and time-consuming process, which has major drawbacks that limit its effective use [2, 3].

The use of automated knowledge acquisition improves the knowledge engineering process by allowing the expert to interact directly with a computer (query). While it provides an enhancement to the previous method, it still acquires knowledge by asking the expert to verbalize the expertise in some form or another. Even when it is assumed that experts can articulate their knowledge well, the extraction process is restricted to acquiring only expert information that can be easily verbalized. Therefore, the system only learns knowledge that can be represented explicitly. Yet, many actions taken by experts involve the use of implicit knowledge that cannot be easily described in words [4, 5].

Explicit knowledge can be defined as expert knowledge that is easily verbalized and represented symbolically. Declarative information such as facts, theories, rules of thumb and general knowledge from books can take on this symbolic form. Since it is easy for an expert to articulate this knowledge, it is relatively easy to elicit and model. The query session type of knowledge engineering technique is very suitable for the acquisition of this type information.

Various knowledge

representation paradigms such as frames, rules, facts, and semantic nets are typically used to model the explicit expertise of a human.

Implicit knowledge, on the other hand, describes a form of “compiled” knowledge that an expert utilizes while dealing with a situation. It is knowledge that is hidden, implied, intuitive or judgmental. Humans learn to match new situations to prototypical situations in memory and automatically apply the appropriate actions. Dreyfus et al. [6] described a five-stage model of skill acquisition. Their model explains the progression from the beginner stage to the expert stage. A beginner

Page 2

relies on following strict rules relating context-free features. As the beginner begins to understand the domain, he or she starts to examine context-dependent aspects of the domain. Finally after a great deal of experience, the expert simply sees immediately what must be done. No attempt is made to apply the basic rules at every step. This hidden implicit knowledge is considered to be non-articulated, experience-based knowledge. It is used by experts to perform a task or solve a problem in an intuitive manner. The actual representations used by experts to model this knowledge are complex and poorly understood. It is for this reason that implicit knowledge is very difficult to acquire and represent.

Experts utilize explicit and implicit situational knowledge when facing real-life situations.

Slatter emphasized the

importance of differentiating between the types of expertise used by an expert while handling a situation [5].

“... the key distinction is between explicit and implicit knowledge ... It is in the nature of implicit knowledge that it is extremely difficult to capture from human experts ... implicit models are rather pessimistic about what can be achieved in knowledge engineering. The lack of existing techniques for capturing implicit knowledge is placing major constraints on expert systems...” [5, pp. 143-144].

The importance of capturing implicit knowledge is further alluded to by other researchers. For instance, Dreyfus [7] discussed how previous attempts to use only rules and symbolic representations failed to produce general intelligence.

The significance of the use of intuition in our everyday decision making can easily be observed by examining our normal daily driving habits.

We build an internal feeling about a situation and act on our intuition when choosing the proper

action. This fact is applied when faced with a typical situation of having to pass another vehicle. The driver assesses the situation and evaluates the threats imposed by other vehicles. The presence and absence of vehicles on the two-lane road is taken into consideration. More importantly, the driver does not calculate the speed of the oncoming vehicles but rather, relies on intuitive judgment about the dynamics of the vehicles involved as well as their capabilities. Using previous experiences, the driver learns to map situations into actions. If asked to describe the decision making process that was used to handle the situation, most experts could not articulate their reasoning process. It is very difficult for experts to explain

Page 3

their intuitive judgment by verbally describing what, when, how and why an action to pass another vehicle was taken. One common reply: “it felt like the right thing to do at the time.”

Case-based Reasoning (CBR) [8], an alternative to rule-based inference, attempts to eliminate or significantly reduce the knowledge engineering process by using historical examples of previously solved problems. The basic idea behind CBR is that by finding a previously solved problem that closely resembles the current problem, the solution used for the former could also be successfully applied to the latter. CBR uses a database of historical problems and their solutions in lieu of the knowledge compiled in an expert’s experience-based heuristics.

The advantages are the reduced knowledge engineering

effort, and the elimination of humans (the expert as well as the knowledge engineer) as the interpreters of the domain knowledge. Presumably, if the description of the historical cases includes the implicit knowledge, then this knowledge is captured and able to be used to solve new problems

However, it is not always easy to determine similarity between the current case and the historical cases, nor know what to do when they are not sufficiently similar. Another deficiency with CBR is that the case library (as the database is more commonly referred to) cannot easily capture all the knowledge related to the historical cases. While an expert ensures that all the relevant knowledge is captured and represented in his heuristics, the case library is left to depend on what aspects of the historical cases were documented when they occurred. This hampers the collection and representation of implicit knowledge. And even if all the necessary knowledge was captured in the cases, how to represent the implicit knowledge still remains an open question. CBR helps in identifying similar cases, but it does little to capture and represent implicit knowledge.

2. Learning Through Observation of an Expert in a Simulation

The lack of acquisition and representation of implicit expertise hinders our ability to model intelligent behavior in the computer. Capturing the true implicit situational knowledge of the expert requires an intelligent system that possesses a sense of awareness about the current situation. It must be capable of sensing the presence and absence of objects,

Page 4

hypothesizing about the current state, and monitoring and learning how the expert remedies the situation.

The

procurement and representation of implicit situational knowledge requires a new approach to knowledge acquisition. In many cases, it is more appropriate to allow the expert to demonstrate the expertise rather than try to verbalize it, and therefore learn the implicit expertise by observing the expert.

Humans rely heavily on learning by watching someone else handle new or difficult situations. Sometimes we ask specific questions to narrow down what object(s) or feature(s) are important to consider at the time.

But in many instances, it is

unnecessary to ask why a certain action was taken, as we are able build cause-effect relationships merely by examining the changes in both, the environment and expert actions.

The expert observes, reasons, and then acts, while the learner observes, reasons, and learns. Replacing the learner by a computer that learns equally well requires that the computer have a sense of awareness of the current situation. The most intensive actions taken by both the human expert and the learner involve sensing the changes in the environment in order to form a cause-and-effect relationship.

Fusing meaningful information from multiple sources in the real world is

recognized as a very difficult task.

Thurman et al’s concept of incremental automation [9] introduces the concept of an incrementally more proficient automation software agent that learns on the job, both by observing an expert perform the task, and by asking the same expert when it is confused. It uses the OFMspert system [10] (as referenced in [9]) as its foundation, which, in turn, makes use of blackboard architecture to represent its knowledge.

As a way of efficiently capturing implicit knowledge, it is our intent to build a computer that can learn by unobtrusively observing an expert carry out a task or solve a problem. However, it is clear that the current technology is far from being capable of building a machine that compares favorably with human abilities. Thus, in teaching the computer to learn by observation, we must first alleviate the problems associated with dealing with the real world environment. To that effect, we rely on a simulation to facilitate this task. A simulation is used to model the domain and generate a scenario that requires the expert to apply his/her implicit knowledge. The simulated environment provides the expert with a medium for

Page 5

conveying the implicit knowledge to an intelligent system that is capable of capturing it through observation. The next section describes the proposed framework and technique to do this.

3. General Approach

The methodology to be described here poses the following question:

How does one implement learning by observation using a simulation such that implicit knowledge can be acquired, represented, and reused?

The answer will result in an efficient framework that gathers, represents, and learns expert knowledge by monitoring the expert's actions in a given situation, and being aware of the external environment he faces.

This technique assumes that

the changes in the environment can be identified, and the expert actions are externally noticeable as well as measurable. A behavior or decision-making process that is largely internal and unobservable, or not easily measurable, is not applicable to the techniques being proposed here.

Capturing knowledge from the expert involves learning sequential or time-varying patterns incrementally via observation. Neural networks have proven their capabilities in learning sequence recognition, sequence reproduction and temporal association [11]. Temporal association represents the most general case where both sequence recognition and reproduction is produced. Demonstrating the implicit knowledge through performance creates a complex dynamic system that can be captured in a neural network by using the currently sensed input patterns as well as the historical data which are represented by its past input and output patterns (i.e., past events). Therefore, our approach is partially based on neural networks.

But neural networks alone, operating in an unstructured manner, do not promise to solve the problem unilaterally. Complex domains such as those seen in real life would require too large a network to be practical. We propose to build a

Page 6

supporting structure in the form of a hybrid system that symbolically determines the situation in which the expert finds himself, and applies a neural network (out of several) that correctly addresses the specific situation.

Crowe [12] used an unstructured neural network to learn air combat maneuvers through observation of a pilot of an F-16 simulator in a combat exercise. His ARCADE neural network system is based on a back propagation architecture, and employs a hidden layer of 15 nodes. The inputs are in terms of angles, ranges and velocities, while the outputs are the lateral and axial acceleration commands to the aircraft. However, rather than train it in observation of a human, it was set to observe the performance of a Lead Pursuit/Intercept algorithm over a wide variety of starting geometries.

While the

author declares the project a success, no empirical data are provided to justify that contention.

Pomerleau et al [13] also used an unstructured neural network to learn by observing a vehicle following the road (as driven by a human), and then control the vehicle itself at a later time. The interesting aspects of this project, called ALVINN, is its use of actual video images, rather than derived terrain features, and that it used a real vehicle, rather than a simulation. However, the task being duplicated was not heavily cognitive in nature, as the requirement was that the vehicle remain on the road. Baluja [14] provides a more detailed description of the ALVINN system.

Our approach is to develop a Situational Awareness Module (SAM - see Figure 1) to provide a hybrid structure and therefore handle the complex reasoning required to associate expert actions to the current environmental state. It senses the changing environmental stimuli and monitors the expert’s reaction to this change. The main responsibilities of this module are two-fold: clarifying the current state; and learning the implicit expert actions and skills necessary to remedy the situation. It modularizes implicit knowledge as a finite group of skills that can be observed and learned using neural networks. Furthermore, it applies high level symbolic knowledge to generalize and replay what is learned. Previous experiences and current knowledge in its knowledge base are used to formulate new situational knowledge. Its knowledge base is augmented with easily accessible explicit knowledge provided by a knowledge engineer or other sources of inputs such as books (e.g., driver's education book). The explicit knowledge is used to help the system generalize the learned actions to handle more complex situations not previously encountered.

Page 7

Figure 1 - A general framework for capturing the implicit expertise

SAM operates in three distinct modes: Data Collection, Knowledge Formulation and Knowledge Utilization & Testing. These three modes of operation are responsible for gathering, representing, testing and using the expert implicit knowledge via learning by observation. In the data collection mode, data are collected from the simulation and placed in a format to be used for learning. This mode can be thought of as the observation mode, and is done in real time. The knowledge formulation mode is where the learning takes place. The data collected are presented to the system and the neural networks are trained. This mode can be done off line. The knowledge utilization mode represents allowing the newly trained system to perform the same task or solve the same (but somewhat different) problem as the expert did in the observation pahse. It is in this mode that the proficiency of the system can be evaluated. These modes generally take place in the same temporal order in which they were described above. They will be discussed in greater detail in section 4 below.

Page 8

4. Implementation of Approach – the SAM System

In her paper [15], Bainbridge states that new concepts are required in order to appropriately model the human execution of complex dynamic tasks. She specifically calls for situation-sensitive models that more accurately describe how humans actually perform these tasks. Our formulated approach defines a methodology for modularizing the implicit primitive expert knowledge as a finite group of skills that are situation-related. It is composed of two parts: 1) the modeling of low level functions that represent implicit knowledge, and 2) the symbolically represented high level cognitive process that determines the situation in which the expert finds him/herself. Our approach focuses on the first part. The skills related to the low-level functions can be observed and learned using neural networks.

Furthermore, a global symbolic reasoner

applies high level symbolic knowledge for assessing the current overall situation. This reasoner permits the learned knowledge units to handle new compound situations not previously encountered. The formulated ideas are intended to be generic, so that one may apply the same concepts to capture and represent knowledge from different simulations.

4.1 Definition of Essential Variables and Attributes Regardless of the simulation domain, there are a number of steps that must be taken to prepare SAM for learning by observation. These include: • • •

Identification of the Control Entity(ies) Identification of the Control Variables Selection of the attributes that should be monitored

A control entity is the set of mechanisms that permit an expert to manipulate the state of the current environment based on the changing dynamics of his/her present surroundings. The selected control entity used by the expert depends heavily on the simulation domain. In a driving simulation the control entity represents the car or vehicle used by the expert to travel from one location to another. It must be easily identifiable. Selection of the appropriate control entity(ies) is domain dependent and is left to the expert to make the final decision.

Page 9

The second step involves choosing the appropriate Control Variables that are associated with the chosen control entity. Control variables describe the various features available for expert manipulation. Control entities of different types would most likely involve different sets of control variables. The control variables available for each type of control entity must be identified a priori. Examples of control variables in the driving domain include acceleration, braking, and steering. During the data collection and learning modes, the human expert controls the control entity by manipulating the available control variables. Similarly, during the testing mode, the system must apply what it learned to autonomously control the entity while dealing with the various encountered situations. The control variables must be defined by the expert depending on the control entity and the current simulated domain.

In addition to the continuous monitoring of expert actions as described by the pre-defined control variables, the changes in the simulated environment must also be sensed. Hence, the third preparation step involves defining what features must be considered at the environment and the object level. It is evident that the ability of humans to draw out what is important in any situation is dependent on their previous experiences and the amount of explicit knowledge they have about the applied domain. Experts warrant their actions on the information obtained from a few critical features that summarize the current situation. The selection of these critical features affects the degree and manner by which an action is invoked. The important features identified by two human experts encountering the same set of objects and the same situation might be different depending on their background knowledge.

The attributes that must be monitored are Environmental and Object-related. Environmental attributes describe the current state of the environment (i.e., the road is wet, it is cloudy, or it is dark). Object attributes are assumed to define the object characteristics considered important. For example, in a driving simulation, it would be important to consider the size, speed, and heading of a car (other than the control vehicle), whereas it would only be necessary to examine the distance and the light state of any traffic light. It is the responsibility of the developer to identify the important attributes that must be monitored at all times during the current situation. Static and dynamic object databases are used to hold the various objects that will be activated in a training session. Each object identifies the important attributes about itself that must be monitored during a session. The use of object-oriented techniques in the simulation helps shift the burden of knowing what to monitor onto each object. Each object is responsible for continuously monitoring its own state and storing a history.

Page 10

Figure 2 illustrates a pre-defined abstraction of a hierarchical class structure that could be used to describe the available objects in a driving simulation. Each class encapsulates the features that must be monitored at that level. For instance, it is important to monitor the position of any object; therefore the position features would be included in the PARENT-OBJ class. A TRAFFIC-LIGHT class illustrates an example of a multi-states object where color indicates the current state. A DYNAMIC-OBJ must include features that describe the rate of travel and direction.

PARENT-OBJ

DYNAMIC-OBJ

INANIMATE

AUTOLIKE

STATIC-OBJ

SINGLESTATE

ANIMATE

ANIMAL

DOG

HUMAN

SPEED LIMIT

MULTISTATE

STOP SIGN

TRAFFICLIGHT

PEDESTRIAN

Figure 2 - A hierarchical objects structure.

Selecting the appropriate attributes to monitor is considered one of the most crucial steps. The dynamically changing states of the chosen attributes form the input sequence that is used to train the neural network. The neural network learns to map the sequence of patterns in the input domain to the sequence of expert actions in the output domain.

Page 11

4.2 Data Collection Process - Observation The objective of this mode is to accumulate the data that describe the current situation and the expert’s reaction to it. The resulting data are pre-processed and saved for use in the learning mode that involves training an Artificial Neural Network (ANN). The human expert plays an active role during data collection while SAM acts as a passive observer. A set of Critical Training Events must be defined which embody the situation in which we want to place the expert and observe and learn his behavior or reaction. Each event must be created in the simulation of choice, initialized and executed for use by the expert.

4.2.1 Critical Training Events. The selection of the appropriate training events is an important step since it heavily influences the system's ability to generalize learned knowledge to new situations. It is highly unrealistic to expect the expert to be presented with all the possible variations that could occur in a situation. For that reason, the system should be trained on basic concepts that warrant generalization.

In defining the Critical Training Events, we must first define the set of basic skills that form the primitive knowledge in the current domain. The set of basic skills is then used to generate the training events. For example, in the driving domain, the basic skills might involve accelerating, decelerating, maintaining speed, and steering. The chosen critical training events might include handling a speed limit sign, a traffic light in various states, a pedestrian walking parallel to the vehicle, a pedestrian crossing the street, passing another vehicle, etc.

A sequence of data must be collected for each session that the expert experiences. A session is defined as one expert run through the situation defined by the current active event. The number of sessions that must be presented to the expert for each active event is a factor of the length, the complexity, and the variability of the training event. For instance, a training event which is set up to capture how the expert handles a stop sign involves less variability than learning to react to a traffic light event. A traffic light event requires multiple sessions to capture the correct timing that the expert uses to deal with its various states (e.g., green, yellow, and red). The goal in collecting the proper data is to ensure that the sessions cover the most likely expert actions as well as the range in which these actions apply. In a red traffic light event, the sessions must

Page 12

capture the earliest and latest times the expert stops before the light. The maximum and minimum stopping distances from the traffic light define the acceptable expert boundaries (upper/lower bounds) for successful stopping.

4.2.2

Sampling and Preprocessing the Data.

The Data Sampler and Pre-processor components of SAM identify critical points in time and select appropriate sampling rates to reduce the collected data without losing important information about objects and time relationships. The runtime data generated by the expert actions, the environment and the active objects are sampled at a fixed rate. The resulting session data file is further preprocessed to train specialized neural networks (i.e., recurrent networks) to learn to mimic the expert behavior. The preprocessing of the data sequence depends on the type of neural network used, the number of monitored features, and the range of values generated by the simulation. There are no fixed methods for training a neural network. A few rules of thumb, when followed, can increase the efficiency and the accuracy of the training process.

Sometimes running one session is not sufficient for learning certain events (i.e., handling a yellow traffic light). Here multiple sessions are required to capture the general behavior displayed by the expert. The goal in collecting the proper data is to ensure that the sessions cover the most likely expert actions as well as the range in which these actions apply. In a red traffic light event, the sessions must capture the earliest and latest times the expert stops before the light. The maximum and minimum stopping distances from the traffic light define the acceptable expert boundaries (upper/lower bounds) for successful stopping.

4.3 Knowledge Formulation Architecture - Learning This mode is comprised of ANN customization and training, and knowledge encapsulation. The framework creates and trains individual neural networks to handle each basic skill. This section describes the architecture of the neural networks used.

Encoding dynamic behavior in some neural network architectures allows for the handling spatio-temporal patterns. Delays were introduced into some networks in order to make them responsive to the time-varying signals. Having sensitivity to time-varying signals makes recurrent networks ideal for learning the dynamic behavior of an expert. It was also shown that

Page 13

there are many different types of recurrent networks. The framework guides the developer into selecting the appropriate ANN architecture based on the constraints, limitations and demands of the problem. Using a functional approach to classify recurrent networks used in modeling dynamic systems, Horne [16] describes two forms: the nonlinear recursive form, and the nonlinear state space form.

The nature of the problem in implicit learning lends itself to the state space form of recurrent neural networks. The nonlinear state space form builds memory of past events by forming a set of internal states that represent the context units [16]. These units represent a summary of the accumulation of state information from previous input/output patterns. They are used in conjunction with the current input to predict future behaviors. The dynamic equations describing the state space form are defined as:

s(t + 1)  O(t )  =  

f

sm

 s(t )       I (t )

where O(t) is the output vector at time t. I(t) is the input vector at time t. S(t) is the state vector at time t. fsm is the learned input/output mapping function.

Consequently, we use a modified version of the Elman [17] network which responds to the same inputs differently at different times depending on the previous inputs. The network possesses long term memory of past events in the context units by feeding back the internal states defined at the previous step, (see Figure 3).

The figure shows two slabs (cluster of neurons) in the input layer. The first slab represents the current input vector I(t) of size n at time t. The second input slab is a copy of the nodes in the hidden layer. Hence, the capacitance vector of the context slab, C(t), is of the same size, h, as the hidden layer.

At first, the input vector, along with an initial random set of context unit values, are fed forward to the hidden layer. The hidden layer is said to summarize the incoming input sequence by creating its own internal representation. At the same time that the internal states defined by the hidden activations, Hi(t), are fed forward to the output layer, a copy of the same

Page 14

activations (representing recent memory of the internal states) is fed back to the context slab in the input layer. The output of the network is then compared with the desired expert actions and an error term is calculated.

The network

backpropagates if the error exceeded the pre-defined threshold. Backpropagation involves traversing back to the input layer and adjusting the weights connected to the links between the layers. The change in weights represents the learning process of the network. Unlike the network defined by Elman [17], the network shown in Figure 3 allows the weights between the context slab and the hidden layer to be modified. The number of past events being considered at every step is a factor of the summation in the context units. The context units provide the network with the capability to “see” features that were detected in the past, possibly all the way back to the start of the sequence.

Input Vector 1

Context-units n-1

2

n

1

Alpha h-1

2

h

Input Layer

1

h-1

2

h

1-Alpha

Hidden Layer

1

2

r-1

r

Output Layer

Figure 3 - Partially recurrent neural network for learning temporal association

The capacitance activations at time t+1 are reflections of some percentage, Alpha, of the capacitance at time t plus a corresponding percentage of the hidden layer activations at time t. This is shown by the following equation.

Page 15

C (t + 1) i

= α Ci ( t ) + (1 - α ) H i (t )

Where, Ci(t), capacitance activation at time t for neuron i, Hi(t), hidden activation at time t for neuron i, α, percentage of feedback

As evident by the context units update rule, its view of the past is dampened and the degree to which the past detected features diminish is dependent on the value of Alpha.

The alpha term defines the amount of past history that is

remembered from one time step to another. During the next time step, the network augments the internal states from the previous step with the inputs at the current time step. In essence, at the next time step, the network is examining the current inputs as well as the previous states that summarize previous input/output patterns.

neural net NNKU-ANIMAL-CROSSING Name : Descriptor: Trained_net: Object_type: Object_instance: Monitored_features: Object_link: Obj_rules: Expl_Rules: Num_Inputs: Num_Outputs:

object rules

cat

event rules -------------------

Figure 4 . An NNKU Event Object

Page 16

-------------------

Each skill learned by the system is modeled as an individual knowledge unit called a Neural Network Knowledge Unit (NNKU). High level symbolic knowledge is linked to each knowledge unit in order to increase its level of generalization and use. A modular design is implemented that encapsulates the trained neural network with its corresponding explicit information. Object-oriented techniques are used in defining the NNKU’s. Pre-defined attributes and behaviors are added to each NNKU, (see Figure 4).

The importance of gathering and maintaining background and explicit information at the event and at the object levels is necessary for making decisions concerning the generalization of learned events. All of the expert’s basic skills are modeled into individual NNKU’s and stored in the Neural Network Knowledge Base (NNKB) for later use in the KnowledgeUtilization and Testing mode.

4.4 Knowledge-Utilization and Testing Mode

The testing phase focuses on developing the needed mechanisms by which the system (i.e., the computer) can apply the formulated knowledge and generalize that knowledge to handle new complex situations.

A symbolic reasoner was

implemented in SAM to give it the capability to replay learned knowledge and to add new knowledge. Three components are defined in this reasoner to handle the selection of applicable NNKU’s, the activation of these knowledge units, and the evaluation and selection of the proper actions based on the examination of the overall current picture, (see Figure 5). They are the Event Recognizer, the Event Activator, and the Action Resolver, respectively. In this mode of operation, the system is given full control over the control entity and variables during the current situation.

Initially, the system loads the latest NNKB as well as all the general explicit background knowledge for the current testing domain. The knowledge engineer selects and triggers one or more testing events in the simulation. When this testing event is activated, a Scene Analyzer (step 1) monitors the simulation for the activation of dynamic and static objects. The recognizer (step 2) then defines the type of event(s) that are taking place in the simulation and searches the NNKB for one or more applicable NNKU(s). The chosen NNKU(s) become activated (step 3) and get added to the list of current active

Page 17

events. The general explicit knowledge (step 4) associated with each active NNKU is loaded into memory. Each activated NNKU is given the responsibility to focus solely on its own set of objects (step 5). The result is a set of outputs generated by all active NNKU’s to deal with their corresponding detailed picture (step 6). However the actions resulting from each active NNKU may not agree (one NNKU might indicate that the system should increase speed while another NNKU requires it to decelerate). Resolving this conflict in actions is done by examining the overall picture and the relationships between objects (step 7). Once the proper action is chosen, the vehicle control variables are adjusted (step 8) and the process repeats.

>>>Dissertation figure p. 181