A Conceptual Foundation for Autonomous Learning in ... - CiteSeerX

A Conceptual Foundation for Autonomous Learning in Unforeseen Situations Catriona M. Kennedy Technical Report Arti cial Intelligence Institute Dresden Universtity of Technology 01062 Dresden, Germany e-mail: [email protected]

Abstract

A cognitive agent should have the capability to learn autonomously in completely unforeseen situations. "Unforeseen" means something that was not taken into account in an internal representation of the world (i.e. it is "impossible"). However it is detectable in the form of unusual sensor measurements. Two problems must be solved: rst, the "newness" of the situation must be detected, i.e. it should not be allocated (wrongly) to an existing category. Secondly, new concepts must be learned so that when a similar situation occurs again it is no longer anomalous. A conceptual framework is presented here based on a form of symbol grounding which emphasises a continual distinction between model-driven expectancy and actual reality. Large dierences between expectation and reality indicate that a new concept is required which corresponds more accurately to the sensor data. This results in the autonomous growth and change of symbol groundings. Genetic programming is considered as a tool (both on the symbolic and the subsymbolic levels).

keywords: anomaly, anticipation, cognition, symbol grounding

1 Introduction This paper addresses the problem of cognition and symbol grounding from the viewpoint of anomaly detection and autonomous concept generation. In particular, how does an agent detect the "newness" of an unforeseen situation which was not explicitly taken into account within its design? Such a situation would appear as an "anomaly" in sensor measurements which contradicts expected values determined by a world model (which may contain e.g. a theory of actions and/or causality). Then the agent should detect that its model is no longer sucient. Once this has been detected, a degree of adaptation and concept-generation should take place so that the original goal can be satis ed if this is still physically possible. This paper presents a conceptual framework based on the idea of "anticipatory agents" (see e.g. Ekdahl et.al. [7]). However, our approach is dierent in that an agent uses anticipations in order to detect their invalidity when this is appropriate, i.e. theories are there to be tested against reality. Anticipations are model-driven internal simulations of an action and its eects on the world. Whereas normally this takes place on the deliberative level to calculate the bene ts of performing an action in reality, our approach includes some anticipation on the reactive level so that postconditions of simulated actions are continually compared with the eects of the same action in reality. If there is a signi cant dierence, an anomaly is detected and the deliberative component is activated to search for new concepts. Although the 1

2 1 2

3

4 ........................8

1

Anomalous result of PICKUP :

2 3

1

4 :

2

Initial state

Predicted state

Actual state:

1

1

1

1

0

0

0

0

0

0

0

0

1

1

1

1

0

0

0

0

0

0

0

0

:

1

1

1

1

0

0

0

0

1

1

1

1

8

1

1

1

1

0

0

0

0

1

1

1

1

:

3

Figure 1: Cracked block anomaly framework for concept-generation presented here is still rudimentary, it provides an answer to a question posed by Davidsson [3]: "how should an agent decide when to create a new concept?". Furthermore, our approach extends the idea of "symbol grounding" introduced by Harnad [9]. Our use of the term "grounding" not only includes a currently known concept acquired through previous physical interaction but also "disturbances" to the concept, i.e. the "grounding" for a symbol is a continual interaction between model-driven expectancy and reality. For example, the symbol "block" may be associated with applicable actions (e.g. stacking, throwing) and predicted eects of actions. If a discovery of a new kind of block is triggered by an anomaly, the grounding for the symbol is extended by adding a new context in which the newly discovered "meaning" applies (e.g. a block which is unusually heavy). In this way the new meaning is autonomously generated.

1.1 Why do we need Anomaly Detection?

When considering unpredictable environments, it is normal to make the system less brittle by including strategies for recognizing similarities between a new situation and prede ned situations within a model. Examples are case-based reasoning (e.g. Ram et.al. [17]) and fuzzy rules. Probability-oriented approaches do not dier signi cantly because their decisions are also con ned to choices oered by the model. No account is made for the failure of the model itself. Yet it is precisely those failures that fuel adaptation and innovation in nature. This has been argued in detail by McMullin, [15]). Moreover, there are cetain kinds of anomalies for which a "fuzzy" or "similarity" approach will give poor results. If we take the example of the blocks world, an anomaly would occur if a block were to break in two when lifted (because it may have been cracked for example). An extremely simpli ed 2-dimensional picture of this is shown in gure 1. Assume blocks have indices 1, 2, 3, .. and that each is visible using a kind of low-resolution scanning, with 0 indicating absence of a block and 1 indicating its presence. The gure shows the appearance of the pixels in two dimensions. Each pixel value corresponds to a sensor value. If we consider location 3, the sensors scanning that location have dierent values from those expected. This particular anomaly is not only unexpected but it also involves changes in sensor measurements that will not t any prede ned condition according to the model, since it is assumed that blocks always remain whole. The precondition of an action Pickup(1) may be described symbolically as At(1; 3) which translates to the pixel values shown for that location. The expected postcondition is the absence of a block at location 3 (pixels are all 0). The

3 actual postcondition in gure 1 neither maps onto At(1; 3) nor :At(1; 3). It may be allocated "approximately" to one or the other using fuzzy or similarity-based methods. However, this is wrong!. What needs to be detected is the fact that the new situation does not correspond to any of the symbolic conditions. Other examples of anomalies are nding that the block cannot be put down because the top surface is sticky or nding that it is far too heavy to be lifted. With the "traditional AI" approach, the world model would have to be made suciently complex to take into account all of these situations. In many real-world environments this is unrealistic, not only for reasons of complexity, but also because some environments may be largely unknown to a human user (e.g. analysis of Mars atmosphere). Although "universal" models of physical causality, space, time etc. may often be useful, they have some disadvantages. First, it is surely preferable that a symbolic ontology should contain only those high-level categories that are thought to be relevant to the problem to be solved (with the possibility of automated revision or re nement later) instead of attempting to include "everything" about the physical world from the start. Secondly, such universal physical models will certainly not be sucient where the environment is "virtual" (software agents) or social (socially situated agents, whose environment consists of humans and/or other arti cial agents). Anomalies in a virtual environment could include, for example, irregularities in a database (possibly indicating fraud) or software irregularities (which may indicate a new kind of virus). Similarly, in a social environment, another agent or human user may act or communicate in a way that is not accounted for in any model. The problem is then to detect the discrepancy with all existing models including those representing classes of "abnormal" behaviour.

1.2 Why do we need a Symbolic Representation at all?

It could be argued that we are trying to solve an arti cial problem which is caused by our symbolic representation. Brooks [1] has argued that an internal symbolic model is not necessary ("The world is its own best model"). Although some behaviour-based AI architectures have produced good results, it will be assumed here that an internal model is desirable for goal-seeking agents. One reason for this is increased transparency. These and other arguments are already in the literature (e.g. Wooldridge and Jennings [22]). We will therefore assume that some sort of hybrid layered architecture is needed.

1.3 Concepts and Terminology

We will use the term "symbol" here in the sense of something that helps humans to concentrate on the problem to be solved, e.g. transparency and "nearness" to human language. The term "subsymbolic" will be used for anything that does not t into a typically human way of describing the world (e.g. bit strings, sensor-readings etc.) and may have many dierent physical realizations (e.g. neural networks). However, this de nition is not always exactly the same as that in the literature, e.g. Steels [20] de nes "subsymbolic" as a dynamical system. Since we will assume here that the hardware has been hand-crafted according to a discrete categorization, this issue does not appear to be signi cant. Only if we are considering more radical methods such as evolvable hardware does the distinction between the two de nitions become relevant. There is no deep or mathematical distinction between "symbolic" and "subsymbolic" as used here. In the second part of this paper we use genetic programming (Koza [13]) as an explorative tool for anomaly-detection and learning because it can evolve structures which can t into either of these two levels.

4 In our approach, symbol groundings for symbols on the higher level are patterns (anticipations) on the subsymbolic level together with external physical eects on sensor-values (i.e. internal concept + external disturbance). Furthermore, we use the terms "reactive" and "deliberative" independently of symbolic and subsymbolic. Having de ned the terminology, anomaly-driven learning can be divided into two subproblems: 1. what changes should take place in the subsymbolic layer? In particular: (a) how to detect the anomaly? (b) how should new behaviour patterns be learned as a result? 2. what changes (if any) should take place on the symbolic level?

2 A Suggested Architecture We now propose an architecture and provisional algorithms which can be used as a conceptual framework. The blocks world will be used throughout as an illustrative example. The architecture can be divided into three main components, the world model (including symbolic and subsymbolic aspects), a reactive component and a deliberative component.

2.1 World Model

We will rst consider more precisely what is meant by a "world model" and what its relationship is with hardware sensors and eectors. Three levels may be de ned as follows: L: set of symbolic ("linguistic") components I : set of subsymbolic ("implementation") components H : set of hardware components (sensors and eectors) The relationships between the rst two layers are shown schematically in gure 2. Each component will be explained in detail in the next three subsections which will continually refer to this diagram.

2.1.1 Symbolic Level

We will assume that L is partitioned into the following four subsets: LT : a set of s object types (classes). In our blocks world example, we will assume they are the de nitions of two predicates fBlock(x) ! p1(x); Location(x) ! p2 (x)g, where p1 and p2

are the lists of properties for blocks and locations repectively. For our purposes we do not need to consider the details of those properties. LO : a set of r object instances to be acted on fo1; o2; :::; org For our example, each oi is a statement associating an object instance with its class fBlock(b1); Block(b2); Block(b3); Location(l1); Location(l2); Location(l3)g.

LA: a set of n actions fa1; a2; :::; ang. In our example, the actions are: Pickup(x): where x is any block, Move(x; y; z): move block x from location y to location z, and PutDown(x). LC : a set of m possible conditions (situations) fc1; c2; :::; cmg. For our example, the conditions are as follows:

5 O

L :B1

T

L :BLOCK

Properties

A

Predictions

L :PICKUP

L I

A

Data structure

T

f1

f2

T

Block class I 1

T f2

A

Meaning of actions in I 1

Passive desc.

p1

m1

Active desc.

p2

m2

pn

mn

Context 1

A

f2

Context 2

Figure 2: A world model showing symbolic and subsymbolic levels

On(x; y): block x is on blocky Holding(x; y): robot is holding block x over location y At(x; y): block x is at location y Empty(x): location x is empty.

The rst two sets contain statements; the second two contain atoms. These constructs are shown in the upper half of gure 2. We will assume that actions are de ned according to their eects, i.e. there is a function do : LA  LC ! LC which is typical in classical planning systems. For example do(Move(b1; 2; 3); Holding (b1; 2)) = Holding (b1; 3). In gure 2 this is shown in the box labelled "predictions" associated with an action.

2.1.2 Hardware Sensors and Eectors E: a set of available hardwired eectors (assume they are discretely de ned) fe1; e2; :::g S: a set of available sensors (attributes) fh1; h2; :::g For each sensor si there is a set of possible values: Vi : fvi1 ; vi2 ; ::::g.

A state is any list of attribute-value pairs. The attributes appearing in the list are a subset of S . The value vij associated with si must be a member of Vi.

2.1.3 Subsymbolic Level

This level contains subsymbolic "implementations" of constructs on the symbolic level: I T : a set of subsymbolic concepts fo1 ; o2; :::g where each concept is a "meaning" for a symbolic object-class. In gure 2 it is divided into a passive and an active description. We are interested primarily in the active description. I C : a set of low-level states (attribute-value lists) fs1 ; s2; :::g. Each state implements a symbolic condition. Each attribute can have a "legal" range of values (like a fuzzy interval). I A : a set of pairs f(m1; p1); (m2; p2):::g. Each pair is composed of a motor schema and

6 an "expectancy" associated with it. mi is a sequence of eector activations implementing the symbolic-level action ai . (Similar to "microcode"). The associated pi is the expectancy (prediction) for mi and is analogous to that on the symbolic level, only that preconditions and postconditions are hardware states as de ned above, i.e. it has the form pi : I C ! I C . We will assume for the moment that all sensors are included in the states. A low-level expectancy can have a varying "resolution". The lowest resolution is to make a prediction about the result of the whole action, which can be re ned by making predictions about fragments of the motor schema. The highest resolution would involve predicting the result of every single eector activation (predictions are then maximally interwoven with the eector sequence). For higher resolutions, an element (mi ; pi) becomes a list of motor schema fragments and associated expectancies. We will assume for simplicity that the initial I A is compiled from a high-level symbolic model as with e.g. "situated automata" (Kaelbing [12]). In particular, the initial expectancies may simply be a compilation of the symbolically de ned predictions for the action. Then the low-level expectancy will not have a ner resolution than the high-level one. It is not necessary to have an I O since its elements will be no more than indexed names which point to their respective classes in I T which already exists on the symbolic layer. For example, the name b1 in LO could point to two dierent components: the symbolic user knowledge about it and the subsymbolic concept acquired through the agent's own physical interaction.

2.1.4 Translation from Symbolic to Subsymbolic Levels If we assume that there is complete symbol grounding, i.e. every relation, function and constant symbol is "visible" to the subsymbolic layer, then there is a translation from each subset LT ; LA ; LC of L to the corresponding subsets of I . We will consider each component in detail: First, actions themselves have a "meaning":

f A : LA ! I A For example, the symbol Pickup will translate to a sequence of eector actions with its associated expectancy. Names for conditions such as On also map onto low-level sensor-states:

f C : LC ! I C Object class names are associated with subsymbolic concepts approximately satisfying the speci cation in gure 2. They are de ned "actively" in terms of manipulations on the object and their eects. f T : LT ! I T For example, the symbol Block will translate into the structure in gure 2 which point to a passive and an active component. The passive component is expressed in terms of attribute combinations (e.g. admissible pixel-shadings, weight-range) and is initially a direct compilation of the symbolic-level properties asscociated with a block. The active component points to applicable motor schemas and expectancies (i.e. some elements of I A ).

2.1.5 Learning

The kind of learning required is the generation of dierent versions of the components of I . A new version should be created every time an anomaly is encountered. This is like generating a new context. For example, what it means to Pickup a block or what it means for a block

7 to be On another could change if a new kind of block is discovered. One such additional context is shown in gure 2. It may also be appropriate to generate new versions of all actions independently of any object class. For example, if an anomalous state is discovered which is a general property of the environment (e.g. the ground is everywhere very slippery) then most actions (moving, picking up and putting down) must be carried out slowly and carefully even if nothing anomalous was found about a particular object. In this paper we will concentrate on only new kinds of objects, mainly from the point of view of dierent actions which are applicable, (i.e. the "active description" component of gure 2). For the moment we will not consider "passive" features de ned in a data structure such as expected weight etc. This means that for newly created contexts, the box labelled "data structure" will remain a xed compilation of the hand-crafted "properties" on the symbolic level. However, the active description may become very dierent from the initially compiled version in context 1. It is possible for Pickup to be translated into a completely new motor schema and associated expectancy. Moreover, it is also possible to discover new kinds of applicable actions (i.e. actions with an eect) that were not included on the symbolic level at all.

2.1.6 Translation from Subsymbolic to Symbolic Levels There are two situations: 1. No changes take place on the symbolic level. For example, the robot may have discovered many dierent ways of picking up a block but all of these methods are still called by the one name, i.e. Pickup. Then for each new version generated, there is a translation fiA : LA ! IiA A change from one version to another is then similar to a context change, i.e. the semantics of Pickup changes when an instance of an amomalous class is operated on and then changed back again when actions on "normal" objects are resumed. 2. Changes are required on the symbolic level, e.g. when an explanation of a new form of behaviour is requested. In this case, a new version of L is also produced. The user may then observe the new action and assign a name to it, e.g. Pickup ? carefully . Alternatively the agent itself may produce a temporary name such as Pickup2 to indicate that a new concept has been discovered. The association between the two names is then an example of mediation.

2.1.7 Reactivity and Deliberation In contrast with other architectures which include anticipation, for example Edkahl et.al. [7], Davidsson [5], in our architecture anticipation is interwoven with reactive execution. This is necessary if we wish to have anomaly detection on the reactive level. The degree of interwovenness depends on the resolution of the expectancy as de ned above. In the blocks world example, deliberation can be limited to the following situations: (1) the deliberative component operates in "normal" mode on the symbolic level only, when a search is made for a good plan to solve a given problem, (2) it may be activated in an anomaly situation when a default expectancy is contradicted. Then it must have access to the subsymbolic level. The architecture is shown in gure 3. The component labeled "world model" in gure 3 contains everything in gure 2 (i.e. symbolic representations with their continually growing semantic "groundings"). The most important distinction in the diagram is that between simulation (model-driven expectancy)

8 Simulation Evaluate

predicted

D

effect world

Activate

model

current state

Modify

Action

R

Action Sensors

Effectors

Reality

Figure 3: Complete agent architecture and reality (sensors and eectors). Simulation simply involves calculating the expected postcondition according to the prediction function. The reactive component R executes motor schemas rst as simulation and then by activating physical sensors and eectors. This may seem strange at rst because R does not reason about anticipated results or evaluate them in any way. (This is done by the deliberative component D). Anticipation on the reactive level is only necessary so that anything that contradicts it can be detected. Furthermore R may use anticipation with a much lower "resolution" than D; it is simply a tradeo between eciency and sensitivity to anomalies.

2.2 Towards Implementation 2.2.1 Software Tools

Koza's Genetic programming (GP) [13] can be considered initially as a tool for the required anomaly detection and concept discovery. It is not only general, but also has the property that it can be used as a subsymbolic paradigm (e.g. wall-following) and as a form of concept learning on the symbolic level (e.g. genetic logic programming, Wong, [21]). It will be assumed here that massively parallel on-line evolution for realistic problems is a future possibility with GP. For current work on this see e.g. Juiles et. al. [11] for massively parallel GP, and Steels [20] for on-line evolution. It will also be assumed here that data structures can be evolved (a detailed report is given in Langdon [14]). The translations f A and f C can be implemented using GP. When the tness of a single genetic program (which we will indicate in lower-case gp) is evaluated, the gp (which is like a genome) is translated into its phenotype; i.e. each symbol encountered (whether action or condition) is "expanded" into a user-de ned micro-procedure (assuming it is a non-trivial function). Each of these micro-procedures may themselves be evolved as genetic programs.

9

2.2.2 How is an Anomaly Detected? We will now consider how to use GP to nd a plan and an anomaly occurs during this search. A primary goal in the blocks world could be the stacking of blocks in one corner of the room. If we assume rst that the environment contains no anomalies, this means the micro-structure of each high-level action symbol can be used reliably. Then the main problem is simply to nd a good plan which will achieve the goal state or get maximally close to it. This sort of problem has already been solved with GP (see e.g. Handley, [8]). However, if we wish to be prepared for anomalies, tness evaluation becomes more complex because we are dealing with both simulation and reality, whereas normally the tness evaluation would be concerned with only one of these (depending on whether the evolution is "oine" or "online"). The tness value relates to the eectiveness of the step, i.e. whether it brings the state nearer to the goal or not (it will be assumed for simplicity that each step of a good plan should do this, although it may not be a valid assumption for all problems). We will not consider the tness of a prediction at this stage, although this may be signi cant for later phases. A suggested algorithm for the anomaly-detecting FitnessEvaluation procedure is shown in table 1(a). It rst calls the prediction function with the current precondition (i.e. the action is simulated). If the predicted state is closer to the goal state than the current one, the plan step seems good and it is executed for real, i.e. hardware eectors are activated which have actual side eects. The new sensor values are then compared with expected values. If there is a signi cant dierence, an anomaly has been found. We will assume that initially the paramater "signi cant" is prede ned. If we de ne "non-signi cant" dierences as an interval within which sensor-values can vary and still correspond to the expected state (e.g. up to two pixels can have the "wrong" values), then a "signi cant" dierence is anything falling outside of this. In particular, it can mean one of two things: 1. The anomalous state is within the interval of an admissible state which is not the expected one. e.g. actual sensor-values for postcondition of Pickup(1) in precondition At(1; 3) ^ :Holding(1) correspond much more closely to At(1; 3) ^ :Holding(1) than they do to the expected state :At(1; 3) ^ Holding (1). 2. the anomalous state is outside the intervals of all admissible states (as in gure 1). The algorithm handles both these situations implicitly. Anomaly-detection is also possible during execution of a plan. An algorithm is shown in table 1(b). With the exception of the anomaly-handling components, these algorithms are based on standard genetic programming and traditional planning.

2.3 Towards Concept Generation

An architecture for concept generation could be built in several phases. The rst, which we will describe here in detail, is simply to generate a new version of I A only, and for each (mi ; pi), generate only a new mi . i.e. a new concept is created where motor-schemas are dierent but the prediction function remains the same, e.g. the micro-procedure specifying how to pick up up a block will change, but everything else remains constant. This means that the prediction remains a compilation of the symbolic-level prediction. Once an amomaly is detected, the deliberative component is activated. In the example of the broken block, a subgoal must be generated (pick up block successfully) and a search is made for a "micro"-plan which satis es the subgoal. This will then become the new "meaning" of Pickup. GP can now be used on the subsymbolic level. The only practical problem appears to be the need for considerably sets of elementary building blocks (function and

10

procedure EvaluateFitness(PLAN)

Local variables: CurrentState = InitialState; fitness = 0; PredictedState, ActualState, D1, D2, D; for each action ai in PLAN do PredictedState Execute(ai , MODEL);//Simulate the action D1 Dierence(PredictedState, GoalState); if D1 < Dierence(CurrentState, GoalState); //is it a good idea? ActualState Execute(ai, WORLD);//Execute it for real D2 Dierence(PredictedState, ActualState); if D2 > Signi cant then ANOMALY (ai); else

D

D ? D 1 ? D2 ; CurrentState ActualState;

end if; end if; else return end if; end for;

fitness = 0; //Assume that plans with useless steps are bad

return fitness MaxFitness ? D; end EvaluateFitness; procedure ExecutePlan(PLAN)

Local variables: CurrentState InitialState; PredictedState, ActualState, D; for each action ai in PLAN do PredictedState Execute(ai, MODEL); ActualState Execute(ai , WORLD); D Dierence(PredictedState, ActualState); if D > Signi cant then ANOMALY (ai); else

CurrentState

end if; end for; end ExecutePlan;

ActualState;

Table 1: Continual Testing of a Simulation against Reality: (a) during tness evaluation of candidate plans; (b) during normal execution of a plan

11 terminal sets) from which to generate motor schemas, since they must include all hardware elements, i.e. S and E . This problem will be discussed again later (with respect to attention focus). procedure Anomaly

(Action) Generate a population of gp's; for each generation do for each motor schema mi in the population do for each tness case (test precondition)) do PredictedState Execute(mi , MODEL); //Simulate the action D1 Dierence(PredictedState, GoalState); if D1 < Dierence(CurrentState, GoalState); //is it a good idea? ActualState Execute(mi, WORLD);//Execute it for real D2 Dierence(PredictedState, ActualState); D3 Dierence(ActualState, GoalState); Fitness(mi ; pj ) MaxFitness ? D2 ? D3; produce new precondition; end for; // each tness case Calculate AverageFitness(mi ); end for; // each motor schema produce next generation; end for; // each generation end;

Table 2: Search for an alterative motor-schema Assuming that a successful motor schema is found, a new object class may be created for the anomalous block. For phase 1, the only change is that of the "applicable operation" labeled Pickup. This will now point to a new motor-schema (with associated expectancy) and the current block index will point to the new object class (see gure 2).

3 Discussion and Future Work

3.1 Limitations

On the whole, the algorithm in table 2 is still very rudimentary as a concept-generation strategy since the only "concept" generated is the new behaviour pattern. It is quite possible that motor schemas will be found which are predicted to satisfy the subgoal but (just as the old motor schema did) none of them produce a good result in reality. On the other hand, if successful, we have simply found a motor schema which causes the environment to behave as predicted. An example of this is where one surface of a block is slippery and this surface is used while picking up a block normally, resulting in the anomaly that the block cannot be picked up as expected. A microplan could be found where other surfaces are used instead, and the predictions about it need not change. However, this is a blind approach and very inecient. It is like trying to solve a problem by avoiding diculties instead of investigating them. It is certainly much more eective to identify the signi cant features of an anomaly

12 and to focus attention on them. Similarly, the resolution of predictions could be temporarily increased. As soon as we have attention focus, active exploration becomes necessary. Otherwise anomaly-detection capability is reduced, since the restricted subset of sensors currently attended to may not be "disturbed" in an anomalous situation. Moreover, to discover a new object/background distinction, it is clear that practical experimentation with the candidate object is required.

4 Related Work

4.1 Symbol Grounding and Autonomous Concept Acquisition

The entities in the three layers in our architecture seem to correspond approximately to Harnad's three kinds of representations: iconic (sensor values), categorical (subsymbolic concepts for object classes) and symbolic (formal strings). However, while Harnad emphasises the dierence between analog and "categorical" perception, our architecture emphasises the continual distinction between something which is under an agent's control (simulation) and something which is not (sensory impressions from outside). Harnad's distinctions are probably also signi cant. The grounding of logic is another very important issue which could not be considered here but has been investigated by Prem, [16]. While Harnad concentrates only on connectionist netwoks, Davidsson ([3] and [4]) proposes a more general framework. It may be said of existing approaches that the autonomy in the growth and change of the groundings through learning is still very restricted (although Davidsson does mention the problem of when to decide to create a new concept). This is the problem that our framework speci cally addresses, since it is concerned with autonomous detection of anomalies and generation of entirely new classes as a result.

4.2 Anomaly Detection

De Giacomo et.al. [6] use logic-based methods to detect anomalies and recover from them by replanning. Their approach is similar to ours in that it emphasises the distinction between model and reality. However, since it does not include symbol grounding, it does not seem to have potential for autonomous concept-generation. An interesting alternative approach to anomaly-detection is that based on the immune system. Examples are Dasgupta and Forrest [2] and Ishida [10]. The idea is to develop a capability to distinguish between "self" and "non-self", for example, using a "negative selection algorithm", where cells which react to "self" in a controlled environment are eliminated and those which are inactive are left intact, leaving them to react to non-self (i.e. any novel situation) when they become exposed to a "real" environment. There are similarities here to the anticipation approach, since one can say that the internal model along with the simulation based on it corresponds to "self" (as there are no external eects outside the agent's control). Those external eects which are signi cantly dierent from "self" are then detected as anomalies. However, immune system models do not take into account anomalies which may be helpful (all anomalies are removed indiscrimminately) whereas the anticipation approach has the potential to respond to help or opportunity.

References [1] Brooks, Rodney A. (1986) "A Robust Layered Control System for a Mobile Robot" in IEEE Journal of Robotics and Automation, vol. RA-2 (1), pages 14-23.

13 [2] Dasgupta,D., Forrest,S. (1997) "Novelty-Detection in Time Series Data using Ideas from Immunology" in Proceedings of the International Conference on Intelligent Systems, 1997 [3] Davidsson, Paul (1993) "Towards a General Solution to the Symbol Grounding Problem: Combining Machine Learning and Computer Vision" in AAAI-Press Fall Symposium Sreies, Machine Learning in Computer Vision: What, Why and How? AAAI Press, 1993. [4] Davidsson, Paul (1993) "A Framework for Organization and Representation of Concept Knowledge in Autonomous Agents" in Proceedings of the Scandinavian Conference on Arti cial Intelligence, IOS Press, pages 183-192. [5] Davidsson, Paul (1996) "A Linearly Quasi-Anticipatory Autonomous Agent Architecture: Some Preliminary Experiments" in C. Zhang and D. Lukose (eds.) Distributed Arti cial Intelligence: Architectures and Modeling Springer, pages 189303 [6] De Giacomo, G., Reiter, R., Soutschanski, M. (1998) "Execution Monitoring of High-Level Robot Programs" in Proceedings Common Sense 98. [7] Ekdahl, B., Astor, E. and Davidsson, P. (1995) "Towards Anticipatory Agents" in Wooldridge M., Jennings, N. (eds.) Intelligent Agents - Theories, Architectures and Languages, Springer, pages 191-202. [8] Handley, S. (1994) "The Automatic Generations of Plans for a Mobile Robot via Genetic Programming with Automatically De ned Functions" in Advances in Genetic Programming, edited by Kenneth E. Kinnear, MIT Press. [9] Harnad, Stevan (1990) "The Symbol Grounding Problem", Physica D 42 pages 335-346. [10] Ishida, Yoshiteru (1996) "The Immune System as a Self-Identi cation Process: a Survey and a Proposal", International Workshop on Immunity-Based Systems 1996, held in conjunction with ICMAS 96. [11] Juille,H., Pollack,J.B. (1996) "Massively Parallel Genetic Programming" in Advances in Genetic Programming 2, edited by Peter J. Angeline and Kenneth E. Kinnear , MIT Press. [12] Kaelbing, L. (1991) "A Situated Automata approach to the Design of Embedded Agents" in SIGART Bulletin 2(4): pages 85-88. [13] Koza, John R. Genetic Programming: On the Programming of Computers by Natural Selection, MIT Press, Cambridge MA. [14] Langdon, Bill (1996) "Genetic Programming and Data Structures", PhD Thesis, University College, London. [15] McMullin, Barry (1992) "Ari cial Knowledge", PhD Thesis, Dublin City University. [16] Prem, Erich (1994) "Symbol Grounding", PhD Thesis, Technical University of Vienna (in German). Also available under: ftp://ftp.ai.univie.ac.at/papers/Thesis/

14 [17] Ram,A., Arkin,R.C., Moorman,R.J., Clark,R.J. (1992) "Case-based Reactive Navigation: A case-based method for on-line selection and adaptation of reactive control parameters in autonomous robotic systems" Technical Report GIT-CC-92/57, College of Computing, Georgia Institute of Technology, Georgia, 1992. [18] Rosen, Robert (1985) Anticipatory Systems, Pergamon Press, Oxford, UK. [19] Rosen, Robert (1991) Life Itself, Columbia Univerity Press, New York, Complexity in Ecological Systems Series, 1991. [20] Steels, Luc (1994) "Emergent Functionality through On-line Evolution" in: Proceedings: Alife IV, MIT Press. [21] Wong, M.L., Leung, K.S. (1995) "Inducing Logic Programs with Genetic Algorithms: The Genetic Logic Programming System" in IEEE Expert 10, no. 5. pages 68-76. [22] Wooldridge, M., Jennings, N. (1995) "Intelligent Agents: Theory and Practice" in Knowledge Engineering Review 10, no. 2. June 1995.