An Instructor's Assistant for Team-Training in dynamic multi-agent ...

An Instructor's Assistant for Team-Training in dynamic multi-agent virtual worlds. Stacy C. Marsella & W. Lewis Johnson May 6, 1998 Longer version of paper to be presented at ITS-98

Abstract

Teams of people operating in highly dynamic, multi-agent environments must learn to deal with rapid and unpredictable turns of events. Simulation-based training environments inhabited by synthetic agents can be eective in providing realistic but safe settings in which to develop the skills these environments require. However such training environments present a problem for the instructor who must evaluate and control rapidly evolving training sessions. We address the instructors' problem with a pedagogical agent called the PuppetMaster. The PuppetMaster manages a network of spy agents that report on the activities in the simulation in order to provide the instructor with an interpretation and situation-speci c analysis of student behavior. The approach used to model student teams is to structure the state space into an abstract situation-based model of behavior that supports interpretation in the face of missing information about agent's actions and goals.

1 Introduction Virtual worlds inhabited by synthetic agents are increasingly being used for training and education (e.g., [12, 7, 16]). These virtual worlds can provide a highly engaging environment in which to develop skills

1

that students can readily apply to real world tasks. In creating these environments, considerable eort is often expended so that the simulated world more faithfully mirrors the real world in which the acquired skill must be employed. This concern for the delity of the training simulation has a long standing tradition (e.g., [4]). Training teams of people to operate in dynamic multi-agent settings presents special challenges, however. The concern for delity has led to training environments that are in turn very dynamic. To faithfully capture the unpredictable multi-agent setting, simulations can be inhabited by multitudes of synthetic agents which ideally exhibit the same kind of complex behaviors that human participants would exhibit. For example, Distributed Interactive Simulation (DIS) training sessions involve teams of human students interacting with potentially thousands of synthetic and human agents within a very dynamic battle eld simulation (e.g., [16]). From an instructor's perspective, the use of very dynamic multiagent virtual environments raises several concerns. Among these is the seemingly simple question of what the student teams are currently doing. In a complex simulated world, conditions in the simulation can be dicult to determine. For instance, information from any one student's perception of events may be unavailable, or an agent (human or synthetic) may not be able to explain their own motives. Furthermore, abstracting from the behavior of individuals to the behavior and goals of teams requires additional eort. Conversely, there may be too much low level information for the instructor to absorb and interpret. And as the number of interacting agents grows, the diculty increases. Accordingly, it may be dicult to determine what teams are doing and why they are doing it. Furthermore, determining how well they are doing and analyzing trends in their behavior are also dicult. This is especially true in a world where interaction is not tightly scripted. Students may undergo unique experiences for which there is no set response against which they can be assessed. We address the instructor's problem with a synthetic pedagogical agent, the PuppetMaster, which provides a high-level interpretation and assessment of the students. As the teacher's intermediary, the PuppetMaster dynamically assigns intelligent probes that monitor student teams and synthetic agents in the simulation. The situation in the virtual world is then assessed from the perspective of high level training objectives. The assessment includes events, trends and ag-

2

gregate reporting over multiple entities (e.g., for revealing teamwork). The resulting evaluation is then used to compose 2-D and 3-D presentations that reduce the instructor's eort during the exercise and assists the instructor's review with students after the exercise. In a very dynamic simulation, student teams are often learning to operate in both goal-directed and reactive fashions. Therefore, it is particularly critical that the PuppetMaster appropriately model the students' loosely scripted interactions with the world, in the face of changing goals, partial information and irrelevant information. This presents a problem for approaches to pedagogical agents and intelligent tutoring that attempt to model students based on detailed plan matching (e.g., [12]), model tracing (e.g., [1]) or plan recognition (e.g., [5]) based on a step-by-step analysis of actions. Arguably, such approaches, even if plausible for the multi-agent dynamic domain we are investigating, are often irrelevant for the instructional support that is needed. To model student teams, we have adapted a model from reactive planning research, the Situation Space [14, 9]. Roughly, a situation space is a structuring of states of the world into classes of problem solving histories whereby an agent's recognition of its current situation guides its goal-directed behavior. The PuppetMaster models both reactive and goal driven behaviors within the framework of a situation space model of the student team. The current situation provides a top-down focus for monitoring, inferring missing information and assessing behavioral trends, based on appropriateness for the students' situation.

2 The Need The domain in which we explored the design of our pedagogical agent is virtual world simulation for the training of military tank crews. These simulations are based on DIS technology whereby independent simulators are tied together over a network, communicating low level simulation events to create a large-scale simulation environment. The various entities interacting in this simulation environment can include teams of students in tank simulators as well as synthetic forces (semiautonomous forces) generated by software such as the ModSAF (Modular Semi-Autonomous Forces) program [2].

3

In a typical platoon exercise, there are four tank simulators, each manned by four students, and approximately fty synthetic forces. Additionally, a team of Army instructors will manage the exercise, performing the roles of superior ocer (e.g., sending out changes in orders), teacher (e.g., providing guidance after the exercise during a review session) and adversary (e.g, dynamically creating and tasking opposing forces). An exercise for a tank platoon might involve traveling in wedge formation to a location in order to occupy a position which blocks the opposing force. While in a wedge formation, there are guidelines as to how the individual tanks follow each other, keep each other in sight, etc. As the platoon travels to the position along the virtual terrain, it may encounter friendly or opposing forces to which it has to respond appropriately. These forces can be either synthetic or human agents and encounters with them can rapidly unfold in unpredictable ways. Likewise, the platoon may encounter terrain features that can slow down and/or damage its vehicles. Thus, the multi-agent dynamic qualities of the simulation can rapidly impact what needs to be done to satisfy the broadly scripted exercise, as well as whether it can be satis ed. During the exercise, the instructors manage the simulation system in order to extract information and modify ongoing events, often to make the exercise more challenging. After the exercise, instructors will review key events for the students on 3-D displays. Based on our observations of these exercises, instructors do not micromanage or even micro-evaluate the students. For instance, they don't analyze student behavior on an action-by-action basis unless the eect of a student's actions is something entirely anomalous in the current situation. An example of the latter would be if a tank comes to a halt while the platoon is supposed to be in a traveling wedge, thus disrupting the wedge. Finally, there are also individual dierences in how instructors evaluate speci c skills in the training context. These characteristics are consistent with the nature of the domain and the skills that need to be acquired. Because of the dynamic, multiagent domain, there is no guaranteed plan for achieving the goals set out for the students. Without such a plan, or other agreed upon objective basis, micro-evaluation at the level of each and every individual action is problematic. Furthermore, understanding the rationale for

4

every action may require modeling the domain from the perspective of every student, e.g., down to the level of every rock which must be circumnavigated to avoid tread damage. Also, because of the dynamic domain, it is necessary to foster the development of initiative in the teams consistent with situation-appropriate goals and behaviors. Although instructors do an excellent job of managing training exercises, the tools that they have at their disposal are severely lacking. Instructors are often forced to write notes to themselves on paper as the exercise proceeds, and must rely on subjective impressions. Also, as an exercise unfolds over time, relevant information concerning transient events or trends may be missed. To see the obstacles that instructors currently face, consider Figure 1 which presents a 3D virtual view that an instructor might have of a platoon in a training exercise. One can see that the tanks are in a particular formation (it is a wedge formation). Things that are hard to tell from this picture are:  How well have they been maintaining their formation over time? That requires an analysis of the formation over time, but only when the formation is appropriate. A formation that is appropriate when traveling may be inappropriate when engaging the enemy.  What is preventing them from maintaining good formation? Is it a problem in maintaining visual contact, diculty navigating the terrain, or problems with the vehicles?  Are they in a position to spot the enemy?  Has the enemy spotted them? The information necessary to answer these questions is present in the simulation, if one looks for it in the right place and the right time. The goal of our pedagogical agent is to extract the information from the simulation automatically and to analyze it from the perspective of the instructor, permitting the instructor to focus on deciding where to provide instructional or adversarial feedback.

3 The Probes Architecture In order to understand how the PuppetMaster works, let us brie y lay out the overall architecture in which it works, the Probes system.

5

Figure 1: An instructor's view of a platoon traveling in a wedge. The architecture of Probes is presented in Figure 2. The simulation exercise analysis is guided by an Exercise Task Description, which describes the mission objectives and instructional objectives. The mission objectives describe the students' goals and what is known about current conditions. The instructional objectives provide a breakdown of the skills that need to be analyzed during particular phases/situations in the exercise. The exercise task description is used to generate a situation space. The PuppetMaster uses the situation space to determine what information is required to recognize and analyze a situation. Monitoring instructions are sent to the Articulate Adaptive Monitors that are embedded in the instrumented ModSAF entities. Depending on their instructions, these local monitors collect the data requested by the PuppetMaster and report back, either on a regular basis, or when interesting events occur, or in response to queries from the PuppetMaster. The Puppet Master uses the data that it receives to interpret and assess the situation. Output to the instructor is controlled by the Display Manager, which in turn sends commands to the display software. The situation space and its role in the PuppetMaster's evaluation is central to this paper. It is this aspect that we consider next.

6

Figure 2: The Probes architecture.

4 Situation Spaces As developed in reactive planning research, Situation Spaces structure the states of the world into situations and links between situations. The behavior of the agent is organized around its current situation which determines both the (sub)goal(s) it should try to achieve (or maintain) and how to monitor the world. In turn, monitoring determines whether traversal to a new situation has occurred. Traversals could be caused by successful achievement of a (sub)goal, as well as fortuitous or bad turn of events. Therefore, a traversal could be advantageous or dis-advantageous in terms of achieving the overall goal. Traversals through the situation space represent alternative abstract partial decomposition plans which can incorporate both purely goal-directed behavior as well as goal-directed responses to unforeseen events. To make these ideas more concrete, consider the situation space depicted in Figure 3 for the simpli ed example problem presented earlier. Recall the problem was to travel in a wedge formation to a blocking position, responding to encounters with the opposing force en route. The current situation at the start of the exercise is assumed to be \Traveling" (Wedge Formation). As noted above, the current sit-

7

Action on Contact

Contact?

Arrived?

At Goal Location

Traveling Wedge Formation Disabled?

Situation

Disabled Transition

Figure 3: Example Situation Space. uation determines the goals to achieve. When a platoon is in this Traveling situation, their goals include traveling along some route towards the blocking position, and the maintenance goals of maintaining a wedge formation and scanning for the enemy. As they pursue these goals, various kinds of expected or un-expected transitions can occur which must be monitored for while in this situation. For instance, the students must monitor for reaching their objective, noted as the transition to the Goal situation. Also, they must monitor for contact with other forces. This could cause a transition to an \Action on Contact". If that transition occurs, the goals in this new situation would include assessing the threat, targeting opposing forces and insuring one is not a good target. (In actuality, actions on contact encompass dierent kinds of responses and one might expand this situation into several situations in order to explicitly capture these dierences.) Again as noted by the transition arcs, when in the Action on Contact situation, transitions to \Disabled" or back to \Traveling" must be monitored. Even from this simple example, several characteristics about situation spaces can be noted. There are multiple such paths through the space, in fact, an in nite number since looping is possible (between traveling and action on contact in this example). This compactly expresses the dynamic characteristics in these training environments whereby students can undergo repeated and unexpected encounters. The situation space models the necessary reactivity within an overall goal-directed declarative plan. At the same time, the situation space does not separately represent each possible plan { to do so would be implausible. Rather, the situation space models sets of possible plans at a high level, with each situation modeling many possible problem solving histories. Finally, the situation space is noncommittal con-

8

cerning how the plan generation occurs within a situation or even what planning architecture should be used. Rather, it is a framework in which goal-directed behavior can be planned for and realized in a dynamic world. Of course, our interest in situation spaces does not follow from an intent to use them for single agent planning. Rather, we saw situation spaces as a good declarative model around which to organize a pedagogical agent's analysis of team behavior in a dynamic world. As a consequence, we needed to transform them from an aid for planning to an aid for analysis of team behavior.

4.1 Situation Spaces from an Instructor's perspective

We made two moves to use Situation Spaces to analyze Team behavior. First, we transformed the goals indicated by a situation from goalsto-be-achieved to behaviors-to-be-analyzed. For instance, the goal associated with traveling in a wedge, along a route to the blocking position, becomes an analysis goal to determine how well the team is traveling. Similarly, the action on contact goals of \targeting" and \avoid being an easy target" become how well they are targeting and avoid being targeted. The other move we made was to allow for multiple perspectives. This move is necessary because the team of students may have a different perspective from the pedagogical agent as to what the current situation is, and such discrepancies provide key pedagogical assessments. For instance, the students may transition to an action on contact situation and start ring at what they think is the opposition, but actually are distant rocks. Meanwhile, the pedagogical agent, due to its larger perspective, will know that the transition to action on contact was a mistake. Conversely, the pedagogical agent may identify a viable enemy threat which the students have not seen, perhaps due to a failure in their scanning for the enemy. A consequence of maintaining multiple perspectives is that, for those transitions that could result in a bifurcation of perspective, there needs to be a transition arc for each perspective. So for the transition from the traveling to the action on contact situations there need to be 2 arcs, one for the pedagogical perspective and one for the team perspective. (Figure 3 shows only one of these arcs.)

9

Another consequence of maintaining multiple perspectives is that we need to determine which situation is the appropriate basis for monitoring and reporting analyses. Currently, we have found it sucient to report analyses based on the team perspective, for several reasons. Reporting an analysis of the team from a situational perspective dierent from that guiding their behavior tends to generate marginally useful information. In contrast, analyzing team behavior from their perspective generates the information that is needed to infer the point at which they nally transition into the \correct" situation. For instance, when the team is traveling in a wedge, there are certain behaviors they are supposed to exhibit such as constantly scanning their turrets and following each other at a certain distance. These behaviors are not typically appropriate during an active engagement so their cessation is a key indicator for inferring the transition of the team perspective. On the other hand, the pedagogical perspective is key in analyzing potential failures of the team in assessing their situation. It has also been our experience that the bifurcation into two perspectives persists only for very short periods of time. There are several reasons for this. The situation space is at a very high level of abstraction so a bifurcation tends to indicate a dramatic misread of the environment. In addition, given the pressure that the domain exerts on the team's behavior, such a misread is likely not to persist for long. For example, incoming rounds are a solid indicator to the team that those \rocks" o in the distance are actually the opposition ring at them. Allowing for these two perspectives has proved sucient to date in characterizing the state of a platoon exercise. However, we could go further. Recall there are four tanks in a platoon. One might break the team perspective into individual tank perspectives, thus having the pedagogical agent track what it considers to be each tank's perspective on the current situation. Now discrepancies between tank perspectives re ect factors such as breakdown in communication or coordination. This move towards multiple perspectives assumes that the perspectives share the same situation space. However, if there were multiple platoons being assessed, performing dierent missions, then dierent situation spaces would be more appropriate. Conversely, we may model a single agents overall behavior as being composed of distinct situations in multiple situation spaces. These points of course beg the question of composition over situations which would need further

10

study.

4.2 Situation Spaces in Probes

Probes uses the situation space to control its monitoring, form its assessment of the team and selectively report that assessment. In Probes, a situation space is de ned as a set of situations, with each situation being a 3-tuple consisting of:  List of transition predicate functions which might re-set the current situation and thereby cause a transition out of the situation  Set of evaluation functions for forming an assessment  Initialization function which invokes the monitors pertinent to the situation and performs any necessary initializations. The situation space discussed here was designed as follows. The Army's instructional material laid out the mission goals, the kinds of situations (our term) that occur in pursuing the exercise goals along with the subset of behaviors that need to be assessed in those situations. Reinforced by our observations of actual training sessions, this material essentially provided the structure of the situation space at a high level: the types of situations, as well as the nature of the transitions and evaluations associated with those situations. From there, the actual transition, evaluation and initialization functions were de ned, roughly in that order. For instance, the transition from the \Travel" to \Action on Contact" situations required Probes to determine the conditions under which transitions could be assumed for both team perspective and pedagogical perspectives. If the information to make these determinations was unavailable or hidden, then a means of inferring the transition was developed, in particular by detecting changes in behavioral trends. Next, evaluation functions were de ned. For instance, the instructional material and our observations revealed the importance of evaluating travel formation, as well as how it was assessed in doctrine and in practice. Additionally, a limited set of data from actual exercises was available that included instructor evaluations on wedge formations. That data was run through a system for inducing oblique decision trees [10] and the resulting decision tree was used as an additional source of information, in order to objectively characterize what the instructors' considered important. Based on these various sources, we

11

created an evaluation function that assessed the key characteristics of a wedge and re ned it by testing against ModSAF tank platoons. To delimit the information being presented to instructors, the reporting of the assessment was restricted to when those characteristics qualitatively changed. Given the information used by the transition and evaluation functions, the initialization functions and associated monitors could be de ned. The various monitors used by Probes were built into ModSAF and invoked via a software interface. As examples, periodic monitors would report the team's location, event driven monitors would report when the team had been spotted by an opposing team and asynchronous monitors were used to assess intervisibility between vehicles. Across all the exercises for platoon training, the instructional material was laid out in a modular fashion that shared exercise goals, situations and assessment criteria. This suggests that the application of situation spaces across the full suite of platoon exercises could be semi-automated.

4.3 Situation-based vs Event-based Modeling

Several characteristics of this training domain lead us to structure assessment around a situation space. Due to the dynamic, unpredictable nature of the domain, there are no set plans for students to learn which could form the basis for student evaluation. Nevertheless, some structuring is necessary to ascertain, for instance, whether students are getting closer to their goals. Further, it was necessary to deal with the volume of information and to account for missing information. By structuring the state space into a situation space, monitoring is organized top-down around the situation, which in turn usefully constrains interpretation and assessment. Behavioral trends can be monitored and assessed according to their appropriateness within a situation. Changes in behavioral trends within a situation can be used to infer missing information, such as whether the platoon has spotted the enemy and is going into action on contact. Unnecessary details about the state of the world are not monitored. And the transitions between situations provide a principled basis of coupling analysis of planned and reactive behaviors. A key feature of our approach is that the high level analysis ap-

12

propriate to the situation is based mainly on partial state descriptions and trends in those descriptions. This is quite dierent from work on student modeling and plan recognition which use a complete action model as the basis of checking/inferring a plan with an agent's actions. These action-based approaches are not as tolerant of incomplete knowledge nor can they easily infer from low level actions the kinds of higher level analysis that are important for the instructors here. For instance, recognizing that a platoon is trying to travel in a wedge formation would be at best dicult if the recognition was based on the low level actions that the 16 crew members in the 4 tanks were executing. And evaluating this team behavior is best done at the level of the abstract, partial state descriptions, especially trends in those descriptions, and not at the level of individual discrete actions these various team members are performing. The relation between the levels is not strongly xed. Moreover, recognition and evaluation is best done at the level at which this joint behavior has consequences in this dynamic environment. Still, causal analysis at the level of individual actions would be of diagnostic utility but only in the context of the higher level analysis and not as a way of deriving the higher level analysis. Some form of constrained plan recognition, for example, may be useful for determining why a wedge formation is falling apart.

5 Example Let's now consider an example run of the Probes system using a situation space to support an instructor. In the following example, the team is training on the exercise discussed earlier, traveling in a wedge to a blocking location. Blue is the platoon being assessed whereas Red is the opposition. Here, both sides are comprised of synthetic agents being run by ModSAF, but in an actual exercise some of the agents would be human crews in simulators. The Probes system provides instructors with a coordinated set of presentations. Figure 4 shows two presentations: the situation space for the exercise on the left, with the current situation (team perspective) highlighted. On the right is a log of high-level events and assessments. It is the role of the PuppetMaster to automatically determine which situation is currently in force, by monitoring the

13

activities in the simulation. Probes also uses synthesized speech to notify the instructor of state changes, so that instructors know the current situation regardless of where they are gazing.

Figure 4: Probes' Situation Presentation. The exercise analysis log records events and analyses that are likely to be relevant to the current situation. Note that in the example, when Blue initiates Situation Travel (Wedge) (i.e., starts traveling in a wedge formation), the PuppetMaster reports that vehicles in the platoon are in a poor wedge formation. This analysis is performed only when PuppetMaster recognizes the platoon should be in a wedge. In contrast, when Situation Act occurs (i.e., Blue is in a Action on Contact Situation) PuppetMaster starts reporting that the platoon is still in a wedge when it should probably be going \on line", in eect modifying their formation in a fashion consistent with the exigencies of an active engagement. At this point it stops assessment of the the wedge alignment, since the wedge formation is no longer appropriate for the current situation. These analyses use partial state descriptions and trends in those descriptions, in particular, persistence in the relations between tanks over time. The analyses are not trying to evaluate (or infer) travel in a wedge by reasoning about the actions which the four tanks are executing (or the 16 crew members in the case of human platoons). Also note how the exercise analysis log reports when the red force (the opposing force) spots the blue force (the platoon being assessed) and vice versa. PuppetMaster monitors not just the movement of the vehicles, it also accesses state information internal to the vehicle simulations. This allows PuppetMaster to report on the opposing forces tactics and situation assessments. When Red spots Blue, their internal assessment recognizes a threat and, based on that assessment,

14

the PuppetMaster's pedagogical perspective transitions to Action on Contact (\ActualSit Act"). The pedagogical perspective also would have transitioned if Blue's team perspective had transitioned and the transition was valid (e.g., the opposing force did not turn out to be rocks). In the case of the Blue forces (especially when they are human crews), PuppetMaster infers their intent based on the current situation, the objectives of the exercise and trends in the partial state descriptions that are being monitored. For instance, if Red can (potentially) be spotted and is in range, plus Blue has turrets aiming at Red, breaks out of wedge formation, or attens the wedge, then a situation transition for Blue's perspective can be inferred. As the situation changes, Probes automatically displays statistics about the unit's performance as appropriate to the current situation. For example, when the platoon is traveling in a wedge formation a diagram appears, such as the one in Figure 5, that shows the distances between tanks and the overall depths of the wedge.

Figure 5: A wedge formation gauge. If the instructor wishes to see more information about an observed event, he can obtain it simply by clicking on the event noti cation in the event log. This causes Probes to do some shallow reasoning and bring up an additional window showing local terrain conditions, damage to the vehicles, etc. (See Figure 6.) This information helps the instructor evaluate whether or not the trainee behavior is appropriate under current circumstances. This same capability may be used during the subsequent review with students. Ideally, Probes analysis can be coupled to a 3D (stealth) view of the virtual world. Thus, the instructor could click on an event in the

15

Figure 6: Probes' presentation of speci c details concerning awed wedge formation. exercise analysis log and see a 3D view of the situation at that time. We have experimented with ways of annotating the 3D display with relevant analysis data but the details are beyond the scope of this paper.

6 Status The Probes system has been fully implemented, with all the capabilities illustrated in the \Example" section. We are currently pursuing collaborations that will allow us to evaluate Probes in various training contexts. Our research is also focusing on techniques for automating the construction of situation spaces, including the use of a scenario generation as a front-end to the situation space construction [11]. We are studying the implications of applying situation spaces to more complex teams of students where individual students would be best modeled as distinct situation spaces. In particular, we are concerned with how to formalize the relation between these situation spaces. Work in formal models of teamwork (e.g., [8]) may potentially be useful here. We expect that our design will have applicability to pedagogical agents in domains that share the unpredictable nature of team training of tank platoons. To evaluate that, we are currently studying the applicability of the method to other domains, including Robocup soccer simulations [6].

16

7 Related Work This work is closely related to intelligent tutoring systems that employ some form of student modeling (e.g., [1, 12, 17, 3]) Student modeling typically assumes some task that needs to be performed and some overall plan(s) of action or method(s) are known to achieve that task. In turn, the plan is used to interpret student actions so as to determine, for example, when a student is making a mistake or is at an impasse. Model tracing [1] diers somewhat from this approach in that a production system is used as a cognitive model of the method and the events generated by its execution behavior are matched to the student's behavior. Such approaches are tied to detailed modelling and matching of the actions or events that will occur. Therefore, they tend to have ready application when the skill to be acquired can be relatively tightly scripted both in terms of what speci c actions need to be taken and the order in which they should be taken. Further, they presume a complete model of the problem solving task in the form of a plan or a production system which can be used to draw inferences about student behavior. The pedagogical goals for the domains we have been considering, however, represent the other extreme. On the battle eld, there are multiple tasks to be performed and multiple agents/teams performing those tasks. Moreover, there is no overall plan of action either at the individual or team level that can be guaranteed to achieve the goal; events can enfold in such a way that obviates any plan. Rather student behavior is tied to the situation the student teams are in. The situation strongly impacts what goals need to be achieved and what tasks are to be performed. This view gains support from the fact that it is consistent with how the Army lays out its exercise plan as well as how it evaluates the student teams. There are student modeling techniques that lie between these extremes (e.g., [3, 12]). For instance, Situated Plan Attribution [3] is a system for tutoring satellite link operators. Because of the interactive, dynamic nature of the operator's task, Situated Plan Attribution attempts to loosen how actions and their goals are t into an overall plan. Nevertheless the intent to do a detailed tting still exists and the plan structure, a temporal dependency network, poses stricter causal relations than the Situation Space places on the transitions between situation. The dierences between the two techniques follow rather

17

directly from dierences in the tutoring domains they focus on and the kinds of tutoring interaction that is required in those domains. Our work is somewhat more loosely related to plan recognition (e.g., [13, 5] and agent tracking (e.g., [15]). Plan recognition involves inferring an agent's unknown plan based on their actions. Our pedagogical agent work diers in several ways. The intent is not to infer a plan. We have knowledge of the abstract plan. Rather the issue is evaluating how well the various student behaviors serve the objectives of the plan. Furthermore, plan recognition often presumes plans that are step-by-step procedures. This is inconsistent with the nature of the reactive plans we are considering { very abstract goal decompositions represented only indirectly by alternative paths through the situation space. Our concern with reactive components in behavior is shared by the agent tracking work (e.g., [15]) which does not assume plans are step-by-step procedure but rather a mix of plan-based and reactive procedures. In contrast to the intent of our work, agent tracking does share with plan recognition the goal of inferring an agent's plan. As a consequence, it assumes access to the information and agent's action models that can support that analysis. Finally, in such action-based approaches, there would need to be a way to t the modeling of an individual agent's behavior into the modeling of team behavior, which is our focus.

8 Concluding Remarks Simulation-based training of teamwork skills for dynamic, unpredictable multi-agent settings present special challenges from a pedagogical standpoint. It is our view that the instructor can bene t immensely from a pedagogical agent that is an intelligent interpreter of events in the simulation. Further, only a synthetic agent can play this role: human agents have great diculty even assimilating the information coming from all the agents' viewpoints and as a consequence have diculty assessing that information. To be useful, such a pedagogical agent also needs to be able to interpret events from a viewpoint of instructional objectives. Finally, the agent must be able to handle missing information as well as dierent kinds of agent architectures. We have developed a pedagogical agent that can assess a training session from the standpoint of instructional objectives. This pedagog-

18

ical agent uses a situation-based approach to modelling behavior that is consistent with the information it has available and the analyses it needs to perform.

19

References [1] Anderson, J.R., Boyle, C.F., Corbett, A.T. & Lewis, M.W. R. Cognitive Modelling and Intelligent Tutoring. In Arti cial Intelligence, 42(1), 1990. Pp. 7{49. [2] Calder, R.B., Smith, J.E, Courtemanche, A.J., Mar, J.M.F. & Ceranowicz, A.Z. ModSAF Behavior Simulation and Control. In Proceedings of the Third Conference on Computer-Generated Forces and Behavioral Representation. Orlando, Fla: University of Central Florida Institute for Simulation and Training, 347-356. [3] Hill, R. & Johnson, W.L. Situated plan attribution for intelligent tutoring. In Proceedings of the National Conference on Arti cial Intelligence. Menlo Park, CA: AAAI Press, 1994. [4] Hollan, J.D., Hutchins, E.L., & Weitzman, L. STEAMER: An Interactive Inspectable Simulation-Based Training System. In The AI Magazine, Pp. 15-27. AAAI, Seattle WA, Summer, 1984. [5] Kautz, H. & Allen, J.F. Generalized Plan Recognition. In Proceedings of the National Conference on Arti cial Intelligence. Menlo Park, CA: AAAI Press, 1986. [6] Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I. & Osawa, E. RoboCup: The Robot World Cup Initiative. In Johnson, W.L. (Ed.) Proc. of The First International Conference on Autonomous Agents. New York: The ACM Press, 1997. Pp 340{347. [7] Lester, J.C. & Stone, B.A. Increasing Believability in Animated Pedagogical Agents. In Johnson, W.L. (Ed.) Proc. of The First International Conference on Autonomous Agents. New York: The ACM Press, 1997. Pp 16{21. [8] Levesque, H.J,, Cohen, P.R. & Nunes, J.H.T. On Acting Together. In Proceedings of the Eighth National Conference on Arti cial Intelligence (AAAI-90). Los Altos: CA, Morgan Kaufman, 1990. Pp. 94{99. [9] Marsella, S. C. & Schmidt, C.F. Reactive planning using a "Situation Space". In Hendler, J. (Ed.) Planning in Uncertain, Unpredictable, or Changing Environments (Working Notes of the 1990 AAAI Spring Symposium). Available (preferably) as SRC TR 90-45 of the Systems Research Center, University of Maryland, College Park, Maryland. 1990.

i

[10] Murthy, S. K., Kasif, S. & Salzberg, S. A System for Induction of Oblique Decision Trees. In Journal of Arti cial Intelligence Research, 2, 1994, Pp. 1-32. [11] Pautler, D., Woods, S. & Quilici, A. Exploiting Domain-Speci c Knowledge To Re ne Simulation Speci cations. Accepted at The Twelfth IEEE International Conference on Automated Software Engineering, Incline Village, Nevada November 1997. Also available at http://spectra.eng.hawaii.edu/ alex/Research/Abstracts/aseconf97.html. [12] Rickel, J. & Johnson, J.L. Integrating Pedagogical Capabilities in a Virtual Environment Agent. In Johnson, W.L. (Ed.) Proc. of The First International Conference on Autonomous Agents. New York: The ACM Press, 1997. Pp 30{38. [13] Schmidt, C. F., Sridharan, N.S. & Goodson, J.L. The plan recognition problem: an intersection between psychology and arti cial intelligence. Arti cial Intelligence, 11(1,2), 1978. Pp. 45{83. [14] Schmidt, C.F., Goodson, J.L., Marsella, S.C., Bresina, J.L., Reactive Planning Using a Situation Space. In Proceedings of the 1989 AI Systems in Government Conference, IEEE, 1989. [15] Tambe, M. & Rosenbloom, P. Architectures for Agents that Track Other Agents in Multi-Agent Worlds. In Wooldridge, M., Muller, J.P. & Tambe, M. (Eds) Intelligent Agents II, Agent Theories, Architecture, and Languages. Berlin: Springer-Verlag, 1996. Pp. 156{170. [16] Tambe, M., Johnson, W.L., Jones, R., Koss, F., Laird, J. E., Rosenbloom, P.S. & Schwamb, K. Intelligent Agents for interactive simulation environments. In The AI Magazine, 16(1), Spring, 1995. [17] VanLehn, K. Student Modelling. In Polson, M.C. & Richardson, J.J. (Eds.), Intelligent Tutoring Systems. Hillsdale, NJ: Lawrence Erlbaum, 1988. Pp. 55{78.

ii