and the troublemaker can give him advice. But, the troublemaker is not a reliable assistant: it sometimes gives correct solutions and sometimes tries to mislead ...
A Multi-Agent Architecture for an ITS with Multiple Strategies Thierry Mengelle, Claude Frasson Université de Montréal, Département d'informatique et de recherche opérationnelle 2920 Chemin de la Tour, Montréal, H3T 1J4, Québec, Canada E-mail: {mengelle, frasson}@iro.umontreal.ca
Abstract. This research aims to implement an intelligent tutoring system with multiple co-operative strategies involving several pedagogical agents (for instance: tutor, co-tutor, companion, ...). Such agents are able to play different pedagogical roles, consequently, we called them actors. We describe a general architecture for actors, and illustrate it in the context of a new strategy : learning by disturbing. A prototype of an environment for building actors has been implemented using an object oriented language; it allows to develop new co-operative pedagogical strategies.
1. Introduction Initially, Intelligent Tutoring Systems (ITS) considered only two partners: the computer which tried to simulate a human tutor and the real learner. The resulting systems were generally difficult to control and not always efficient from a pedagogical point of view. Since the mid-eighties, various alternatives to this one to one tutoring strategy have been proposed: learning with a co-learner [5], learning by teaching [10], or, more recently, learning by disturbing [1]. All of these new approaches suggest that the computer can simulate various pedagogical agents which can co-operate with the learner in a constructive process. Because the efficiency of these co-operative strategies depends on the learner’s characteristics, our objective is to implement an ITS with multiple strategies. In this paper, we describe a distributed architecture for this kind of ITS. We first show how ITS architecture can be structured to distribute pedagogical expertise among various agents. Each agent is considered to be an actor which can assume various pedagogical roles according to the current strategy. Concerning the different kinds of intelligent agents, we define the main properties that are required for these actors. We then describe the architecture of an actor and illustrate it in the context of the learning by disturbing strategy [1]. To conclude, we briefly describe a prototype of an authoring environment allowing the implementation of similar cooperative strategies involving several actors.
2. Global Architecture of an ITS with Multiple Strategies
Traditional ITS consider only two partners (tutor and human learner); their architecture generally include a curriculum, a learner model and a pedagogical module. However, as mentioned above, we need a more flexible communication between the learner and the system allowing various interactions between several partners, for instance a companion which simulates the behavior of another learner [3], a co-tutor that helps the tutor [1], ... Session Manager
Curriculum
Troublemaker
Tutor
Supervisor
Learner Model
Co-Tutor
Companion
Non-Didactic Resource
Artificial Learner
Human Learner
Fig. 1. Architecture of an ITS with multiple strategies.
Figure 1 presents an architecture for an ITS with multiple strategies. The session manager uses the curriculum and the learner model to choose an activity allowing to reach a specific objective (a non-didactic resource1) and the pedagogy to manage this activity. In this paper, we focus on the modelization of pedagogical strategies that supervise the interaction between the learner and a non-didactic resource (for instance: a problem, a multiple choice questionnaire, ...). Each pedagogical strategy involves some specific pedagogical agents. For instance, the one-to-one tutoring strategy involves only two agents: the tutor and the artificial learner. (which is intended to synchronize the human learner’s activity with the different agents); the learning with a companion strategy [3] calls a third agent: the companion. A same agent can act in several strategies; alone with the learner, the previous companion will play the role of the co-learner as suggested in [5]. Since the same agent can 1
In the context of the SAFARI project, which supports our work, we consider two kinds of resources : didactic resources that are self-managed (for instance, a CBT module), and non-didactic resources. In this paper, we are interested in the exploitation of this second kind of resources by the pedagogical agents.
play different roles according to the chosen strategy (the tutor acts differently whether it is alone with the learner or whether it is also with a third agent) leads us to define pedagogical agents as actors. In the next section will also distinguish actors from other intelligent agents. To illustrate the different paradigms, we will take the example of the learning by disturbing strategy [1]. This strategy aims to strengthen the learner’s selfconfidence; it involves three partners (Figure 1): the tutor, the troublemaker and the human learner (by way of the artificial learner). The learner works on a specific task and the troublemaker can give him advice. But, the troublemaker is not a reliable assistant: it sometimes gives correct solutions and sometimes tries to mislead the learner in order to check and improve his self-confidence. Though the companion shares with the troublemaker the characteristic of giving right or wrong solutions, their motivation and functioning are quite different. The companion tries to model a human student; so when it does a mistake, it is only because it supposes that, according to the current knowledge state, a real student would act like this. The troublemaker does not give wrong solutions accidentally; it ’voluntarily’ decides to mislead the learner. This difference of motivation has an impact on the knowledge of these actors: the troublemaker needs pedagogical knowledge to decide when it is suitable to mislead the learner, while for the companion only the domain knowledge is absolutely required (one way of implementing the companion is to use a machine learning approach [3]). Unlike the companion, the knowledge of the troublemaker is quite superior to that of the real student. A first experiment (learning of highway code) has shown that the learning by disturbing strategy becomes efficient for advanced students [2]. Some experiments on the other pedagogical strategies are under way in order to find rules that will allow the session manager to select the best strategy according to learner’s characteristics.
3. Intelligent Agents for ITS: Actors 3.1. A new kind of Intelligent Agent: Actors The architecture of an ITS with multiple strategies described in figure 1 requires that the different pedagogical agents have some specific properties. In presenting these properties we follow the evolution of research in the field of intelligent agents. The first point concerns the autonomy of the pedagogical agent. An agent can operate without human control in various situations involving different other agents (for instance, in the one-to-one tutoring strategy, the tutor interacts directly with the student; in the learning by disturbing strategy, it co-operates with the troublemaker). To ensure this social ability, each agent needs some perception capabilities. In our context, pedagogical agent must show ability to react according to the activity of the other agents. It seems suitable to consider two kinds of reactions: immediate responses to stimuli without reasoning (like in reactive agents), and controlled reactions that require planning, prediction and diagnosis capabilities [8]. Because an agent can act in different learning strategies, we need to allow it to evolve, to be adjusted to new learning situations. To ensure this adaptation, the designer can simulate the system and modify the agents’ knowledge. This is the
main characteristic of instructable agents [6]. These can receive instructions or new algorithms. To help the designer, we want to allow agents to dynamically improve their behavior. In particular, like adaptive agents, they have to adapt their perception and their decisions (reasoning methods, control strategy, ...) according to current objectives, available information, and performance criteria. Beyond this adaptation ability, for an ITS with multiple strategies, it may be suitable to design cognitive agents which model human behavior in learning situations. They should be able to learn from experience. This extended notion of intelligent agent is what we consider an actor. An actor is an intelligent agent which is reactive, instructable, adaptive and cognitive. Actors are able to play different roles according to the learning situation in a co-operative environment. 3.2 Architecture of an Actor The architecture (Figure 2) of an actor contains four modules (Perception, Action, Control and Cognition) distributed in three layers (Reactive, Control and Cognitive). 8
COGNITION
7 CONTROL
6
3 2
4 PERCEPTION
ACTION
1
EXTERNAL VIEW
Environment Other actors + Common Information
INTERNAL VIEW
5 Previous behavior
Reactive Layer Control Layer Cognitive Layer
9
Other actors
Functioning modes :
1 2 3 4
Reflex mode Control mode Reasoning mode Observation mode
Impacts of cognition layer :
5 6 7 8
Improvement of perception Creation of actions Improvement of control Change of access permissions
9 Improvement of cognition
Fig. 2. Conceptual architecture of an actor [4].
An original point of this architecture is that each actor can observe the previous behavior of the other actors, a record of their actions (and not only the results of actions). As we will later show, the view an actor will have on the behavior of the others will depend on its position inside the system. To allow this capability each actor has an external view on the other actors and an internal view allowing other
actors to consult its own behavior. So, beyond a common memory, the environment of an actor consists of all the other actors. The architecture of an actor relies on four modules. • The perception module detects changes of the environment, and identifies the situations in which the actor may intervene. Evolution of the environment results from the activity of the other actors (the fact that the troublemaker has just given a misleading information or that an answer of the learner becomes available in the common memory). This module consists of a set of typical situations [7]. Each typical situation describes a specific condition of activation according to the characteristics of the environment. • The action module allow the actor to act on the environment. It consists of a set of action tasks. The elementary action tasks that are directly perceptible by the other actors are called operating tasks (for instance: display an answer, congratulate), the others are named abstract tasks (for instance: in the case of the tutor, FindNew-Problem is an abstract task that can be hidden; in the case of the troublemaker, the Mislead abstract task calls the Display-Answer operating task with a wrong answer as a parameter). • The control module handles situations which imply a planning aspect in order to determine the actions to be activated (e.g. the tutor can decide to stop or continue the tutoring session, the troublemaker may decide to give a right or wrong solution). The control module contains several control tasks which are activated by typical situations or by other control tasks. The goal of a control task is to participate in a decision process which selects and activates a sequence of action tasks (see next section). • The cognition module concerns the improvment ability of the actor. This module will allow the actor to dynamically improve its performance. In a first step, we want to allow this module to help the designer when adjusting the actor's behavior to new situations (for instance, by advising him of what seems wrong in the actor's behavior). Then we will move toward an automatization of the improvment process. To reach these goals, this module consists of several cognitive tasks. Each cognitive task attempts to improve a specific aspect of the actor, for instance improve actor's perception (arrow labelled ) or expand the control (❺)... Cognitive tasks are not activated from other components (typical situations and tasks) but are permanently running; they possess two parts: a learning algorithm and an action part. For instance, a cognitive task dedicated to the improvement of perception will use a case-based reasoning algorithm in order to infer new typical situations. The links between these different modules allow four functioning modes. The reflex mode (❿) involves perception and action modules; in that case, there is a direct association between a typical situation and one task of the action module (abstract or operating task) without reasoning capabilities (spontaneous action). In the control mode (❡), the control module co-ordinates the perception and action subsystems; starting from the activation of a typical situation, a control task takes a decision among possible alternatives and calls the suitable action tasks. The two other functioning modes will involve the cognition module. The reasoning mode
(➆) will allow the cognitive tasks to override the knowledge of the other tasks in order to improve the actor current behavior. While the primary purpose of these three modes is to have the actor interact with its environment, the actor may learn (since the cognitive tasks are always active). In the observation mode (➘) the actor will remain passive but will try to learn from the observation of the others. So, this mode will only involve the perception and cognition modules. This mode will allow the actors that are not directly involved in the strategy to learn from the others. Since tasks are classified according to four categories (operating, abstract, control and cognitive), the actor’s behavior can be observed according to several levels of abstraction or views. The basic view (operating view) which presents operating tasks only, roughly, shows what the actor has done while the other views explain why. So, actors can have a more reliable behavior by knowing the reasons of the other actors’ activities.
4. Example: the Learning by Disturbing Strategy In order to illustrate the previous architecture, we take the syntactical example of a simplification of the learning by disturbing strategy, which has been briefly described in section 2. We give below a short informal description of this simplified strategy applied to the management of a multiple choice questionnaire: The tutor asks questions. The troublemaker can give right or wrong solutions, but it can react only once to each of the tutor’s request (intervention before the learner, after the learner, or no intervention). Finally, according to the answer of the learner, the tutor approves or congratulates him, or gives him the right solution.
TUTOR
T-TS3
1
T-TS1
T-TS2
Scene2: Assess
5 3
4 6
Scene3: NewQuestion
Strategic view
Find-NewQuestion
Congratulate
Approve
Abstract view
Learner-answer
Stop?
go on
Operating view
Common Memory
congratulate
t1. (Scene1: Start (Find-New-Question) (Display-Question))
Tactical view
t1. ME (TUTOR) (Scene1: Start (Find-New-Question) Approve-Or(Display-Question) Congratulate t2. TROUBLEMAKER 2 (Mislead (Display-An-Answer) Scene1: Start ) t3. ARTIFICIAL LEARNER (Answer)
Previous behavior
ImproveDecisions
INTERNAL VIEW
Behavior of the actors
DisplayQuestion
EXTERNAL VIEW
TROUBLEMAKER
Previous behavior t2. (Scene1: React-To-Question (Choose-An-Attitude) (Mislead (Display-An-Answer)))
ARTIFICIAL LEARNER
Previous behavior t3. (Scene1: Answer (Answer) )
COMMON MEMORY
Learner-answer
ENVIRONMENT
Fig. 3. Implementation and example of functioning of an actor: the tutor
To define a new strategy, our approach promotes reuse. We encourage the designer to take into consideration the existing actors and to implement this new strategy by using a simulation process [7]. This process allows him to progressively adjust existing actors to fit the new strategy and to define the new actors. Creating, or adjusting, an actor requires the definition or modification of typical situations and tasks. In this example, we have defined two new typical situations for the tutor (T-TS1 allows it to start the session and T-TS2 which leads to the evaluation of the student). Two typical situations allow the troublemaker to intervene before or after the answer of the learner answers. To illustrate the functioning of these actors, let us consider the following situation : we are at time t4, the tutor has asked the first question (t1), the troublemaker has then tried to mislead the learner (t2) who nevertheless has given the right answer which is now available in the common memory (t3). Because the troublemaker has already reacted on the current question (time t2), none of its typical situations is now triggerable; so, the tutor tries to become active. First it tries to rebuild, step by step, the whole behavior of the society according to its view on the two other actors. This explains why the result of this operation (behavior of the actors indicated on the left side of the figure 3) contains only the abstract and operating tasks of the troublemaker, and only the operating tasks of the
artificial learner. Then, the tutor checks each of its three typical situations2. Each typical situation is described by an object with three parts: a focus, a condition, and a conclusion (see the example of the T-TS2 typical situation on the table below). The focus restricts the view of the environment in order to only consider information that is relevant for the evaluation of the condition (here, the behavior of the three actors for the current question and the common memory). The condition is a logical proposition (here, the fact that the student does not need to react to the troublemaker). The conclusion refers to the task to be activated when the condition is true (here, the Scene2: Assess control task). Focus Condition Conclusion Access to behaviors: Yes (Learner-answer is-in Common Memory) Scene2: Assess From: TUTOR last Operating Task & ((LastActor • TROUBLEMAKER) Only: TUTOR, TROUBLEMAKER, | (Remain-Silent is-in LastActor.behavior)) ARTIFICIAL LEARNER Common memory: Yes
The previous typical situation is now triggerable, so the Scene2: Assess control task is activated (arrow labelled ❿ on figure 3). The algorithm of this task first checks the learner’s answer; here, because this answer is correct, another control task is activated in order to choose between approval or congratulation of the learner (❡). To make this choice, the expertise of the ApproveOrCongratulate control task can analyze the affective part of the student model3 and has to consider that the learner has succeed in spite of the intervention of the troublemaker. To define this expertise, the designer can decompose this task into several subtasks. Finally, the elementary control tasks can be encoded using rulebases, or other formalisms. Figure 3 supposes that the decision is to congratulate the learner, so Scene2: Assess calls the Congratulate operating task (➆). Then, Scene2: Assess calls another control task ( ) which decides to go on; this decision leads to the activation of the Find-New-Question abstract task ( ). This task returns a new question which is finally given as a parameter for the Display-Question operating task (★). Activation of these various tasks are automatically stored in the previous behavior area of the tutor. Then, the different actors will try again to become active. The process will stop if there is no actor who has a typical situation which can be triggered (here, typically after the tutor decides to stop the session). The previous scenario presents a typical example of the activity of the actor in the control mode. To see examples of functioning in the reasoning mode (i.e. preemption of a control task by a cognitive task after learning), the reader can refer to [4]. 2 TM-TS3
is a typical situation which has been defined for another strategy and allows the tutor to change its question consequently to the request of another actor : the supervisor. 3 In the context of the SAFARI project, the learner model consists of three parts : a cognitive model, which represents the domain knowledge of the student with an overlay on the curriculum, an affective model, which stores the habits and preferences of the student, and an inferential part allowing to dynamically update the learner model [9].
5. Description of the Prototype We have used the Smalltalk object-oriented language to implement a prototype of a generator of co-operative pedagogical strategies. Each strategy involves several actors that are described according to the previous architecture. To define an actor, we supply the designer with two editors: one for the typical situations, and one for the tasks. The functionalities of these editors promote the reuse of components when defining new actors. For instance, when the designer defines the tasks of a new actor, the editor displays the list of tasks that are already implemented for other actors. So, it is easy to make two actors sharing the same task (for instance, the tutor and the troublemaker share the Give-Solution abstract task) or to define a new task by adjusting an existing one. In this prototype, the coding of the tasks and of the condition part of typical situations is done directly in Smalltalk. Primitives of a high level language allowing to express this expertise have been defined but are not yet implemented. To define a new strategy, another editor allows the designer to select the actors that he wants to involve. In the present state, the main restriction concerns the cognition layer which is not implemented in the prototype. So, actors can not dynamically improve themselves; that is why the definition of new strategies requires using a simulation process. This process allows the designer to progressively refine the actors’ knowledge (modification of typical situations and tasks). To make this process easier, when playing a session, a window displays the sequence of tasks activations and the state of each actor is symbolized with a specific color (green when active, orange when trying to become active and red when passive). We have also implemented a tool which allows to replay a given session. The designer see the sequence of activations; he can define breakpoints, consult the parameters and results of task and, so, understand the behavior of the system. We have first used these tools for implementing the simplified version of the learning by disturbing strategy as described in section 4 (tutor, troublemaker and artificial learner). We have then defined the learning by supervising strategy (definition of a new actor, the supervisor, which can ask the tutor to change its question). We have experimented the simulation process in order to try different combinations of these four actors. This process has leaded us to modify some typical situations in order to reach a reliable behavior for the system. The pedagogical expertise that has been implemented is quite limited. A parallel study on the learning by disturbing strategy will lead to the implementation of a concrete expertise in a few weeks.
6. Conclusion In this paper, we have described a general architecture of actor allowing to implement ITS with multiple strategies. Parts of this architecture have been yet implemented in a prototype which allow to edit and to combine pedagogical strategies. To implement such strategies, the designer uses a simulation process allowing to progressively adapt the actors’ behaviors. In the next step, the
implementation of the cognitive module will facilitate the design of new strategies by providing the actors self-improvement capabilities.
Acknowledgments This work has been supported by the Ministry of Industry, Trade, Science, and Technology (MICST) under the Synergy program of the Government of Québec.
References 1.
Aïmeur, E., Frasson, C., Stiharu-Alexe, C.: Towards New Learning Strategies In Intelligent Tutoring Systems, Brazilian Conference of Artificial Intelligence SBIA’95 (1995) 121-130 2. Aïmeur, E., Frasson, C.: Analyzing a new learning strategy according to different knowledge levels, Computer and Education, An International Journal (1996), to appear 3. Chan, T.W., Baskin, A.B.: Learning Companion Systems. In C. Frasson & G. Gauthier (Eds.), Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education, Chapter 1, New Jersey, Ablex Publishing Corporation (1990) 4. Frasson C., Mengelle T., Aïmeur E., Gouardères G.: An Actor-Based Architecture for Intelligent Tutoring Systems. Third International Conference ITS’96, Montréal. Canada, LNCS (1996), to appear. 5. Gilmore, D., Self, J.: The application of machine learning to intelligent tutoring systems. In J. Self (Ed.), Artificial Intelligence and Human Learning, Intelligent computerassisted instruction, New York: Chapman and Hall (1988) 179-196 6. Huffman, S. B., Laird, J.E.: Flexibly Instructable Agents, Journal of Artificial Intelligence Research , Volume 3 (1995) 271-324 7. Mengelle, T.: Etude d'une architecture d'environnements d'apprentissage basés sur le concept de préceptorat avisé. PhD Thesis, University of Toulouse III (1995) 8. Morignot P., Hayes-Roth, B.: Why does an agent act ? In M.T. Cox & M. Freed (Eds.), Proceedings of the AAAI Spring Symposium on Representing Mental States Mechanisms. Menlo Park, AAAI (1995) 9. Nkambou, R., Lefebvre, B., Gauthier, G.: A Curriculum-Based Student Model for Intelligent Tutoring System. Fifth International Conference on User Modelling , Kailua-Kona (1996) 91-98 10. Palthepu, S., Greer, J., McCalla, G.: Learning by Teaching. The Proceedings of the International Conference on the Learning Sciences, AACE (1991)
