multi-stage cooperation algorithm and tools for agent-based planning ...

MULTI-STAGE COOPERATION ALGORITHM AND TOOLS FOR AGENT-BASED PLANNING AND SCHEDULING IN VIRTUAL LEARNING ENVIRONMENT1 Leonid B. Sheremetova,b, Gustavo Núñeza a

Centro de Investigación en Computación, Instituto Politécnico Nacional, Unidad Profesional Adolfo López Mateos, Apdo# 75476, México, D.F., C.P. 07738 Fax: (+52) -5- 5862936 e-mail: [email protected] b St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences Abstract In this paper we describe a problem of generating of study plans in the virtual learning environment, which we have named EVA (states for Virtual Learning Spaces in Spanish). Planning of trajectories and scheduling of learning activities in loosely coupled knowledge domains are performed for each student within a number of temporal and spatial constraints. A model of knowledge for each domain is represented as a type of semantic network, called a concept graph. Cooperative agents responsible for local planning and scheduling are associated with each knowledge domain. A planning procedure over AND-OR graphs is used to generate plan alternatives. A number of heuristics are applied while agent is committed to a particular local plan, which is communicated to the agents controlling related domains. Multistage negotiation algorithm provides means by which an agent can acquire enough knowledge to reason about the impact of his local planes and to achieve globally consistent solution. A facilitator agent coordinates agent’s activity. Prototypes of agents have been developed using JDK 1.2 and JATLite packages. Keywords: multi-agent systems, planning, negotiation, virtual learning environment.

1. Introduction Since their conception more than a quarter of a century ago, knowledge-based learning environments have offered significant potential for fundamentally changing the educational process [2, 16, 25]. The key feature of these systems is the possibility to acquire, represent and use knowledge. This knowledge usually contains a model of problem domain, a model of student's beliefs, and a model of teaching strategies and styles. A problem domain model is an agglutinating center, which relates the concepts to be taught, is used to define student’s state in the knowledge space and to find solutions and applicable rules to present knowledgeable feedback to students. Thereby it allows to include learning, reasoning and activities planning capabilities into the characteristics of virtual learning environments. Nevertheless, despite of many expectations, few learning environments have made the difficult 1

The work is supported by CONACyT and IPN, Mexico

transition from the laboratory to the classroom, so the development of pedagogically sound tools has been the time challenge. The investigation project, which we have named EVA (Espacios Virtuales de Aprendizaje in Spanish - Virtual Learning Spaces), applies the methodology and tools of Distance Learning and Intelligent Tutoring Systems to obtain a new paradigm of the Configurable Collaborative Learning [18]. This project is dedicated to the research and development of pedagogic models and information technologies that provide spaces of knowledge, collaboration, consulting, and experimentation for supporting the learning activities of teams separated geographically that maintain common conversations, matters, and projects. Agent technology is the promising way to approach these problems. The notion of agents is the central part of contemporary learning environments, where they act as virtual tutors, virtual students or learning companions, virtual personal assistants that help students to learn, mine information, manage and sched-

ule their learning activities [1, 9, 14, 17]. The use of intelligent agents is supposed to help to make a further step in developing customized learning experiences composed of customized sequences of units of learning material (ULM), each of them located anywhere [11]. The main purpose of our project is to develop models, architectures and multi-agent environment for collaborative learning and experimentation. The focus of this paper is one of these problems: learning activities planning and scheduling. The conceptual architecture of EVA is structured into four essential knowledge elements formed by four information deposits and a set of programs called Virtual Learning Spaces. These spaces are: • knowledge - all the necessary information to learn, • collaboration - real and virtual companions that get together to learn, • consultation - instructors or assessors (also real and virtual), who give the right direction for learning and consult doubts, and • experimentation - the practical work of the students in virtual environment to obtain practical knowledge and abilities. To navigate these learning spaces, a learner needs his personal routes (study plans) suggested in an automatic manner by EVA. So, the purpose on the planning system is to design a particular learning trajectory for each student in the learning spaces and schedule it in time. At the next stage, personalized books, called Multibooks, are armed by concatenating of selected ULM along the learning trajectory for each knowledge domain. In the same way, groups of students with similar interests are arranged. The problem of planning and scheduling belongs to the area of combinatorial problem solving, difficult to be efficiently solved in a traditional way, including traditional knowledge-based approach developed during last two decades. Now it seems, that multiagent system (MAS) technology is one of the most promising ways to manage this challenge due to a distributed way of tasks solving [5, 10, 15], where agents make their local decisions on plan fragments and negotiate the global decision. A distributed way of decision making means that each agent must make his local decision having a deficit of information about environment and other agents, resulting in conflicts between the decisions. The common idea in all distributed artificial intelligence (DAI) contributions is that agents use negotiation for conflict resolution and hence the basic

idea behind negotiation is reaching a consensus. Probably, the most commonly used negotiation protocols for task and resource allocation and coordination among agents is the Contract-Net Protocol (CNP) and auction-based protocol, both developed for centralized way of resource allocation [7, 21, 24]. Recently, these protocols were investigated by a number of authors, where it was shown that they propose an efficient way for self-interested agent behavior coordination [8, 22]. In this paper, we consider the use of multistage negotiations algorithm among agents of MAS that have to solve a complex combinatorial task of planning and scheduling of learning activities that makes it possible to detect and to resolve subgoal interactions and conflicts. It can be considered as an extension of CNP, because multiple contracting mechanism is applied. The most close task statement is one considered in [5], which was applied to traffic planning and control of a complex communications system. In the framework of their model, a cooperation strategy in which agents iteratively exchange tentative and high level partial results of their local subtasks, was generated. This strategy results in solutions, which are incrementally constructed to converge on a set of complete local solutions, which are globally consistent. A specific feature of the task statement in this paper, in general, is that we aim at solving tasks of planning and scheduling, which complexity is conditioned by real time, temporal, and other constraints imposed on synchronous collaborative learning activities. In our case agents are not self-interested, they cooperate to satisfy a global goal - to generate acceptable study plans. In the paper, we focus on the problem of planning of collaborative learning activities, negotiation algorithm, domain agent and MAS architectures, and implementation. The rest of the paper is organized as follows. In section 2, we discuss the problem statement of formation of study plans in the virtual learning environment on conceptual level. The problem is specified as a task of dynamic planning and scheduling of learning trajectories in the knowledge domains represented as a type of semantic network, called a concept graphs. In section 3, a model of problem solving based on goals definition within the constraints is considered. In section 4, the main idea of algorithm of multistage negotiation is described. In section 5, domain planning agent’s and MAS architectures are considered and implementation details are discussed. Finally, section 6 is devoted to the discussion of the proposed algorithm in the context of different negotiation techniques. In conclu-

sion we present the main results of the paper and outline directions of future work.

2. Problem statement At the current stage of experiment, the Virtual Learning Spaces are associated with the knowledge taxonomy of Computer Science at the Master of Science level. To represent this taxonomy we have proposed a model based on the hierarchy of knowledge domains and concept graph representation of knowledge.

2.1 Trajectory planning in the knowledge space Let us consider the problem of planning of student’s trajectories over the knowledge space. The model of knowledge is represented in the form of a concept graph G. The concepts at any level of abstraction (courses, ULM or elementary concepts) and relations between them (uni- and bidirectional, in general) correspond to the nodes and arcs of the graph respectively. Each node i has a number of attributes, including its weight (node ratio) that means the importance to achieve a final goal of learning Ii , knowledge constraints (prerequisites) Pi,j and estimated time to learn the concept Ti . A precedence relation i→ j is used to relate concepts that means that concept 'i' has to be learned directly before the concept 'j'. Knowledge constraint attribute Pk,j captures a prerequisite relation k-->j that means that concept 'k' is needed to learn the concept 'j', but not necessarily precedes it. Each arc (i, j) also has its weight (relation ratio) that means the strength of relationship or the necessity of the previous concept for the next one Ni,j. All the ratios are estimated from “not necessary” to “obligatory” and are represented using 5-valued numeric ranking scale from 0.2 to 1. To capture the fact that it can be necessary to learn two or more concepts to proceed with the next one, AND arcs are used. Alternative paths mean different options for learning. Alternative routes are analyzed basing on the following decision criterion (to be maximized): n

∑ I *N i =1

i

i, j

, where n is a number of nodes on the route.

The semantics of this criterion is that generally more long paths are preferable to be selected if and only if they are consistent with a temporal constraint:

n

∑T i =1

i

≤Tc , where Tc is a time constraint

Figure 1 shows an excerpt from the domain model for the "Distributed Intelligent Systems" (DIS) course, adapted syllabus of which is shown in Table 1. It can be seen that the ULMs of this Multibook use concepts from a number of other Multibooks as prerequisites: Object Oriented Programming (OOP), AI, Mathematical Logic (ML) and Distributed Systems (DS). It should be mentioned that at this figure from all the prerequisites only a fragment of AI course concept model is shown. While selecting the alternatives, we are usually interested not in one and the only but in a number of them. An E-conformation is introduced, which means the possibility to initially accept first E alternatives. It is extremely difficult to generate learning trajectories in a general case even at the level of ULM, because of a concept graph dimensions. So, the idea of the proposed model is to capture the natural differences and similarities between the graph fragments, introducing the concept of knowledge domain.

2.2 Model of knowledge domains A subset G’of graph G can be divided in knowledge domains if and only if the following conditions are fulfilled. There exist a collection D0 ,D1 , ... , Dn , n≥2, of subgraphs G’, for which: • D0 ∩ Di = ∅ • The subgraph D0 has outgoing edges (of precedence) to each subgraph Di • The subgraph D0 has no incoming edges (of precedence) from any subgraph Di • In each domain Di there exist at least one node that has no outgoing edges (of precedence). The subgraph D0 is called the “common domain” for the domains D1 , ... , Dn. Note that the division of the graph G in knowledge domains not necessary is unique. Domains also can have common (shared) nodes or intersections. This definition can be used to obtain decompositions successively from whatever subgraph. An example of knowledge domain decomposition is shown in fig. 2. The node N16 is the intersection of the domains D1 and D3 and pertains to the both of them. The common domain for the domains D1, D2, D3 is D0 = {N1, N4, N7}. In the subgraph G’= {N5, N6, N12, N13, N14, N15} of the domain D2, D’0 = {N5, N6, N13} is the common domain for the subdomains {N12, N15} y {N14}.

Module or section Node X AI

L23 (1.0) B1

B2

L25 (1.0)

Identification number

B3

Objective Prerequisites

L21 (0.8)

A4

OOP

Time duration

A9

R6(1)

Importance ratio

L8 (0.4) L10 (0.6) L2 (1.0)

R3 (1)

Edge(X,Y)

Relation ratio (RR) L3 (0.4)

A1

A2

L5 (0.4)

A3

L1 (0.8)

L4 (1.0)

L6 (0.6)

A5

A6

L9 (1.0)

R5 (0.6)

A8

R1 (1)

L11 (1.0)

A11

L12 (1.0)

R1 (0.5)

L7 (0.4)

ML

A10

A7

DS

y

Edge, y=RR

ULM

Attribute

Preceding domain

Multibook: Distributed Intelligent Systems

Figure 1. A fragment of a concept graph for a Multibook on DIS Node number A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11

ULM title

Prerequisites

Introduction to agents and MAS Agents: definitions y classifications Shared knowledge and Artificial Intelgeneral ontology ligence Agent oriented software Object oriented engineering programming Agent oriented proMathematical gramming logic Communication in MAS Distributed systems Interaction in MAS Mobile agents Distributed systems Formal specifications of MAS MAS frameworks and development tools Examples of software agents

Importance Time ratio duration 0.6

4

1.0

4

0.4

6

0.8

4

1.0

10

1.0

12

1.0 0.4

6 4

0.8

6

1.0

6

0.6

4

Table 1: A fragment from the "Distributed Intelligent Systems" Multibook syllabus

This model has the following semantics. Knowledge domains have a hierarchic structure that corresponds to different areas and levels of abstraction: common domain, specialty domains, subspecialty domains. For example, common domain consists of OOP, Data Base Design (DBD), Discrete Mathematics (DM) courses, etc. Domain of AI consists of AI, Logic Programming (LP), ML courses, etc., and contains several sub-domains: Knowledge Based Systems (KBS), Automatic Learning, Natural Language, Vision and Robotics. It should be mentioned that the DIS course is an example of the domain intersection, it pertains to the KBS subspecialty, DS and Software Engineering specialty domains. Usually, courses pertaining to different levels of abstraction can be studied simultaneously. It means that they do not have precedence relations between them. Maximum time duration for each level in the global study plan is also constraint. These temporal constraints are used to schedule student’s learning activities. Initial study plan and learning activities scheduling is generated on the following basis: (i) Student’s initial knowledge state is detected by means of a knowledge prospecting evaluation in the common domain, which is also related to the areas of Computer Science at the graduate level according to the model, proposed by the ACM [4], (ii) Student’s interests in terms of sub-specialty or separated courses from the area, which defines student's final state in the knowledge space. Initial student's knowledge is considered as initial conditions for the common domain, which means that students even with the same interests have different learning trajectories. The difference between the two types of final state definition is also very important for planning. The later case is more general one, because it can result in plan generation, initiated from different knowledge domains. Planning process always starts from the final state in a backward chaining manner. Later on, each time a student pass through the exam, the system evaluates his knowledge and tries to infer the reasons of his misunderstandings. It can result in the goal redefinition and, as a consequence, reestablishment of a new study plan, taking his current state into account. Since a student is studying simultaneously a number of courses, the conditions which give rise to goal instantiation may be observed at more than one place on the general con-

cept graph, and the same goal may be instantiated in two or more domains independently. To perform planning and scheduling within the MAS paradigm, we propose to associate a planning agent with each domain model. These agents must have goal and belief representation capabilities to fulfill the task. Since knowledge domains are interconnected, their local goals and decisions are also interconnected. A multistage negotiation algorithm provides means by which an agent can acquire enough knowledge to reason about the impact of his local decisions and modify its behavior accordingly to construct a globally consistent decision. A special coordinator agent facilitates communication and performs managing activities. This conceptualization can also be expanded to other problem areas, where problem decomposition into domains and subdomains is possible and necessary depending on the problem's complexity.

3. Knowledge representation for planning In this section we characterize a problem-solving model in which multistage negotiation is useful. For the illustration purposes, throughout the rest of the paper we shall consider a simplified example of study plan generation at the level of ULM, which involves a student, who has to study the "Intelligent Distributed Systems" course. Its concept graph model was discussed in the previous section. When viewed from a global perspective, plan generation produces a number of alternative study plans for each student. Each plan fragment, as represented in Table 2 for the DIS course, is a list of alternating nodes and links, traversing the proposed study paths. Suppose that a planning agent A (responsible for the KBS sub-specialty domain) is associated with it. To clarify the example, we have adopted a naming convention for goals and alternative plans, which incorporates the goal and plan number; thus some of alternative plans for the student alm-1 are designated g1/p1, gl/p2, gl/p3 and gl/p4. The decision criterion value for each plan alternative is shown in the second column of Table 2. Plan fragments for goal g1 to plan learning activities for the student alm-1 (agent A domain): Goal/ Deci- Route plan sion criterion g1/p1 5.24

(A1:L1: A2:L4: 'DS' : A8:L7: A6:L9: A7:L11: A10:L12: A11:L13: A12)

g1/p2 8.2

g1/p3 3.88

g1/p4 5.24

(A1:L1: [A2:L3: 'AI' : A3:L5: 'ML' : A5:L6], [A2:L2: 'OOP' : A4:L8: A9:L10], [A2:L4: 'DS' : A8:L7]: 'DS' : A6:L9: A7:L11: A10:L12: A11:L13: A12) (A1:L1: A2:L2: 'OOP' : A4:L8: A9:L10: A10:L12: A11:L13: A12) (A1:L1: A2:L3: 'AI' : A3:L5: 'ML' : A5:L6: 'DS' : A6:L9: A7:L11: A10:L12: A11:L13: A12)

Table 2. Alternative study plan fragments In this example, two plan alternatives (p1 and p3) have the same value of the decision criterion. For the local conflicts resolution the minimum commitment heuristic rule is applied. It means that the plan alternative with fewer relations with other domains will be given a major priority (p1 in this case). An agent cannot simply satisfy a local goal by choosing any plan fragment according to the decision criterion, but must coordinate its choice so that it is compatible with those of other agents. Relations with other domains are reflected in Table 2 by including respective prerequisite courses into the learning trajectory notation. When viewed from the perspective of the system goal, the global study plan appears as an AND-OR tree progressing from the system goal (global study plan at the root), down through goals and plans, to local plan fragments distributed among the agents. Formulation of a plan as a conjunction of plan fragments induces a set of compatibility constraints on the local choices an agent makes in satisfaction of global goals. An agent generally does not have complete knowledge about these compatibility sets. In our application domain, constraints are divided into two categories: local and global constraints. Global constraints are usually temporal ones, e.g. the time at student’s disposal to learn the material. Local constraints include a number of students, simultaneously studying the same material, a number of students to form teams (usually from 5 to 7), available resources, etc. A common domain has a special sort of constraints: each plan fragment has to meet student’s initial conditions. From an agent-centered perspective, plan fragment selection is constrained by local resource availability. An agent cannot choose to execute a set of alternative plan fragments that require more local

resources than are available. For example, if an agent has already assigned 7 students to go through the second route, it can select only other alternatives because of team size constraint. So, from each agent's perspective, the search is over a group of alternatives subject to a set of local resource constraints and a set of global constraints imposed by actions of other agents. Multistage negotiation provides a mechanism by which agents coordinate their actions in selecting plans subject to both local and global constraints. Knowledge about domain model, constraint and heuristic rules is stored in the agent’s local knowledge base (LKB). Dynamic information, such as plan alternatives and constraints is stored in form of beliefs, which along with capabilities and committed alternatives form agent’s mental states. As additional constraints are added to an agent’s mental states, its local feasibility tree is augmented to reflect what it has learned. Details on constraint representation and satisfaction can be found in [23]. Domain agent architecture reflecting described model is shown in fig. 3.

4. Algorithm of multistage planning In this section we discuss the general strategy, then provide more details to the role of negotiation in it. To make these concepts concrete, multistage negotiation is applied to the simplified planning problem, which has been discussed.

4.1 A general planning strategy Multistage negotiation provides means by which an agent can acquire enough knowledge to reason about the impact of his local decisions and modify its behavior accordingly. The algorithm consists of the following stages of planning and scheduling: 1. construction of the space of alternative plans, 2. plan commitment, 3. scheduling of committed plan alternatives. The first stage of agent’s activity is a phase of study plan generation. At this stage, a coordinator agent initializes a terminal agent (that controls a domain model having terminal nodes for study plan - usually a sub-specialty domain agent) to generate alternatives for study plans for the controlled area. Learning routs are generated based on the information contained in the concept graph. While doing that, each planning agent ascertains what alternatives for partial goal satisfaction are locally possible and assigns a rate for each plan according to the decision criterion.

Figure 2. A model of knowledge domains

Figure 3. Multilevel architecture of domain planning agent

Which alternative choice obtains the highest rate, depends upon the adopted criterion. In our case, it is the integrated importance of the route with time duration of the route as a cost function. In the case of competitive learning, for example, it can be the most complete study plan (maximum time). Since at this stage, only one stage of negotiation, as in conventional CNP, is used, we shall not consider the details of plan generation here. A modification of a well-known Dijkstra shortest path algorithm is used for each plan fragment generation [6]. These alternative plan fragments are used as a starting point to initialize other agents.

ers. Impact of local actions is reported as confirmation that a tentative local choice is a good one or as negative information reflecting nonlocal resource conflict. The agent rerates its own local goals using the new knowledge and possibly retracts its tentative resource commitment in order to make a more informed choice. This process of information exchange continues until a consistent set of choices can be confirmed. Synchronized global termination is required in our application, so coordinator agent, which receives all the information of the confirmed plans issues a stop-negotiation message to all involved agents.

At the next stage, the agent A tenders contracts to appropriate agents for furthering satisfaction of the goals needed to complete established plan fragments. Currently, an active agent provides the coordinator agent with the information on agents from other domains whom to contact. Using Econfirmation principle, the following 3 alternatives will be selected for our example: g1/p1, g1/p2 and g1/p3. According to the applied criterion, the second alternative g1/p2 is the best one. So, as we have mentioned this alternative is related to 3 other domains: AI, DS and common domain. Let us suppose that three other agents, B, C, and D are associated with these domains respectively.

4.2 Negotiation algorithm

On completion of this phase, a space of alternative plans has been constructed which is distributed among the agents. Coordinator is responsible to establish communications, transmit partial goals and recollect the alternatives, performing facilitator functions. It should be mentioned that a coordinator agent constructs global plans, which have been generated in a distributed manner, and no single planning agent necessarily knows of all plans or any one complete plan. It is also responsible for global temporal constraint satisfaction and partial constraint relaxation. At the next stage of plans confirmation, each terminal agent examines the goals it instantiated and makes a tentative commitment to the highest rated feasible set of plan fragments relative to these goals. It subsequently issues requests for confirmation of that commitment to agents who hold the contracts for completion of these plan fragments. A protocol applied at this stage can be considered as a multiple CNP. In general case, e.g. when planning occurs for the separate courses, belonging to different domains, each agent may receive two kinds of communications from other agents: 1) requests for confirmation of other agents' tentative commitments, and 2) responses concerning the impact of its own proposed commitments on oth-

When a planning agent begins its activity, it has knowledge of a set of top level goals, which have been locally instantiated. A space of plans to satisfy each of these goals is formulated during plan generation without regard for any subgoal interaction problems. In a general case of planning for a number of students simultaneously, after the phase of plan generation, each agent is aware of two kinds of goals: primary goals (or p-goals) and secondary goals (or s-goals). In our application, p-goals are those instantiated locally by an agent in response to a query from the coordinator agent for courses from the domain, for which the agent has primary responsibility (because the student is interested in the agent's subregion). These are of enhanced importance to this agent because they relate to system goals, which must be satisfied by this particular agent, if they are to be satisfied at all. An agent's s-goals are those which have been instantiated as a result of a contract with some other agent. A plan commitment phase involving multistage negotiation is initiated next. A fragment from the agent interaction diagram, illustrating our example is shown in Figure 4. As this phase begins, each node has knowledge about all of the p-goals and s-goals it has instantiated. Relative to each of its goals, it knows a number of alternatives for goal satisfaction. An alternative is comprised of a local plan fragment, points of interaction with other agents (relative to that plan fragment), and a measure of the cost of the alternative (to be used in making heuristic decisions). Negotiation leading to a commitment proceeds along the following lines. • Each agent examines its own p-goals, making a tentative commitment to the highest rated set of locally feasible plan fragments for p-goals (sgoals are not considered at this point because some other agent has corresponding p-goals). g1/p2 is selected in our example.

• Each agent requests that other agents attempt to confirm a plan choice consistent with its commitment. Note that an agent need only communicate with agents who can provide input relevant to this tentative commitment. Agent A communicates with agents B, C and D agents for the p2 alternative. • An agent examines its incoming message queue for communications from other agents. Requests for confirmation of other agents' tentative commitments are handled by adding the relevant s-goals to a set of active goals. Responses to this agent's own requests are incorporated in the local belief set and used as additional knowledge in making revisions to its tentative commitment. • The set of active goals consists of all the local p-goals together with those s-goals that have been added (in step 3). The agent rates the alternatives associated with active goals based on their cost, any confirming evidence that the alternative is a good choice, any negative evidence in the form of nonlocal conflict information, and the importance of the goal. A revised tentative commitment is made to a highest rated set of locally consistent alternatives for active goals. In general, this may involve decisions to add plan fragments to the tentative commitment and to delete plan fragments from the old tentative commitment. For example, p2 alternative is deleted from the goal set because of a conflict with the goal g2 established by the agent C. Messages reflecting any changes in the tentative commitment and perceived conflicts with that commitment are transmitted to the appropriate agents. • The incoming message queue is examined again and activity proceeds as described above (from step 3). The process of aggregating knowledge about nonlocal conflicts continues until a node is aware of all conflicts in which its plan fragments are a contributing factor. Negotiation activity in an agent terminates by a coordinator agent, when he recollect the information on all confirmed or rejected alternatives. Actually, when a planning agent either has no pending activity and no incoming communications or if an attempt is made to return to a previous commitment with no new knowledge from other agents, he advertises a Coordinator about the termination of his activities. The other issue of importance at this point is related to the quality of the result obtained through negotiation. In the initial negotiation stage, each agent examines only its p-goals and makes a tenta-

tive commitment to a locally feasible set of plan fragments in partial satisfaction of those goals. Since each agent is considering just its p-goals at this stage, the only reason for an agent's electing not to attempt satisfaction of some top level goal is that two or more of these goals are locally known to be infeasible, which means that the problem is initially overconstrained. In subsequent stages of negotiation, both p-goals and relevant s-goals are considered in making new tentative commitments. If the system goal of satisfying all of the p-goals instantiated by agents is feasible, no agent will ever be forced to forego satisfaction of one of its p-goals (because no agent will ever learn that its p-goal precludes others), and a desired solution will be found. If, on the other hand, the problem is overconstrained, some set of p-goals cannot be satisfied and the system tries to satisfy as many as it can. While there is no guarantee of optimality, the heuristics employed should ensure that a reasonably thorough search is made.

5. Multi-agent system architecture and implementation details Learning activities planning system with multistage negotiation is implemented using JATLite package [12]. It is composed of n planning agents and a coordinator agent, which inherit their methods from the RouterClient class. Agents are implemented as JAVA applets, so they also inherit methods from the Applet and Frame (to support graphic interface) classes of JDK 1.2 package. For communication, agents use the general set of JATLite KQML performatives, namely, advertise, tell, reply, and ask-if. At the first stage of experiments, this set was extended by the following performatives, which were introduced for implementation purposes: • request-planning, initial external query to start planning process; • start-negotiation, a broadcast performative announcing the start of negotiation process; • stop-negotiation, a broadcast performative announcing the termination of negotiation process. At the current stage of experiments this additional set of performatives is also changed to the standard one, e.g. start- and stop-negotiation functions are transmitted by a broadcast performative. The role of coordinator is also to supply the agents with initial beliefs that include information about general constraints and resources, retrieved from the global knowledge base and names of agents responsible for each domain. Another function of the coordinator is to generate global study plans and schedules from the proposals submitted by each agent.

Coordinator

Agent A

Agent B

Agent D

Agent C

Request-planning(g1) Ask-about(domain1) Reply(domain1, B) tell(g1, p2, B3) ••• Achieve(StartNegotiation) broadcast(StartNegotiation) Ask-if(g1, p2, B3) Ask-if(g1, p2, C32) Reply(g1, p2, 24) Reply(g1, p2, conflict(C32 AND ~p-goal g2)) ••• Ask-if(g1, p1, D13) Reply(g1, p1, 80) tell(g1, p1, 112) broadcast(StopNegotiation)

Figure 4. A fragment from the agent conversation diagram

Figure 5. A screenshot of the user interface of the concept graph development tool.

Figure 5 shows a screenshot of the user interface of the concept graph development tool. Figure 6 shows a screenshot of a planning system, including JATLite Client Applet, Coordinator agent and 4 planning agents (A-D) for the situation discussed earlier in this paper. The following four agents are involved in the planning process: A, controlling the KBS sub-domain, B, controlling AI domain, C, controlling DS domain, and D, controlling common knowledge domain. For the illustrative purposes, all agents run on the same platform, though really each planning agent runs on the platform, containing domain model files and tools for model development. Agents Graphic Interface consists of two boxes: the left one - for sent messages and auxiliary information (such as committed or conflicting requests), and the right one - for received messages. All the messages maintain KQML format. Agent windows show different stages of negotiation. For example, coordinator agent has already started the negotiation process, having knowledge about existing knowledge route alternatives for each of two sub-goals, established by agents A and C.

agents in response to the announcement, and the evaluation of the submitted bids by the contractor, which leads to awarding a subproblem contract to the contractor(s) with the most appropriate bid(s). Although CNP is considered by Smith and Davis as well as many DAI researchers to be a negotiation principle, other researchers believe it is more a standardized coordination method for the following reasons [13]: 1. There must not be a conflict at all between the agents to start the CNP, hence there is no possibility of bargaining between the agents. The manager does not communicate its minimal condition, nor do the bidders have a second choice (no constraint relaxation), 2. A mutual decision is eventually given by the decision of the offer, hence there is no two-way agreement.

6. Discussion

The last observation seems doubtful because negotiation metaphors can be considered in the both different senses in respect to application. For example, in the well-known auction-based model of agent behavior coordination, agent contractors can be representatives of really competing companies each is self-interested and aimed at each own benefit only. In this case, auction is a mathematical model of a real competition of companies. On the other hand, if we have to solve a centralized planning and scheduling task for activity of a large company the auctionbased model is used only as a metaphor that plays the role of a coordination mechanism for partial solutions made in distributed way. For such a case, agent-contractors are not self-interested and aim at solving an only complex task jointly.

In this section we shall compare our algorithm with different types and strategies of negotiation. Two types of negotiation is usually considered in the MAS research: Competitive and Cooperative Negotiation, also sometimes associated with decentralized and centralized planning [17]. In the first case, negotiation is used in situations where "agents of disparate interests attempt to make a group choice over well-defined alternatives" [20]. Therefore, competitive negotiation involves independent agents with independent goals that interact with each other. They are not a priori cooperative, share information or willing to back down for the greater good, namely they are competitive. Another type of MAS, includes systems where agents have "a global goal/single task envisioned for the system" [24], in that sense these agents are called "collaborative" [3]. In the conventional CNP approach, a decentralized market structure is assumed and agents can take on two roles, a manager and contractor. The basic premise of this form of coordination is that, if an agent cannot solve an assigned problem using local resources/expertise, it will decompose the problem into subproblems and try to find other willing agents with the necessary resources/expertise to solve these subproblems. The problem of assigning the subproblems is solved by a contracting mechanism. It consists of contract announcement by the manager agent, submission of bids by contracting

To follow this, its limitations involve the fact that it does not detect or resolve conflicts, the agents in the contract net are considered benevolent and nonantagonistic (which in real world scenarios is not realistic).

Close task statements to that considered in this paper, but for the case of competitive agents in collaborative planning for Air Traffic Control was proposed in [3, 19]. A cooperative negotiation model, in which the participating agents (pilots, air traffic control) collaborate on developing the best plan in terms of the interests of the agents as a group was developed. This model involves an agent detecting conflicts regarding proposed actions and beliefs from other agents and initiating collaborative negotiation to resolve such conflicts. During negotiation, an agent modifies the proposal with appropriate justification for this modification, based on its own beliefs. In this way, agents collaborate to achieve mutual beliefs, thereby resolving conflicts.

Figure 6. A screenshot of the system with 4 planning agents.

The algorithm discussed in this paper can be considered as a combination of the centralized and decentralized modes of coordination, which are applied at different phases of planning process. While being applied to the case of collaborative agents, it extends the CNP in the following: propose the mechanism for conflict detection, supports iterative exchange of knowledge, and applies constraint relaxation making possible to achieve two-way agreements. That is why we consider it to be a negotiation algorithm, which can be used for coordination purposes of cooperating agents. It also should be noted that this metaphor could be adapted for use in MAS, where agents are self-interested.

4. Conclusion In this paper, we have presented an approach making use of multistage negotiation algorithm for cooperation in distributed planning and scheduling of students learning activities. It is based on the extension of CNP and permits an agent to acquire enough knowledge to detect conflicts of his local decisions with the others and to negotiate

acceptable global decision. The motivation for the development of this cooperation paradigm has the following reasons: (i) subgoal interaction problems that arise in the context of a distributed planning system, (ii) overconstraintness of planning problems, which needs a strategy of constraint relaxation from the global point of view. The last observation seams to be very important, because in many planning problems, the constraints arising from resource availability determines a satisfactory solution to the planning problem. Temporal and resource availability constraints play a crucial role in our system as well. The distributed planning system discussed in this paper is currently implemented as a full-scale prototype. The protocol of multistage negotiation is under investigation. Planning agents from the student's point of view serve as his personal assistants that assist him to plan and schedule his learning activities. They form a part of the multiagent environment for individual and collaborative learning with artificial learning companions, personal learning assistants with information filtering capabilities, and agents supporting experi-

mentation activities, which is under development. Cooperation of planning MAS with personal learning assistants is under consideration. We are also investigating computational efficiency of the proposed algorithm for different planning tasks.

Bibliography 1.

Barros Costa, E., Perkusich, A. Modeling the Cooperative Interaction in a Teaching/Learning Situation. Intelligent Tutoring Systems, 1996: 168-176 2. Chan, T.W. Learning companion Systems, Social Learning Systems, and Intelligent Virtual Classroom, In Proc. of the World Congress on AI in Education, 1996, Vol. 7, No. 2, 125-159. 3. Chu-Carroll, J. & Carberry, S., Conflict Detection and Resolution in Collaborative Planning, Intelligent Agents II, Lecture Notes in Artificial Intelligent 1037, Springer Verlag, 1995. 4. Computing as a Discipline, ACM report, 1991. 5. Conry, S. E., Meyer, R. A., & Lesser, V. R. Multistage Negotiation in Distributed Planning, In Bond, A. H. and Gasser, L., (eds.) Readings in Distributed Artificial Intelligence. Morgan Kaufmann Publishers: San Mateo, CA, 1988, pp. 367-384. 6. Cormen, T., Leiserson Ch., & Rivest R., Introduction to Algorithms, McGraw Hill, 1990. 7. Davis, R & Smith, R. G. Negotiation as a Metaphor for Distributed Problem Solving, Artificial Intelligence, vol. 20, no. 1, 1983, pp. 63-109. 8. Fisher, K., Muller, J., Heimig, I., & Scheer, A-W., Intelligent Agents in Virtual Enterprises. In Proc. of the First Intern. Conference on The Practical Application of Intelligent Agents and Multi-Agent Technology, London, 1996, pp.205-224. 9. Gordon, A. and Hall, L. Collaboration with Agents in a Virtual World. In Proc. of the Workshop on Current Trends and Artificial Intelligence in Education, 4 World Congress on Expert Systems, Mexico, 1998. 10. Gorodetski, V. 1997, Basic Ideas of Agent Behaviour Coordination and Auction - based Scheduling Model.. In Proc. of the I Int. Workshop Distributed Artificial Intelligence and Multi-agent Systems, St. Petersburg, pp.282291. 11. Harris, D.A., Online Distance Education in the United States, IEEE Communications Magazine, March, 1999, pp. 87-91. 12. JATLite Beta Complete Documentation, Stanford University, 1998, http://java.stanford.edu

13. Jennings, N.R., Paratin, P., & Jonson, M., Using Intelligent Agents to Manage Business Processes. In Proc. of the First International Conference and Exhibition The Practical Application of Intelligent Agents and MultiAgent Technology, London, UK, 1996, pp.345- 376. 14. Kayama, M. and Okamoto, T. A., Mechanism for Knowledge-Navigation in Hyperspace with Neural Networks to Support Exploring activities. In Proc. of the Workshop on Current Trends and Artificial Intelligence in Education, 4 World Congress on Expert Systems, Mexico, 1998. 15. Liu J.-Sh. & Sycara, K. Multiagent Coordination in Tightly Coupled Task Scheduling, In Proc. of the First Int. Conf. On Multiagent Systems, 1996, pp. 181-188. 16. Mark, M.A., and Greer, J.E. The VCR tutor: Effective Instruction for Device Operation. Journal of the Learning Sciences, 1995, 4(2):209-246. 17. Müller, H. J., Negotiation Principles, In O'Hare G.M.P., & Jennings, N.R. (Eds.) Foundations of Distributed Artificial Intelligence, J Wiley & Sons, 1996, pp.21118. 230. Nuñez, G., Sheremetov, L., Martínez, J., Guzmán, A., & Albornoz, A. The Eva Teleteaching Project - the Concept and the First Experience in the Development of Virtual Learning Spaces. In Gordon Davies (ed.) Teleteaching'98 Distance Learning, Training and Education: Proceedings of the 15th IFIP World Computer Congress, Vienna and Budapest, 1998, P. II, pp. 769778. 19. Rao, A. & Georgeff, M., BDI Agents: From Theory to Practice, In Proceedings of the First International Conference on Multi-Agent Systems (ICMAS-95), San Francisco, USA, 1995. 20. Rosenchein, J. and Zlotkin, G., Rules of Encounter Designing Contentions for Automated Negotiation among Computers, MIT Press, 1994. 21. Sandholm, T.W., An Implementation of Contract Net Protocol Based on Marginal Cost Calculations, In Proceedings of 11th AAAI, 1993, pp.256-262. 22. Sandholm, T.W., Negotiation among Self-interested Computationally Limited Agents. Ph.D. Thesis, 1996. http:// www.cs.wustl.edu/~sandholm/dissertation. 23. Smirnov, A.V. and Sheremetov, L.B. Complex Systems Configuring Based on Multi-agent Technology. Automatic Control and Computer Sciences, Allerton Press, N.Y. 1998. 24. Smith R. G., The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver, IEEE Transactions on Computers C-29(12), 1980, pp. 1104-1113. 25. Youngblut, Ch. Government - Sponsored Research and Development Efforts in the Area of Intelligent Tutoring Systems. Institute for Defense Analyses, USA, 1994.