Planning in Multiagent Systems
Goal of this tutorial
Mathijs de Weerdt and Cees Witteveen
• Overview of • multiagent planning problems • multiagent planning techniques
EASSS, May 8, 2008 1 EASSS, May 8, 2008
Faculty of Electrical Engineering, Mathematics and Computer Science, Department of Software Technology
2
CWI, Amsterdam, Sen-4, Han la Poutré
Tutorial Contents
Outline
Part I: 1. 2. 3.
• Introduction to multiagent planning • Relation with multiagent systems • Relation with planning
(Mathijs) Introduction Taxonomy of problems Taxonomy of techniques
• Taxonomy of multiagent planning problems
Part II: Plan Coordination Mechanisms and their Evaluation (Cees) 1. Overview of plan coordination in MAP 2. Design and evaluation of coordination methods 3. Applications of coordination methods EASSS, May 8, 2008
• Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion 3
EASSS, May 8, 2008
Planning problem
Additional difficulties
• How to get from the current state to your goal state?
Because the world is … • Dynamic • Stochastic • Partially observable
• Planning involves • Action selection • Action sequencing • Resource handling
And because actions • take time • have continuous effects • involve multiple agents!
• Plans can be • Action sequences • Policies/strategies (action trees)
EASSS, May 8, 2008
4
5
EASSS, May 8, 2008
6
Why planning in multiagent systems?
Multiagent Planning (MAP) and Multiagent Systems (MAS)
• More efficient system performance on run-time • Come prepared • Prevent deadlock • Lower costs • Accomplish task more quickly
• MAS: Coordination of autonomous entities • MAP ⊆ MAS (hence this tutorial) • MAP: Focus on • Coordination of actions before execution • Finding correct actions to attain goals
• Useful assignment of resources, use of capabilities
• Strongly related to task allocation and auction/negotiation techniques EASSS, May 8, 2008
8
UN Operations
MAP and AI planning
Applications of Multiagent Planning
• MAP ⊇ AI Planning for multiple agents • Execution in parallel (instead of sequentially) → Parallel plans
• Planetary explorations
• MAP ⊇ Planning by multiple agents (distributed) • Incoherent plans: need for coordination; more difficult, less optimal • Why then? • Privacy & autonomy • Local → more efficient reaction on incidents (when communication limited) & no central point of failure • Speed-up: in parallel & smaller problems
• Taxi companies
EASSS, May 8, 2008
• Multi-player video games
9
Planetary explorations
EASSS, May 8, 2008
Multi-player video games
• More or less independent
• Self-interested players
• Communication difficult and costly
• Independent
• Cooperative
• Potential for coordinated behavior
EASSS, May 8, 2008
10
11
EASSS, May 8, 2008
12
Taxi companies
Outline • Introduction to multiagent planning • Relation with multiagent systems • Relation with planning • Taxonomy of multiagent planning problems • Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion
EASSS, May 8, 2008
13
EASSS, May 8, 2008
14
Strongly related → Loosely coupled → Independent
Taxonomy of MAP problems
Strongly related because
Four ways to look at multiagent planning problems
• Joint actions • Limited shared resources
• Strongly related → Independent
PhD research
Requires crisp coordination
• Cooperative → Self-interested • (Resolving conflicts → Exploit efficiency)
Examples
• No communication → Reliable communication
EASSS, May 8, 2008
15
• • • • •
Lift a box together Car assembly Robocup Hospital PhD research Car assembly
Cooperative → Self-interested Independent
PhD research Rovers
UN-operation Hospital
Robocup Strongly related
EASSS, May 8, 2008
Resolving conflicts → Exploit efficiency
Lifting boxes Cooperative
Independent
Warcraft
Warcraft Taxi companies
Hospital
Human soccer Supply chains
Self-Interested / Private 17
PhD research
Traffic
Traffic
Strongly related
EASSS, May 8, 2008
Lifting boxes Resolve conflicts
Rovers Taxi companies
Robocup Supply chains
Both
Exploit efficiency 18
Continuous planning and execution
No communication → reliable communication
• Any system that is used should deal with unexpected events • Plan adaption (repair) or • Start from scratch (replanning)
Examples: • Rescue Robots • Planetary explorations • Military operations
EASSS, May 8, 2008
• Multi-agent planning can help here by locally replanning / plan repair • when no communication possible • to reduce the impact of incidents 19
EASSS, May 8, 2008
Outline
Aim of AI
• Introduction to multiagent planning • Relation with multiagent systems • Relation with planning
• “Intelligent” systems, decide themselves • what to do, and • how to do it.
• Taxonomy of multiagent planning problems
• Planning is… • given what to do (goals), • determine how (and when) to do it (plan).
• Classical Planning • Taxonomy of multiagent planning techniques
20
• Currently planning research community very active • Significant scale-up • Bi-annual planning competition
• Recent focus • Discussion EASSS, May 8, 2008
21
AI Planning background
EASSS, May 8, 2008
22
Classical Planning
• Focus on classical planning; assume none of the above • Deterministic, static, fully observable • “Basic” • Most of the recent progress • Ideas often also useful for more complex problems
• Classical planning • Model • STRIPS • Refinement planning framework • Partial plans • Plan space refinement • HTN planning • Complexity of planning
EASSS, May 8, 2008
23
EASSS, May 8, 2008
24
Classical planning model
Planning is searching
• Origins: • STanford Research Institute Problem Solver (’71) • derived from GPS = human problem solving (’61)
…in state space (or…)
I want to be here! or hereor here
Choose between possible actions • Depth-first • Breadth-first
• States described by propositions currently true
goal states
• Actions: general state transformations described by sets of pre- and post-conditions ?
• Represents a state-transition system (but more compact) EASSS, May 8, 2008
I’m here!
a plan is a route in state space
25
initial state
Searching for a plan
STRIPS Formalism (now in PDDL)
start from initial state, try all possible actions
• action: preconditions, add, delete effects
→ large search space
• pickup(B1, B2) • precondition: empty & clear(B1) &
STRIPS: regression: look at goal state!
on(B1, B2)
• add-effect: holding(B1), clear(B2) • delete-effect: empty, on(B1, B2),
clear(B1)
• problem: initial state, actions/operators, goal description • objects and variables EASSS, May 8, 2008
28
I’m here!
STRIPS algorithm
STRIPS demo
STRIPS( s, g )
(by CI – space, British Columbia (CA))
returns: a sequence of actions that transforms s into g
Action
Preconditions
Add List
Delete List
1. Calculate the difference set d=g-s. 1. If d is empty, return an empty plan
pickup(B1, B2)
empty & clear(B1) & on(B1, B2)
holding(B1), clear(B2)
empty, on(B1, B2), clear(B1)
2. Choose action a whose add-list has most formulas contained in g
pickuptable(B)
empty & clear(B) & ontable(B)
holding(B)
empty, ontable(B), clear(B)
3. p’ = STRIPS( s, precondition of a )
putdown(B1, B2)
holding(B1) & clear(B2)
empty, on(B1, B2), clear(B1)
clear(B2), holding(B1)
4. Compute the new state s’ by applying p’ and a to s.
putdowntable(B)
holding(B)
empty, ontable(B), clear(B)
holding(B)
5. p = STRIPS( s’, g ) 6. return p’;a;p
g p s’ a p’ s
STRIPS planner
Classical planning in a multiagent setting
• Questions? • Optimal? • Sound? • Complete?
• When an agent is unable to do a task • Give task to other • Other: adapt current plan → plan repair Van der Krogt (2005)
EASSS, May 8, 2008
31
EASSS, May 8, 2008
Refinement planning framework
Refinement planning template
• Framework to capture all planning algorithms
Refineplan( P : Plan set)
• Idea: Narrow set P of potential action sequences
1.
If P is empty, Fail.
• Subcontents: • Specify action sequences by partial plans • Refinement strategies • Generic template
2.
If a minimal candidate of P is a solution, return it. End
3.
Select a refinement strategy R
4.
Apply R to P to get a new plan set P’
5.
Call Refineplan(P’ )
32
Termination ensured if R complete and monotonic. EASSS, May 8, 2008
33
EASSS, May 8, 2008
34
Existing Refinement Strategies
Plan space refinement (I)
• State space refinement: e.g. STRIPS
• Least commitment planning (Weld, 94) • search in plan space instead of state space • represent plans more flexible: not a sequence, but a partially ordered set • keep track of decisions and the reasons for these decisions
• Plan space refinement: e.g. Least commitment planning • Task refinement: e.g. HTN
EASSS, May 8, 2008
35
EASSS, May 8, 2008
36
Partial Plans: Syntax Partial plan = (Actions, partial Ordering, causal Links) - causal links = Interval preservation constraint (IPC) (a1 , p , a2) - p must be preserved between a1 and a2
Plan space A state in the (plan) search space is a partial plan (instead of a description of the state of the world)
a4 (a3, q, a4) a2 a3 a1
EASSS, May 8, 2008
37
My plan: I’m here!
POP { goal description }
• Deal with one (g,a) at a time
a1: move b2 from table to b3
b2
¬clear(b2)
a2: move b1 from table to b2
b3
clear(b2) { initial state }
EASSS, May 8, 2008
b3 b1
a0
b2
{ on(b3,b1), clear(b2), on(b1,table), on(b2, table) }
a0 39
POP( (A,O,L), agenda )
EASSS, May 8, 2008
40
Tradeoffs among Refinements
1.
Termination: if agenda is empty return (A,O,L)
2.
Goal selection: select a (g,aneed) from the agenda
3.
Action selection: choose an action aadd that adds g. Update L, O, and A
4.
Update goal set: remove (g,aneed) from agenda, and add its preconditions (…, aadd)
5.
Causal link protection: for every action at that might threaten a link, add an ordering constraint
EASSS, May 8, 2008
{on(b1,b2), on(b2,b3)}
b1
a∞ • Start with • null (empty) plan • agenda = list of (precondition, action) goals = {(g1, a∞), (g2, a∞), (g3, a∞), …}
a∞
POP example
41
State space refinement: • commit to both order and relevance of actions • include state information (easier plan validation) • lead to premature commitment • too many states when actions have durations
EASSS, May 8, 2008
Plan-space refinement: • commit to actions, avoid constraining order • increase plan-validation costs • reduce commitment (large candidate set /branch) • easily extendible to actions with duration
42
HTN Planning
Partial plans in a multiagent setting
task
travel(x,y)
travel by taxi
OR
• Broadcast (abstraction of) part of your plans relevant for others → “partial global plan”
AND
• Keep updating this global plan until nothing changes
airport(x,a) airport(y,b) ticket (a,b) travel (x,a) fly(a,b) travel(b,y)
method
travel by air
travel(Delft, Cap.) airport(Delft,AMS) airport(Cap.,LIS) ticket(AMS,LIS) travel(Delft, AMS) get taxi ride taxi(Delft, AMS) pay driver fly(AMS, LIS) travel(LIS, Cap.) get taxi ride taxi(LIS, Cap.) pay driver
Problem reduction
Generalized Partial Global Planning (Durfee, Decker, Lesser, et al.)
EASSS, May 8, 2008
43
•
Decompose tasks into subtasks
•
Handle constraints (e.g., taxi not good for long distances)
•
Resolve interactions (e.g., take taxi early enough to catch plane)
•
If necessary, backtrack and try other decompositions
AND
get taxi ride taxi (x,y) pay driver
Hierarchical multiagent planning
Complexity of planning
• Task structure for all involved agents • QAFs • SUM (~ AND), SyncSUM • MAX (~ OR), MIN • Non-local effects • Enables, Disables • Facilitates, Hinders
• PSPACE-completeness (NP⊆PSPACE) • PLANEX: plan existence problem is PSPACE-complete for propositional STRIPS • Restrictions on preconditions allowed and on effects, helps to reduce complexity
• TAEMS (Decker), c_TAEMS for Coordinators (Boddy et al.) • Zlot and Stentz (2006) EASSS, May 8, 2008
Complexity of planning
45
•
plan length is at most l=2n
•
hardness: each PSPACE problem can be translated by making Turing-machine operations into actions
•
plan existence with non-deterministic Turing machine in O( log l ) space (polynomial in n) plan(s,g) 1. if s=g return true 2. guess intermediate state s’ 3. return plan(s,s’) and plan(s’,g)
EASSS, May 8, 2008
46
Complexity of planning
complete
PSPACE PLANEX is PSPACE-complete
EASSS, May 8, 2008
pre: preconditions
complete
* pre * effs
effs: effects
NP
* pre 1 effs
P
1 pre * effs
PSPACE-complete
2 +pre 2 effs
NP-complete
* pre * +effs 1 pre 1 effs
O(log l)
Polynomial 47
1 +pre 1 effs
1 pre * effs k goals
0 pre * effs
Future work
Recommended reading
Apply other planning techniques to multiagent planning: • Non-deterministic actions (surprises) • Partially observable world (exact costs unknown) • Durative actions (move, travel) and continuous variables (capacity, fuel, distance, time) → optimality
• Kambhampati, S. (1997). Refinement planning as a unifying framework for plan synthesis. AI Magazine, 18(2):67-97.
EASSS, May 8, 2008
49
Other references •
• Ghallab, M., Nau, D., Traverso, P. (2004). Automated Planning: Theory and Practice, Elsevier. • Vlahavas I., Vrakas, D. (2005). Intelligent Techniques for Planning, Idea Group.
EASSS, May 8, 2008
50
Outline
Boddy, M., Horling, B., Phelps, J., Goldman, R. Vincent, C. Long and B. Kohout, C_TAEMS Language Specification, Version 1.06. Decker, K. S. and Lesser, V. R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent and Cooperative Information Systems, 1(2):319-346. Decker, K. (1996). TAEMS: A Framework for Environment Centered Analysis & Design of Coordination Mechanisms. Foundations of Distributed Artificial Intelligence, Chapter 16, G. O'Hare and N. Jennings (eds.), Wiley Inter-Science, pp. 429-448.
• Introduction to multiagent planning • Relation with multiagent systems • Relation with planning
•
Fikes, R. E. and Nilsson, N. (1971). STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 5(2):189-208.
• Taxonomy of multiagent planning problems
• •
Newell, A. and Simon, H. (1963). GPS: A program that simulates human thought. In Feigenbaum, E. and Feldman, J., editors, Computers and Thought, pages 279-296. Van der Krogt, R. and de Weerdt, M. (2005). Coordination through Plan Repair. In Gelbukh and de Albornoz and Terashima-Marin (Eds.). MICAI 2005: Advances in Artificial Intelligence, pages 264-274. LNAI 3789.
•
Weld, D. S. (1994). An introduction to least-commitment planning. AI Magazine, 15(4):27-61.
•
Zlot, R. M. and Stentz, A. (2006). Market-based multirobot coordination for complex tasks. International Journal of Robotics Research, Special Issue on the 4th International Conference on Field and Service Robotics, 25(1):73-101.
• •
EASSS, May 8, 2008
51
• Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion EASSS, May 8, 2008
52
Taxonomy of MAP techniques
Where are plans created?
• Where are plans created? • Centralized • Distributed
Algorithm: Centralized • Optimal (potentially) 1. Label actions with agent • Communication only names twice (before & after) 2. Plan
• When coordination in MAP process? • Before planning • During planning • Post-planning
3. Decompose into subplans
• How are plans coordinated? • Dependency checks? • Task/resource allocation? EASSS, May 8, 2008
4. Add synchronization actions 53
EASSS, May 8, 2008
54
Where are plans created?
Where are plans created?
Examples: Distributed • Ephrati (1995): by • Reduce computation plan merging time • Keep privacy (potentially) • vd Krogt (2005): by plan repair • Scalable • Execution and control are also distributed
Partially distributed • Share parts of your plan
EASSS, May 8, 2008
• Durfee, Decker, Lesser (1986-) Partial global planning (shared plan) 55
Where are plans created? Distributed, for a centralized plan • Specialized planning agents
EASSS, May 8, 2008
56
When coordination in MAP process?
Examples: • Kambhampati (1991): Combining specialized reasoners and planning
1. Global task refinement = AI planning 2. Task allocation 3. Coordination before planning 4. Individual planning
• Wilkins (1998): Multiagent planning architecture EASSS, May 8, 2008
Examples: • Corkill (1979): Distributed NOAH (shared world model)
5. Coordination after planning 6. Plan execution 57
EASSS, May 8, 2008
When coordination in MAP process? (pre)
When coordination in MAP process? (during)
3. Coordination before planning (pre-planning coordination) • Social laws (e.g. traffic rules): Shoham & Tennenholtz (1992) • Derive specific constraints for agents: Buzing, Valk et al. (2006)
4. Individual planning Coordination during planning • Distributed NOAH (Corkill, 1979) • Partial global planning (Decker, 1992) • Through plan repair (vd Krogt, 2005)
EASSS, May 8, 2008
EASSS, May 8, 2008
59
58
60
When coordination in MAP process? (post)
When coordination in MAP process? (continual)
5. Coordination after planning • Plan merging (Ephrati, 1993)
• Plans are being executed during planning and coordination • May break and re-make commitments • unexpected event/failure • goal change
6. Coordination during (plan) execution • When communication is unreliable (Koes, 2006) • “Normal” MAS/distributed systems solutions • FIFO-queues • Semaphores
EASSS, May 8, 2008
(DesJardins, 2000)
Distributed continual planning Task allocation During-planning Execution Task refinement Pre-planning Post-planning coordination coordination 61
EASSS, May 8, 2008
Difficulties of continual planning
How are plans coordinated?
• Chain reactions of changes
• Maintenance of dependencies? • Distributed check for cycles • Shared global plan • Afterwards: plan merging
• Cyclic dependencies ?
62
• Distribution of tasks and resources? • Contract net protocol • Auctions (combinatorial?)
EASSS, May 8, 2008
63
EASSS, May 8, 2008
Taxonomy of MAP
Outline
Problems: • Strongly related → Independent • Cooperative → Self-interested • (Resolving conflicts → Exploit efficiency) • No communication → Reliable communication
• Introduction to multiagent planning • Relation with multiagent systems • Relation with planning • Taxonomy of multiagent planning problems
Techniques: • Where are plans created? • Centralized → Distributed • When coordination in MAP process? • Before planning → During planning → Post-planning • How are plans coordinated? • Dependency checks? • Task/resource allocation? EASSS, May 8, 2008
64
• Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion 65
EASSS, May 8, 2008
66
Recent focus
Self-interested planning agents
• Mixed initiative multiagent planning • using domain knowledge (in task networks)
Problem setting • Planning routes for internet packages along selfinterested routers
• More application-oriented work: • search & rescue • military operations (“Coordinators”) • logistics
• Coordinating transport companies in a supply chain • As a government, set up a market for coordinated medical treatments, or public transport
• Self-interested agents • Probabilistic planning: Markov-Decision Processes
→ mechanism design!
• Distributed Constraint Optimization EASSS, May 8, 2008
67
EASSS, May 8, 2008
68
Mechanism design
Multiagent planning as a social choice
• Sub-field of economic theory from engineering perspective
• Given a set of n agents that each have their own goals and actions. Find a multiagent plan that is “good” for every agent (Pareto-efficient).
• Assuming agents are rational, find a way to obtain a social choice (joint decision), eg • elections • markets • auctions • government policy • Apply the theory…
Q. Is it possible to find a mechanism to solve this problem for selfinterested agents? How/why not? A. No. Gibbard-Satterthwaite (’73) say: • Agents will have incentive to lie about preferences (or operations), or • social choice will be a dictatorship. Maybe introduce money or tax to incentivize agents…
EASSS, May 8, 2008
69
EASSS, May 8, 2008
Mechanisms with money
Mechanisms with money
• Def: Vickrey-Clarke(’71)-Groves(’73) VCG mechanism: assign payments p1,…,pn to each agent such that • f maximizes social welfare: f(v1,…,vn) ∈argmaxa∈A’Σivi(a) • payment pi does not depend on vi and is related to the sum of values for all other players: pi(v1,…,vn) =-Σj ≠ivj(f(v1,…,vn)) + hi(v-i)
Goal. Find a multiagent plan that optimizes Σivi. Q. Is it possible to find a mechanism to solve this problem for selfinterested agents (if vi is private information) ? How/why not? A. Yes, using a VCG-based mechanism, but: • we retrieve some “tax”; what to do with that? • social choice must be optimal, general planning is PSPACEcomplete and if not optimal then cost minimization allocation problem with possibly very bad results • lying about actions may lead to infeasible solutions: another penalty afterwards?
• VCG theorem: every VCG mechanism is incentive compatible. EASSS, May 8, 2008
70
71
EASSS, May 8, 2008
72
Problems, problems, …
Recommended reading
• How to determine the valuations for the agents when planning is PSPACE-complete? May also depend upon other agents’ utilities (inter-dependent utilities)
• Noam Nisan (2007). Ch. Introduction to Mechanism Design (for computer scientists) in Algorithmic Game
Theory.
• Multiagent planning problem has often on-line context • Sometimes we cannot use a central, but need a distributed mechanism
EASSS, May 8, 2008
73
Other references •
K. Arrow. Social Choice and Individual Values. Yale University Press, 1951. E. H. Clarke. Multipart pricing of public goods. Public Choice, pages 17–33, 1971.
•
Allan Gibbard. Manipulation of voting schemes: a general result. Econometrica, 41:587–601, 1973.
•
T. Groves. Incentives in teams. Econometrica, pages 617–631, 1973. Philippe Jehiel and Benny Moldovanu. "Efficient Design with Interdependent Valuations," Econometrica, Econometric Society, vol. 69(5), pages 1237-59, 2001.
•
David Parkes. Ch. Online Mechanism Design in Algorithmic Game Theory, 2007.
•
K. Roberts. The characterization of implementable choice rules. In Aggregation and Revelation of Preferences, J-J. Laffont (ed.), North Holland Publishing Company., 1979.
•
Mark Allen Satterthwaite. Strategy-proofness and Arrow’s condition: Existence and correspondence theorems for voting procedures and social welfare functions. Journal of Economic Theory, pages 187–217, 1975.
•
W. Vickrey. Counterspeculation, auctions and competitive sealed tenders. Journal of Finance, pages 8–37, 1961.
EASSS, May 8, 2008
• Privacy & autonomy • First negotiating, then (optimal) planning • No central point of failure • Just introduce redundancy • Limited communication (on execution) • Contingent planning on forehand & with constraints on-line • Speed-up: in parallel & smaller problems • Use a grid and parallel computing techniques
75
Recommended reading
EASSS, May 8, 2008
76
Other references • •
• Durfee’s chapter (3) on Distributed Problem Solving and Planning in Weiss (1999), Multiagent systems: A modern approach to
• •
Distributed Artificial Intelligence
• DesJardins, M. E., Durfee, E. H., Ortiz, C. L., and Wolverton, M. J. (2000). A survey of research in distributed, continual planning. AI Magazine, 20(4):13-20. • (Links to) material can be found on: http://www.st.ewi.tudelft.nl/~mathijs/tutorial.php
• • • • •
EASSS, May 8, 2008
74
Discussion: Reasons for not doing multiagent planning
•
•
EASSS, May 8, 2008
77
Buzing, P., ter Mors, A., Valk, J. and Witteveen, C., Coordinating Self-interested Planning Agents, Autonomous Agents and Multi-Agent Systems 12(2):199-218. Corkill, D. (1979), Hierarchical Planning in a Distributed Environment. In Proceedings of the Seventh International Joint Conference on Artificial Intelligence, pages 168-175. Decker, K. S. and Lesser, V. R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent and Cooperative Information Systems, 1(2):319-346. Ephrati, E. and Rosenschein, J. S. (1993). Multi-agent planning as the process of merging distributed sub-plans. In Proceedings of the Twelfth International Workshop on Distributed Artificial Intelligence (DAI-93), pages 115-129. Kambhampati, S., Cutkosky, M., Tenenbaum, M., and Lee, S. (1991). Combining specialized reasoners and general purpose planners: A case study. In Proceedings of AAAI-91, pages 199-205. Koes, M., K. Sycara, and I. Nourbakhsh (2006), A Constraint Optimization Framework for Fractured Robot Teams. In AAMAS. Shoham, Y. and Tennenholtz, M. (1995). On social laws for artificial agent societies: Off-line design. Artificial Intelligence, 73(1-2):231-252. van der Krogt, R. and de Weerdt, M. (2005). Coordination through Plan Repair. In Gelbukh and de Albornoz and Terashima-Marin (Eds.). MICAI 2005: Advances in Artificial Intelligence, pages 264-274. LNAI 3789. Wilkins, D. and Myers, K. (1998). A multiagent planning architecture. In Proc. of the 4th Int. Conf. on AI Planning Systems (AIPS), pages 154-162.
EASSS, May 8, 2008
78
Plan Coordination Mechanisms and their evaluation
Cees Witteveen Faculty EEMCS, Delft University of Technology
[email protected] Plan Coordination and Autonomy
Planning in Multi Agent Systems Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
1
2
disasters
disasters
coordination in preparation
What’s the difference coordination in action Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
team sport
3
overview of the talk (i)
4
overview of the talk (ii) Part II
Part I
we discuss an application to multi-modal transportation planning and show that in some cases - !optimal coordination can be approximated very efficiently - !the coordination method can be used to reuse single agent ! planning technology in a multi-agent context
we present an overview of plan coordination in MAP discussing factors relevant to coordination and types of coordination in planning we concentrate on the design of coordination methods suitable for selfish agents that are not able or willing to revise their autonomously developed plans
next, we show how aspects of time can be incorporated by discussing a coordination method ensuring autonomous temporal planning autonomy. finally, we discuss an application to taxi-routing on airports discussing a sequential reservation-based planning approach offering agents to plan autonomously given the knowledge about resource usage of other agents
to evaluate the quality of coordination methods we discuss the price of autonomy measuring the quality loss of planning using autonomous planning. Plan Coordination and Autonomy
EASSS 2008
team sport
Plan Coordination and Autonomy
EASSS 2008
5
EASSS 2008
6
problems we want to deal with airport planning
Part I
planning systems for arrival, departure, gate assignment, ground handling taxi-route planning
overview of plan coordination in
patient planning planning systems for treatments of patients, making reservations for resources
multi-agent planning
multi-modal logistics planning planning systems for traveling & transportation using different transportation modalities Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
7
common characteristics
8
common characteristics
catering
several autonomous planners each have to perform a disjoint set of tasks
arrive ! taxi ! dock ! cleaning ! taxi ! depart fueling
tasks might be interdependent (within and across agents) intake ! specialist1! scan ! specialist2 ! surgery
to avoid conflicts each agent has to construct a plan for its set of tasks. each plan has to satisfy the given ordering constraints and might impose new constraints.
truck - train - plane - truck - ship - truck
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
9
10
Plan coordination : preliminaries
common characteristics
First: Preliminary definition
several autonomous planners each have to perform a disjoint set of tasks
plan coordination refers to a collection of methods to obtain a set of jointly feasible plans for a set of goals, given agents who
• •
tasks might be interdependent (within and across agents)
want to achieve a specific subset of these goals by making plans individually
REQUIRES: PLAN COORDINATION to avoid conflicts each agent has to construct a plan for its set of tasks.
Next: Preliminary questions
each plan has to satisfy the given ordering constraints and might impose new constraints.
Plan Coordination and Autonomy
•
what are the main characteristics of the Multi-Agent Planning (MAP) process relevant to coordination?
•
what are the main types of plan coordination given this MAP process?
Plan Coordination and Autonomy
EASSS 2008
11
EASSS 2008
12
Multi Agent Planning: process view tasks
Multi Agent Planning: process view
agents
tasks
agents
tasks consisting of interrelated subtasks and specifications of capabilities
set of agents together with description of capabilities.
task allocation
task allocation task allocation: process of assigning tasks to agents such that capabilities offered by agents are sufficient for capabilities required by tasks.
planning
planning planning by individual agents; result should be a feasible joint plan
environment
environment plan execution
Plan Coordination and Autonomy
plan execution
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
13
factors influencing coordination properties dependencies
Properties of Tasks
capabilities relationships
tasks
14
•
loosely coupled
•
moderately coupled
•
tightly coupled
agents
environment
no (dependency) relations between subtasks assigned to different agents
dependency relations between subtasks assigned to different agents
relations between tasks that require close interaction during execution of the plan
resource availability Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
15
Properties of Environment
Properties of Agents •
cooperative agents are willing to revise and to adapt their plans to the plans of other agents
•
willing/able to communicate
•
self-interested requiring planning autonomy
agents are willing and able to exchange plan information
agents want to plan autonomously, are not willing/able to share information or to revise plans upon request.
Plan Coordination and Autonomy
16
•
resource availability
•
execution interaction
•
stability
do the agents have to compete for resource usage or not
does the execution of tasks negatively (hinders) or positively (facilitates) influence the execution of other tasks
is the environment stable or incident prone and dynamic.
Plan Coordination and Autonomy
EASSS 2008
17
EASSS 2008
18
Plan Coordination methods tasks
Coordination methods
agents
tasks
agents
coordination before planning task allocation
task allocation
coordination during planning
planning
environment
planning
environment plan execution
Plan Coordination and Autonomy
plan execution
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
19
Coordination methods tasks
20
coordination: types, goals, methods
agents
coordination type
goals
methods
after planning
conflict resolution
plan merging plan revision
during planning
conflict resolution conflict prevention
plan communication plan revision
before planning
conflict prevention
constraint management
task allocation
planning
coordination after planning
environment plan execution
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
21
22
coordination: types, goals, methods coordination type
goals
methods
after planning
conflict resolution
plan merging plan revision
during planning
conflict resolution conflict prevention
plan communication plan revision
before planning
conflict prevention
constraint management
Plan Coordination and Autonomy
Part I
designing plan coordination mechanisms
Plan Coordination and Autonomy
EASSS 2008
23
EASSS 2008
24
a simple construction example
problem: general statement
There are two agents A1 and A2 involved in a construction task. A1 has to deliver bricks to A2, who has to use them to build a wall. A2 has to ensure that garbage will be collected and has to deliver it to A1 who will collect it and bring it to a dumping ground.
Construct a coordinated plan for a set T of interdependent tasks distributed over a set of autonomous agents A, where the agents enjoy planning autonomy, that is the agents ! are free in making their own plans for their set of tasks
t1
! do not want or are not able to exchange information during the planning process ! are not willing to revise their plan in composing
A1
a feasible joint plan for the set of tasks
transport garbage
pickup garbage
Plan Coordination and Autonomy
task level
t2
deliver bricks
build wall
t6
A2
collect garbage
t4
t5
Plan Coordination and Autonomy
EASSS 2008
t3
deliver garbage
EASSS 2008
25
26
task and plan level 1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A 1
tasks and plans
PLAN of A2
t1
t2
deliver bricks
A1
transport garbage
pickup garbage
1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A1
1. take coffee; 5. take coffee; 2. build part wall; 6. collect garbage 3. take coffee; 7. deliver garbage 4. build part wall
t3
Plan Coordination and Autonomy
collect garbage
t2
deliver bricks
A2
A1
transport garbage
t4
t5
PLAN of A1
t1 build wall
t6
1. take coffee; 5. take coffee; 2. build part wall; 6. collect garbage 3. take coffee; 7. deliver garbage 4. build part wall
deliver garbage
pickup garbage
build wall
t6
A2
collect garbage
t4
t5
Plan Coordination and Autonomy
EASSS 2008
t3
deliver garbage
EASSS 2008
27
tasks and plans
28
tasks and plans
1. drive to construct- 5. load bricks 1. take coffee; 5. take coffee; ion place; 6. drive to construct2. build part wall; 6. collect garbage 2. pickup garbage ion place 3. take coffee; 7. deliver garbage 3. drive to dumping 7. deliver bricks relationship Task-Plan 4. build part wall place 4. take coffee A1 PLAN ofofA1 induceof partitioning on set plan steps • tasksPLAN
1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A1
• plan step ordering has to satisfy task ordering t1 t2 of wall deliver bricks • plan step ordering may induce refinement build
A1
task ordering.
1. take coffee; 5. take coffee; 2. build part wall; 6. collect garbage 3. take coffee; 7. deliver garbage 4. build part wall
PLAN of A1
INFEASIBLE JOINT PLAN t1
t2
deliver bricks
build wall
A2
additional constraints induced by plans
A1
transport garbage
pickup garbage
t6
t3
A2
transport garbage
t4
t5
Plan Coordination and Autonomy
collect garbage
deliver garbage
pickup garbage
t6
29
collect garbage
t4
t5
Plan Coordination and Autonomy
EASSS 2008
t3
deliver garbage
EASSS 2008
30
coordination: during / after
recapitulation: our framework partially ordered set of tasks
autonomous agents
A1
A2
1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A
where tasks require different capabilities
with different capabilities
A3 COORDINATION PROBLEM
1
Ensure that a feasible joint plan for the set of A2to tasks is produced if eachA1 agent wishes plan individually.
A1
A3
PLAN of A2
t1
t2
deliver bricks
transport garbage
build wall
t6
t3
A2
collect garbage
t4
t5
pickup garbage
task assigment
1. take coffee; 5. take coffee; 2. build part wall; 6. collect garbage 3. take coffee; 7. deliver garbage 4. build part wall
deliver garbage
dependencies between agents
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
31
32
coordination: during / after 1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A 1
A1
NEW PLAN of A2
t1
transport garbage
pickup garbage
t2 build wall
revision and communication required
t3
1. take coffee; 2. collect garbage; 3. take coffee; 4. build wall 5. deliver garbage
PLAN of A1 Coordination before planning?
A2
A1
t1
t2
deliver bricks
collect garbage
transport garbage
deliver garbage
pickup garbage
t4
t5
Plan Coordination and Autonomy
1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A1
1. take coffee; 2. collect garbage; 3. take coffee; 4. build wall 5. deliver garbage
deliver bricks
t6
coordination
build wall
t6
t3
collect garbage
t4
t5
deliver garbage
Plan Coordination and Autonomy
EASSS 2008
A2
EASSS 2008
33
coordination problem: definitions
34
example
example, just for illustration
Planning instance Is this instance coordinated ?
A plan coordination instance is a tuple ( [ T1, T2, . . ., Tk ] , < ) where each Ti is a set of tasks given to agent Ai. The tuple ( Ti, lab(x).
•
as a consequence, every inter-agent path is monotonically label-increasing => no inter-agent cycle is possible => plan coordination achieved.
traversing an intra-agent edge u < v, the label setting algorithm ensures lab(u) ! lab(v).
1
for all tasks t, t’ ! TA do if lab(t) < lab(t’) then add ( t < t’ ) to
example Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
47
EASSS 2008
48
Multi-modal transportation Part II
A1
A2
A3
A6
coordination design and price of autonomy in multi-modal planning
A5
A4
order is a sequence of simple transportation tasks.
cost of plan is measured by #moves + #(un)load actions
we are given n of those orders Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
49
designing coordination mechanism • •
50
the price of autonomy: theoretical results paut =
applying the label setting algorithm results in a nearly optimal coordination mechanism with approximation ratio " 2
sum cost optimal local transportation plans
=
cost optimal total transportation plan
1.14
We know that finding an optimal local transportation plan is difficult. There is a simple 2-approximation algorithm for these problems:
significance
peff aut =
approximation algorithm decomposes complex multi-agent planning problem into several independent single-agent planning problems: existing single agent planning technology can be re-used
sum cost approximated local transportation plans cost optimal total transportation plan
=
1.22
=
1.11
The best approximation algorithm for the optimal solution realizes: popt =
sum cost approximated total transportation plan cost optimal total transportation plan
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
51
52
plan cost comparison
paut : empirical results
500 We performed two types of experiments using an AIPS logistic benchmark set
375 250
plan cost
minimum plan STAN coordination TAL planner HSP
comparing the coordination approach (= coordination + local heuristics) with best state-of-the art planners.
125 comparing state-of-the art planners with state-of-the-art planners as single agent planners + the coordination mechanism.
0 20
30
40
50
60
nr of orders Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
53
70
EASSS 2008
54
using planners as single agent planners STAN time
100%
STAN #steps
HSP time
Autonomy vss Efficiency
HSP #steps
minimizing makespan and independent planning
80%
• suppose each subtask t in a complex task T j
has a given duration dj
savings
60%
• we are interested in both independent planning by
40%
the agents as well as minimization of the makespan of the plan
20%
results 0% 1
6
11
16
21
26
31
36
41
46
• minimal set of constraints can be computed efficiently while
51
garanteeing independent planning and minimal makespan
-20%
problem
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
55
56
Autonomy vss Efficiency idea compute time constraints for scheduling tasks of agents
Part II
• for every task t in T compute its depth d(t) and its height h(t) • set the following constraints (d(t), h(T) -h(t) +1)
coordination design
• for every t, t’ such that t < t’, remove overlap
in temporal planning
results
• every set of locally satisfiable schedules s(t) constitutes a globally minimal schedule
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
57
coordination in temporal planning •
•
how to deal with time windows
adding time information •
quantitative time information: time windows for tasks
•
qualitative time information: timing relations between tasks
representing time windows For every task ti a time window [ tlb , tup ] is provided.
problem to solve we have to ensure that for every set of individual plans chosen by the agents, a feasible joint schedule can be found
plan coordination
method
how to ensure that autonomous planning agents come up with independently chosen plans such that there always exist a feasible schedule for the joint plan satisfying the temporal constraints.
Plan Coordination and Autonomy
58
[19,45]
[20,50]
t1
t2
t3
t4
[19,55]
[18,55]
we reduce the problem to standard plan coordination by eleminating time windows. The set of time windows is represented by an additional agent executing “critical timepoint tasks”
Plan Coordination and Autonomy
EASSS 2008
59
EASSS 2008
60
removing time windows
qualitative time constraints
ATime 56
A1
A2
A1
A2
[19,45]
[20,50]
51
t1
t2
t1
t2
46
t3
t4
Every task t can be represented by a timed interval [ ts a, te ] of two tasks ts and te such that ts < te
•
Allen [1983] distinguished 13 basic relations between timed intervals ( 6 + 6 converses + 1 equals relation) t
19 t3
[18,49] [18,55]
[19,55]
•
t
t4
t’
18
t t’
precedes
meets
17
with temporal info
t’
overlaps
t
t
t
t
t’
t’
t’
t’
finished by
contains
starts
equals
without temporal info
Plan Coordination and Autonomy
Plan Coordination and Autonomy
EASSS 2008
EASSS 2008
61
converting time interval ts
results for qualitative time
te
t
precedence
t’
t’s
precedes ts
synchro nization
meets
t’s
Plan Coordination and Autonomy
t’e
•
all 13 time interval relations can be represented by precedence (
