To acquire robust AGH schedules, which respect the selfish ambitions of ..... The Selfish Gene: 30th Anniversary Edition â with a New Introduction by the Author.
Distribute the Selfish Ambitions Xiaoyu Mao ab
Nico Roos b
Alfons Salden a
a
Almende B.V., Westerstraat 50, 3016 DJ Rotterdam, The Netherlands b MICC, Universiteit Maastricht, P.O. Box 616, 6200 MD Maastricht, The Netherlands Abstract The distributed project scheduling problem is a distributed resource allocation problem extended with temporal constraints. Herein a project consists of a series of activities, the scarce resources are assigned to those activities at certain time windows such that all the activities are performed and certain objectives are optimized. In the past, methods from Operations Research (OR) and Artificial Intelligence (AI) were proposed to tackle this problem in a centralized manner. However, such centralized methods appear to be for many real-world applications impractical and infeasible due to the distribution of information, changes in willingness to cooperate and the decision autonomy of individual actors. Furthermore, long term schedules are likely to be invalidated by the dynamic changes in the environment. In this paper, we investigate this more dynamic problem in the context of Airport Ground Handling (AGH), where airlines and ground handlers are working together to deliver ground services during aircraft turnaround times. Although parties involved in AGH process cooperate, they may have their own selfish ambitions. For instance, in the scheduling process, airlines and ground service providers do not share a unified objective. In this paper, a multiagent online heuristic approach is proposed. We employ in this approach a market mechanism that balances the interests of airlines and ground service providers for scheduling the AGH services. The experimental results show that this multiagent approach provides a schedule that is of comparable quality as a centralized heuristic approach, and that by letting self-interested agents behave cooperatively, the overall schedule is significantly improved.
1
Introduction
The “on-ground” time of an aircraft at the airport starts with safe landing and finishes with next take-off. This period of time is called turnaround time in the aviation industry. During this turnaround time, a set of ground servicing tasks (e.g., passengers/baggage handling, aircraft maintenance, fuelling, cleaning and catering, etc.) must be performed. Ground handling involves very diverse tasks: some of them form a workflow (a sequence of services) with precedence constraints among each other, while others are independent service tasks. All such activities together form a so-called Activity-on-Node network [10]. Operations research defines scheduling the ground handling services of an individual aircraft as a resource-constrained project scheduling problem (RCPSP), in which the project refers to the aircraft turnaround and the activities to the ground services. The overall scheduling problem for the complete airport is refered to as Airport Ground Handling (AGH). As AGH deals with the scheduling of multiple aircraft ground services, this makes it a resource-constrained multi-project scheduling problem (RCMPSP) [3]. Both exact and heuristic centralized scheduling approaches to RCMPSP are proposed in OR and AI literature [8, 9, 11]. However, centralized approaches appear to be in many real-world applications impractical and infeasible when the decision has to be made by distributed and resource constrained actors. Assuming that these actors are independent economic units, it might be very well that these individual decision makers aim at optimizing their own objectives rather than the overall system performance. Such behaviours of actors call for project scheduling models and techniques that take the strategic behaviour of individual units into account [4]. For instance in AGH, ground handlers collaborate with airlines to deliver acceptable ground services during aircraft turnaround. On the one hand, during the process of scheduling ground service activities, airlines try to schedule their ground services as early as possible in order to minimize the turnaround
138
Xiaoyu Mao, Nico Roos and Alfons Salden times - therewith to handle more flights a day. On the other hand, ground service providers which perform the ground services strive for as low as possible resource usage variation during the day, such that they can keep resource investments as low as possible. The dynamic nature at an airport requires robust AGH schedules under various incidents. The actual arrival time of an aircraft is often different from the one on the original flight plan, either earlier or later caused by the uncertainties during the flight. Ground services may also take more time or less than expected. Under these dynamic features, the planned AGH schedule becomes obsolete, and requires re-scheduling or schedule-repair. An online scheduling approach reduces the risk of re-scheduling by starting the scheduling process of a project when it is released or the release time is approaching. To acquire robust AGH schedules, which respect the selfish ambitions of distributed stakeholders, we propose a multiagent solution consisting of distributed autonomous agents representing aircraft (project agents) and service providers (resource agents), respectively, that either cooperate or not during scheduling of AGH servicing tasks by negotiating the slot price for these tasks. Herein the aircraft and service provider agents take flow time and resource usage into account. Finally, we will compare the performance of these multiagent scheduling approaches (non-cooperative and cooperative) with some well known centralized heuristic approaches. The remainder of this paper is organized as follows. In Section 2, we define the problem of multiproject scheduling and discuss the objective functions to be optimized. Section 3 investigates the priority rule based heuristic methods for RCMPSP with different objectives. In Section 4 we present our distributed online heuristic approaches to solve RCMPSP based on slot-price negotiation amongst agents. Section 5 compares some experimental results obtained by our distributed agent heuristic method with the centralized OR heuristic methods. Section 6 concludes this work with a look to the future.
2
RCMPSP with different objectives
Many works published in the area of project scheduling make reference to the scheduling of a single project under resource limitations and time constraints. The scheduling problem we are investigating in AGH deals with multiple projects (i.e., aircraft turnarounds), mapping it to a RCMPSP [3]. A RCMPSP is a problem with M projects sharing K types of resources. Each project i consists of j = 1, . . . , Ni activities ai,j and has a release time tri . A precedence constraint ai,j ≺ ai,j 0 may exist between two activities in a same project. Two fictitious activities ai,0 and ai,Ni +1 are added for representing k the “start” and the “end” of project i. Each activity ai,j requires a processing time pi,j and ri,j amount of resource of type k ∈ K for its completion. Resource type k has a renewable capacity of Rk . The RCMPSP entails finding each activity a schedule si,j (i.e., the start time) that forms an overall schedule S = {si,j | 1 ≤ i ≤ M, 1 ≤ j ≤ Ni } satisfying both precedence and resource constraints and optimizing certain objectives. Often, time-based objectives are the criteria to be optimized for RCMPSP. Most time-based objectives are defined as functions f that are nondecreasing in the start times of activities, so-called regular objective functions [10]. An example of such objective functions - also considered in this paper - is the mean project flow time [8, 9]: fM P F T (S) =
M X si,N
i +1
i=1
M
− tri
(1)
where the start time of the last fictitious activity si,Ni +1 ∈ S corresponds to the completion time of project i. Minimizing this criterion is equivalent to minimizing the mean project delay time when a due time is set for each project based on the length of its resource-unconstrained critical path. However, non regular objective functions are needed in practice, for instance the resource-based objectives (e.g, resource investment problem, resource levelling problem, etc). Also in RCMPSP for AGH it is natural to investigate a resource levelling problem: a project scheduling problem where the objective is to minimize the resource usage variation over time. Although there are different measures for resource usage variation, we take the well known variation function called the total squared utilization cost [10]: X X fT SU C (S) = ck u2t,k (2) k∈K
t=0
Distribute the Selfish Ambitions
139
in which resource utilization cost ck > 0 is the cost incurred per unit of resource k ∈ K per unit of time, and ut,k is the utilization of resource k scheduled at discrete time point t.
3 Centralized heuristic methods for RCMPSP Realizing that most scheduling problems have to be done at a stage close to the practical operations, agility in decision making is the most important factor. Heuristics based on priority rules have been considered one of the most important solution techniques for large problem instances. These heuristics have the advantage of being intuitive, easy to implement, and fast due to low computational costs [1]. Here we present priority rule based heuristic approaches for RCMPSP with regard to two incompatible and potentially conflicting objective functions which are i) minimizing the mean project flow time (time-based objective) and ii) minimizing the total squared utilization cost (resource-based objective). Priority rule based heuristic methods are employed in a so-called schedule generation scheme (SGS). In general, SGS starts from scratch and builds a feasible schedule by stepwise extension of a partial schedule. At each step (also called generation), two decisions have to be made: i) selecting an activity from the eligible set1 , ii) deciding a start time for this chosen activity. The different decisions made concerning these two issues result in different schedules. In priority rule based heuristic methods, priority rules are applied to help resolve the first issue of selecting an activity. Subsequently, the heuristic used in determining the start time of the chosen activity critically depends on the objective functions employed. We will recall in the following subsections the best performing heuristics that can be used in either selecting an activity or deciding a start time.
3.1
Selecting an activity
The priority rules are used in order to select an activity from the eligible set in each generation of SGS. In the priority rule based heuristic approach, eligible activities are sorted by a priority rule, and the activity with the highest priority will be scheduled next. Priority rules for single project scheduling had been well studied in the past [5, 6]. When dealing with multiple projects, most of the priority rules for single project scheduling can still be used in an approach called single-project approach, which uses dummy activities and precedence arcs to combine the projects into a single mega-project, thereby reducing a RCMPSP to a RCPSP. On the contrary, a multi-project approach applies priority rules that take into account the project characteristics. The latter turned out to be more efficient when minimizing the time-based objective functions in RCMPSP [9]. Kurtulus and Davis [8] have for instance experimentally demonstrated the efficiency of rules like Maximum Total Work Content first (MAXTWK) and Shortest Activity from the Shortest Project first (SASP) in minimizing the mean project flow time. Experiences by Neumann at al. [10] show that for resource levelling problems the combination of two priority rules, first using Minimum Slack Time (MST) followed by Greatest Resource Demand (GRD) performs best. We denote this combination as: MST+GRD.
3.2
Deciding a start time
When the objective is to minimize the mean project flow time, the objective function (1) is updated when an eligible activity ai,j is selected and added to a partial schedule S c . We define a marginal cost function fiT , which describes the change in objective function as a result of scheduling an activity ai,j : fiT (S c , si,j ) = EstS
c
∪{si,j }
c
(ai,Ni +1 ) − EstS (ai,Ni +1 )
(3)
Sc
where Est (a) denotes the lower bound of earliest precedence-feasible start time of activity a given the c current partial schedule S c . EstS (ai,Ni +1 ) is the lower bound of earliest start time of fictitious ending activity ai,Ni +1 and is therefore the earliest completion time of project i. When SGS is used to solve project scheduling problem with such regular objective functions, deciding a start time in each SGS step becomes trivial - choose the earliest feasible start time. In this case, the priority rule(s) employed in selecting one activity from the eligible set will determine the final schedules. 1 Eligible
set: a set of activities that are not in the current partial schedule and whose predecessor activities have been scheduled.
140
Xiaoyu Mao, Nico Roos and Alfons Salden When the objective functions to be minimized are non regular, choosing the start time of an activity therefore becomes crucial. Recall that in the resource levelling problem, total squared utilization cost function (2) is resource-dependent. When the schedule is constructed successively by SGS, this function is an accumulation of marginal costs added successively when a new activity is scheduled. Suppose that the current utilization of resource k at time point t is uct,k , and that an activity ai,j is to be scheduled at time si,j . Then the marginal resource cost of scheduling activity ai,j on resource k is: si,j +pi,j R fi,j (S c , si,j ) = ck
X
t=si,j
si,j +pi,j k [(ri,j + uct,k )2 − (uct,k )2 ] = ck
X
t=si,j
k 2 k [(ri,j ) + 2 · ri,j · uct,k ]
(4)
Taking the example of Figure 1, we assume that the resource utilization price ck is 1 C. The activity lasting 4 time units with 2 resource units requirement and scheduled from time point 4 will have a marginal cost: (72 − 52 ) + (82 − 62 ) + (62 − 42 ) + (42 − 22 ) = 84C. Deciding the start time of a chosen activity is therefore to choose the greatest minimizer of this marginal cost.
Figure 1: Example of resource usage chart
3.3
Combined objectives
In the real world applications, schedulers are often asked to schedule the projects with an overall goal that combines multiple objectives (i.e., multicriteria objective or multi-goal). For instance, combining mean project flow time and total squared resource utilization cost as our problem objective. Usually, a linear combination of the objective functions is used: f = α · f 0 + (1 − α) · f 00 . Another possibility is to associate costs to the flow-time of a project. By varying the delay cost, we can either emphasize minimizing the flow-time or optimizing the resource leveling. Let dci denotes the delay cost per time unit for project i. Varying the value of dci can turn the problem emphasis from one objective to another. We therefore can define the combined problem objective as f (S) = T R dci · fM P F T + fT SU C , and marginal cost function fi,j (S c , si,j ) = dci · fi,j (S c , si,j ) + fi,j (S c , si,j ). The α relation between the variable dci and α is deducted: dci = 1−α .
4
Multiagent online heuristics
We discussed in the previous section the traditional project scheduling heuristics using a centralized authority to solve an optimization problem. This central decision maker would be equipped with all relevant data of that problem, asked to derive a solution that fulfills all the necessary side constraints, and optimize some kind of global performance criteria, either time-based or resource-based, simultaneously. In decentralized settings, central coordination of a system is replaced by decisions and actions locally taken by distributed computational agents. Those agents are assumed to act rationally with respect to their own interests, they communicate with each other in order to make local decisions and eventually a global schedule will emerge. Multi-project scheduling problems are often studied in a deterministic environment, where the project release time, the activity processing time are all static. In a dynamic environment like AGH, aircraft may arrive earlier or later than the original plan (uncertainty of release time), and ground service may take longer or shorter than expected (uncertainty of processing time). To uncertainties in project release time,
Distribute the Selfish Ambitions we propose an online version of a distributed heuristic. Instead of scheduling all projects at a time, we let agents start making schedules when the project is released. Two types of agents have to be defined in the case of a multiagent project scheduling problem: project agents (PAs) and resource agents (RAs). Each project agent, e.g. an aircraft or airline company in AGH, are responsible for a project consisting of a set of activities, while information about internal precedence constraints among activities of other project agents are hidden for them. Activities in these projects need certain resources provided by the resource agents for their completion. Each resource agent is responsible for a resource type; it receives the resource requirement request concerning a certain activity from a project agent; and it negotiates with the latter project agent the start time of this activity. Agent negotiation is carried out by adopting a slot purchasing market mechanism.
4.1
Slot-price market mechanism
Communication between agents occurs when a resource agent RAk receives a resource requirement request concerning activity ai,j sent by project agent P Ai . In order to complete its project as early as possible, P Ai prefers this activity to be scheduled at the earliest time-slot possible. However, this earliest time-slot for ai,j to be carried out on behalf of agent P Ai may not be an appropriate slot for RAk : it may conflict with the objective of RAk of minimizing the variation of its resource utilizations over time. To resolve this conflict, we employ a market mechanism that is a natural way of letting parties with different interests and goals appreciate goods or services. The price of each time-slot is determined by agent RAk , on the basis of the agent’s internal value system (in the resource levelling case, this price would be calculated by the marginal cost function (4)). Resource agents assign higher prices to popular time-slots requested by late coming project agents. By using this pricing model resource agents can force project agents to choose a cheaper slot in order to reduce their own resource levelling costs. Project agents, choosing a later (maybe cheaper) slot might incur a higher delay cost. The delay cost incurred by scheduling can be expressed in the marginal function (3). The actual price of one slot for project agent P Ai is therefore the addition of price given by resource agent RAk and the delay cost incurred by a late schedule. Now the negotiation amongst project agent P Ai and resource agent RAk to optimize both delay and resource usage costs simultaneously goes as follows: 1. When project agent P Ai is released, it selects one of its eligible activities ai,j to be scheduled and sends the k resource requirement (including the earliest start time ESTi,j , the latest start time LSTi,j , pi,j and ri,j ) to the corresponding resource agent RAk ; 2. RAk by receiving this request uses its pricing model (i.e., function (4)) on the resource requirements to calculate a list of offers, starting from the earliest possible slot; it then sends this list of offers to project agent P Ai ; 3. P Ai receives this list of offers, adds the possible delay costs to these slots, selects the cheapest slot for scheduling activity ai,j and informs resource agent RAk about its selection. 4. RAk will acknowledge this decision, and schedules ai,j on this selected slot, and keeps listening to the new upcoming requesting project agents. 5. P Ai will move on to schedule the next eligible activity.
With this agent negotiation model, we successfully distribute the decision making responsibilities and concerns to individual and different types of agents, and the overall schedule of all projects emerges.
4.2
Cooperative agent scheduling
Instead that both project agent and resource agent keep their personal information private, we propose in this section a cooperative agent scheduling scheme: project agents are willing to share part of their personal information to resource agents, under the condition that they don’t suffer any loss from information sharing. Being cooperative is not contradictory to the selfish nature of individual agents, instead, it has been proved to be beneficial not only for common goods but also for the selfish individuals in a non zero-sum game [2], as in AGH. Let us illustrate this type of agent collaboration. Figure 2 presents a time line of resource type k, assuming that resource agent RAk managing such resource type. Activity B with rB = 2 and pB = 5 has been scheduled on this resource at the time window [3, 8), the current resource levelling value is 5×22 = 20. A newly arrived activity A (rA = 2, pA = 4) is requesting the slot price starting from 4. According to the pricing model of resource agents presented earlier, the slot price of [4, 8) is: 22 + 4 × (42 ) − 20 = 48 (cf. Figure 2, option 1).
141
142
Xiaoyu Mao, Nico Roos and Alfons Salden
Figure 2: Activity A to be scheduled on a resource
In the cooperative scheduling scheme, project agent P AB which manages activity B is willing to reveal some additional information to resource agent RAk . For instance, when all the direct successors of activity B are scheduled (might be on other resources), activity B will have a secure time window - starting from the latest scheduled finishing time of its predecessors, and ending by the earliest scheduled starting time of its successors. Any shifting within this secure time window of B will not cause any further delay of the project that B belongs to. By knowing the secure time windows of activities, resource agents RAs are given more flexibilities to shift the slots for decreasing the resource levelling value. In Figure 2, the black dots on the resource time line are indicating the secure time window of activity B. Option 2 in Figure 2 illustrates the case where B is shifted backward to [2, 7) first, and A is scheduled on [4, 8), this shifting will result in a lower total resource levelling: 3 × 22 + 3 × 42 = 60, therefore, the marginal cost of scheduling activity A is only: 60 − 20 = 40. Instead of shifting B backward, in Figure 2 option 3, B is shifted forward to [6, 11), which gives an even lower marginal cost of scheduling activity A: 5 × 22 + 2 × 42 − 20 = 32. In this cooperative agent scheduling scheme, by providing more information to resource agents, project agents are not suffering any loss but helping resource agents for a lower levelling cost. Subsequently, cheaper slot options are provided to other project agents.
5 Experimental Results We conducted the experiments on two types of data set: 1) a multi-project instance built from the standard benchmark data set J30 of the Project Scheduling Problem Library (PSPLib) [7]; 2) a simulated airport specialized multi-project data set. Both experiments are carried out with three centralized priority-rule heuristic approaches (MST+GRD, MAXTWK, SASP) and two multiagent approaches (Non-cooperative, Cooperative). In the centralized heuristic methods, the differences in three methods are the priority rule(s) applied in selecting an eligible activity. For deciding the activity start times, they all use the same heuristic - choose the T start time resulting in the minimum combined marginal cost function fi,j (S c , si,j ) = dci · fi,j (S c , si,j ) + R c fi,j (S , si,j ). By decreasing gradually the value of dci , the problem will turn from a time-optimization problem (minimizing mean project flow time ) to a resource-optimization problem (minimizing total resource utilization cost). In multiagent heuristic methods, resource agents are assumed using the same pricing model (i.e., marginal resource cost function (4)). We achieve the emphasis turning from reducing mean project flow time to resource levelling by gradually decreasing the delay cost per time units dci . Recall that, project agents are concerned about the delay incurred by scheduling an activity. Choosing a high dci value emphasizes timebased objective, on the contrary, a low dci turns the problem emphasis to resource-based objective.
Experiments for benchmark instances In the first data set, we combine 10 project instances of single-mode projects in PSPLib (J301 1 → J301 10) into one multi-project problem. Each of the projects is composed of 30 real activities with precedence dependencies plus a dummy start and a dummy end activity. 4 types of renewable resources exist
Distribute the Selfish Ambitions
Figure 3: Various approaches in solving benchmark problem with bi-criteria objectives
and, each of which is required for executing an activity. The capacity of each resource type is the summed resource capacity in each instance. Projects are released sequentially, in a pace of every 5 time units.
Figure 4: Various approaches in solving AGH problem with bi-criteria objectives Experimental results in Figure 3 show that i) in all solution methods, the higher resource levelling value goes (cf. x-axis), the less total delay (cf. y-axis: equivalent as mean project flow times in representing time-based objective) will be suffered; ii) Non-cooperative online multiagent heuristic approach provides a schedule that is of comparable quality of centralized heuristic methods. iii) Moreover, by letting selfinterested agents behave cooperative, given the same total delay, we can have lower resource levelling value. Therefore, the overall schedule made by cooperative agent scheduling heuristic is significantly improved: it is comparable with the best heuristic methods.
Experiments for airport specialized instances An instance of AGH multi-project scheduling problem has the features that all projects have similar network structure and similar activity set. In order to build an AGH multi-project scheduling problem instance, we simulate a multi-project set by cloning a single-project instance in PSPLib. We conduct the experiments with 30 identical projects created out of the project instance J301 1, each project represents an aircraft turnaround, and aircraft are released every 5 time units. As in the benchmark problem experiments, five
143
144
Xiaoyu Mao, Nico Roos and Alfons Salden solution methods are investigated in this AGH problem, Experimental results in Figure 4 show similar results in which the cooperative agent approach outperforms two of the centralized heuristic approaches.
6
Conclusions & Future Work
This paper investigated several centralized priority-rule based heuristic approaches for solving multi-project scheduling problems with different objectives. We discuss the futility of these approaches when autonomous decision making is unavoidable, because the stakeholders involved are not willing to share their scheduling or pricing strategies as in the case of AGH. In order to tackle this distributed multi-project problem under release uncertainties, we developed an on online multiagent heuristic approach, and extended this approach with an agent cooperative scheduling scheme. The experimental results show that our online multiagent heuristic method generates schedules that are of comparable quality as centralized heuristic methods. By letting self-interested agents behave cooperatively — share non-critical information — the overall schedule is significantly improved in the sense that it is of comparable quality as the best heuristic methods. To schedule activities online is to be able to cope with a dynamically changing environment, in future work we will investigate the robustness of these online multiagent approaches (non-cooperative and cooperative) in the context of uncertain activity processing duration. Other research directions such as learning, coalition formation and other marketing mechanisms will be also pursued.
References [1] P. Brucker, A. Drexl, R. Mohring, K. Neumann, and E. Pesch. Resource-constrained project scheduling: Notation, classification, models, and methods. European Journal of Operational Research, 112:3– 41, 1 January 1999. [2] R. Dawkins. The Selfish Gene: 30th Anniversary Edition — with a New Introduction by the Author. Oxford University Press, 2006. [3] U. Dorndorf. Project Scheduling with Time Windows: From Theory to Applications. Physica-Verlag, 2002. [4] B. Heydenreich, R. M¨uller, and M. Uetz. Decentralization and mechanism design for online machine scheduling. In Algorithm Theory - SWAT 2006, volume 4059 of Lecture Notes in Computer Science, pages 136–147. Springer, 2006. [5] R. Kolisch and S. Hartmann. Heuristic algorithms for solving the resource-constrained project scheduling problem: Classification and computational analysis. In J. Weglarz, editor, Project scheduling: Recent models, algorithms and applications, pages 147 – 178. Kluwer academic, 1999. [6] R. Kolisch and S. Hartmann. Experimental investigation of heuristics for resource-constrained project scheduling: An update. European Journal of Operational Research, 174(1):23–37, 2006. [7] R. Kolisch and A. Sprecher. Psplib — a project scheduling problem library. European Journal of Operational Research, 96:205–216, 1997. [8] I.S. Kurtulus and E.W. Davis. Multi-project scheduling: Categorization of heuristic rules performance. Management Science, 28(2):161–172, 1982. [9] A. Lova and P. Tormos. Analysis of scheduling schemes and heuristic rules performance in resourceconstrained multiproject scheduling. Annals of Operations Research, 102(1-4):263–286, February 2001. [10] K. Neumann, C. Schwindt, and J. Zimmermann. Project Scheduling with Time Windows and Scarce Resources. Springer, 2nd edition, 2001. [11] A. Yassine, C. Meier, and T. Browning. Multi-project scheduling using competent genetic algorithms. Technical report, University of Illinois, Department of Industrial & Enterprise Systems Engineering (IESE), 2007.
