Learning-Based Negotiation Strategies for Grid Scheduling - CiteSeerX

Learning-Based Negotiation Strategies for Grid Scheduling Jiadao Li, Ramin Yahyapour Institute for Robotics Research - Information Technologies University Dortmund, 44221 Dortmund, Germany {jiadao.li,ramin.yahyapour}@udo.edu Abstract— One of the key requirement for Grid infrastructures is the ability to share resources with nontrivial qualities of service. However, resource management in a decentralized infrastructure is a complex task as it has to cope with different policies and objectives of the different resource providers and the resource users. Recent research indicates that agreement-based resource management will solve many of these problems as it supports the reliable interaction between different providers and users. Here, negotiation is needed to create such bi-lateral agreements between Grid parties. Such negotiation processes should be automated with no or minimal human interaction, considering the potential scale of Grid systems and the amount of necessary transactions. Therefore, strategic negotiation models play an important role. In this paper, a negotiation model and learning-based negotiation strategies are proposed and examined. Simulations have been conducted to evaluate the presented system. The results demonstrate that the proposed negotiation model and the learning based negotiation strategies are suitable and effective for Grid environments.

I. I NTRODUCTION Grid computing [1], [2] is considered a cornerstone of next generation distributed computing, as it tackles many issues in dynamic interaction between autonomous and decentralized resources from different providers. In this Service Oriented Architecture (SOA [2], [3]), a resource user typically requires a certain service quality to be provided by the resource owners. Therefore, agreement based resource management [4] is typically considered as a suitable approach for this scenario. Negotiation is the process towards creating suitable agreements between different parties in a Grid. The whole task of negotiation is challenging as the resources are heterogeneous and the service provisioning is not a standardized good (as e.g. in stock markets) but depends on the individual requirements and preferences of the user for a particular task. During the negotiation process, the conflicts of the different objectives and policies between the resource users and resource providers must be reconciled. The negotiation process in a Grid computing environment should be done automatically and transparently with the growing scale in Grids [5]. For efficient Grid computing, this task must be performed very frequently and it is highly affected by the dynamic resource conditions. Thus, during the negotiation process, every user or resource provider will have an agent or resource broker as a negotiation wrapper which will act on behalf of the participant. In order to automate the negotiation process, suitable negotiation models are required that take the

different policies and objectives of the resource providers and resource users into account and produce suitable service level agreements in reasonable time with minimized or even no user and provider interference. Currently, there is no mature and accepted negotiation model available for the Grid computing scenario. Although negotiation strategies can be found in the area of economic market models, the adaptation and the applicability of these models are not yet clear. In this paper, such negotiation models and strategies for agreement negotiation are addressed. Current Grid research works are towards standardizing resource management functionalities. ”WS-Agreement” [6] is a protocol proposed by the GRAAP working group in the GGF. This protocol can be used as a simple negotiation protocol. In the current WS-Agreement proposal, the negotiation process is a one-shot approach in which negotiation parties can only accept or reject opponent’s proposals. This one-shot negotiation process is very time consuming and inefficient since the negotiation opponents have no means to analyze why a proposal is unacceptable, nor in which dimension or direction of the available agreement space a potential solution may exist [7]. To improve the efficiency of the negotiation, the negotiation process should be multi-rounded based. The design of a suitable negotiation protocol is one of the next issues addressed by the GRAAP working group. In addition, a common infrastructure for the implementation of this dynamic negotiation process is necessary. It is anticipated that the WSAgreement and OGSA-EMS work will provide (at least parts) a suitable foundation. The whole Grid resource management process will involve many service components, like resource discovery, information services, billing and accounting [8], [9]. After the resource discovery many suitable resources may be available for a particular Grid job. In this paper, we focus only on the negotiation process in which a user negotiates with a set of resource providers. In our model the user, or more precisely some meta-scheduling agent or job broker on his behalf, will contact different resource providers, negotiate with several of them and make a decision to commit to a particular agreement with one resource provider. This is considered as the one to many negotiation type [10]. Usually, this negotiation type can be treated as ”reversed auctioning” [11]. However, there are some drawbacks of using auction mechanisms, for instance, there is no flexible way of exercising different strategies

with different negotiation opponents. Moreover, auctions do not support bidirectional offers with counter offers between parties. The negotiation relationships in Grid computing are modeled as multiple bilateral negotiations which can be performed sequentially or concurrently. But the sequential bilateral negotiation is usually quite time-consuming which is not suitable in a realtime-constrained Grid computing scenario. Therefore, the concurrent bilateral negotiation model is more suitable for the Grid computing scenario [11]. This paper is an extension of previous work [12]. In this paper, a strategic negotiation model for Grid computing, which includes the utility functions or preferences of the negotiation parties and suitable learning-based negotiation strategies, are proposed and evaluated. The rest of this paper is organized as follows: Section 2 presents a brief review of related work. In Section 3, the strategic negotiation model as well as the learning-based negotiation strategies are explained. The simulation configuration and results are presented and analyzed in Section 4. Conclusions and information on future works are given in Section 5. II. R ELATED W ORK There are many approaches proposed for the Grid resource management problems, for example, economic methods. An overview of such methods is given, for example, by Buyya et.al. in [13], or Ernemann et.al. in [14], or Wolski et.al. in [15]. In these papers, economic based resource management in Grid computing are investigated, several economic models are evaluated. To this end, a lot of effort has been made on Grid resource management with support of service level agreements (SLAs). In [4], the concepts of agreement-based resource management in the Grid computing environment are introduced and a general agreement model is presented; in [16], a Grid resource broker supporting SLAs called GRUBER is presented and evaluated in a real grid. In the current literatures, learning-based negotiation strategies for multi-rounded negotiation protocols in the Grid computing environment have not yet been reviewed. In this paper, we use learning-based negotiation strategies in the proposed negotiation model for Grid scheduling and present their evaluation with discrete event-based simulation. III. S TRATEGIC N EGOTIATION M ODEL As bilateral negotiation model is the building block of concurrent negotiation model, we will briefly introduce this first in this section. A. Bilateral Negotiation Model There are three parts in the bilateral negotiation model [17]: 1) the negotiation protocol, 2) the used utility functions or preference relationships for the negotiating parties, and 3) the negotiation strategy that is applied during the negotiation process. In this paper, we adopted Rubinstein’s sequential alternating offer protocol in Grids, see [18]. In the negotiation process, when one negotiation side times out or an agreement

is created, the negotiation process will end. Disagreement is considered as the worst outcome, therefore, the negotiation party always try to avoid opting out of the negotiation. In this negotiation model, the negotiation parties do not know the opponents’ private reservation information and their preferences/utility functions. In this strategic negotiation model, the utility function of the job user and the preference relationship of the resource provider are given for their respective automated negotiation agents. Typically, the objectives of a user for a computational job are to obtain a shorter response time and/or to get cheaper resources, while the resource providers expect to gain higher profit and higher utilization. However, the model is not restricted to particular objectives. In real Grid systems, there can be many different negotiation objectives, which are interdependent and should be dealt simultaneously which yields to a multi-criteria optimization problem [19]. To this end, although many Grids are used in the academic domain where price may not be so important, pricing and accounting can help them manage and evaluate resource allocation. In this presented research work, we only consider the expected waiting time of the job and the expected cost per cpu time. However, the model can be applied and extended to other criteria as well. Different job users may have different preferences which can be expressed by the different weights of the negotiation issues. In this model, Uprice denotes the sub-utility function for the price of a job and Utime is the sub-utility function for the job’s waiting time. Pcmax is the maximum acceptable price of the user; Pcmin is the minimum offered price of the user. Tcmax is the maximum acceptable waiting time of the user; Tcmin is the minimum offered waiting time of the user. Uprice =

Pcmax − Pct Pcmax − Pcmin

(1)

Utime =

Tcmax − Tct Tcmax − Tcmin

(2)

The user weight of the price utility is Wprice , the weight of the time utility is Wtime . This leads to the following aggregate utility function of the user: Ujob = Wprice ∗ Uprice + Wtime ∗ Utime

(3)

Because the negotiation time in this case is usually very short, the utilities are not discounted with the negotiation time going on. The weights of different negotiation issues are Pn normalized, that is, j=1 wj = 1. In the negotiation process, an agent may change its preference for an issue by changing the weight associated to that issue. For the resource providers, there are also two corresponding negotiation issues: the expected waiting time until a job can be started Tst (Job), and the expected price Pst (Job). The expected waiting time for the newly incoming job can be obtained from the current resource status and the future schedule plan considering other created agreements which have to be fulfilled. The expected price will be obtained via the negotiation process.

B. Negotiation Strategies

a finite set S of states s of the concerned environment (s ∈ S); • a finite set A of actions a that can be performed (a ∈ A); • a reward function R: S × A −→ r. The agent’s goal is to learn a policy: S −→ A that maximizes the expected sum of discounted rewards V : •

In the strategic negotiation model, the negotiation agents can take different kinds of negotiation strategies developed in the agent community [7]. In the negotiation process, the strategies of the negotiation parties usually change dynamically based on the remaining available negotiation time. Therefore, in this paper, time dependent negotiation strategies [20] are adopted in which the negotiation parties make successive offers or counter-offers depending on the remaining negotiation time. Vj is assumed as the utility function of the negotiation party which associates with the negotiation issue j and the xta→b [t] is the offer provided by one party (denoted by a) to another negotiation party (denoted by b). If Vj is decreasing:

where 0 ≤ γ < 1 is the discounting factor, the negotiation time is from 0 to n. The Q-learning algorithm is based on the estimated values of the agent’s state (s)-action (a) pairs, called Q(s, a) values. Based on these values, the agent updates its Q(s, a) values using the formula:

xta→b [t] = minaj + αja (t)(maxaj − minaj ),

Q(s, a) ←− Q(s, a)+α[r+γ×maxa0 Q(s0 , a0 )−Q(s, a)] (8)

(4)

If Vj is increasing: xta→b [t] = minaj + (1 − αja (t))(maxaj − minaj ),

(5)

There are many ways of defining the function for αja (t). For the initial bargaining value kja is used, for which the following relation holds 0 ≤ kja ≤ 1. We use the following function for αja (t): αja (t) = kja + (1 − kja )(

t 1/β ) , tamax

(6)

The deadline of the negotiation party a for the negotiation is tamax . t denotes the current time instant in the negotiation time set. β(> 0) value determines the concession pace with the time going on. There are three typical strategies by changing the value of β [20]. When 0 < β < 1, the negotiator will be tough [21], which means that he will stick close to the initially offered value until the time is almost exhausted. Close to the deadline he will concede up to the reservation value. In contrast, for β > 1 the negotiator will concede faster which means that this negotiator will go to its reservation value very easily [22]. For β = 1, the negotiator will linearly concede to its reservation value. These negotiation tactics will be used to create the offers in the negotiation process, but they are not flexible enough for a dynamically changing Grid environment. Therefore, we propose learning-based negotiation strategies, which allow the agents to dynamically adapt their α value according to their specific preferences. 1) Learning Based Negotiation Strategies: The following learning-based negotiation strategies apply the reinforcement learning algorithms [23], [24]. In this paper, the Qlearning [25] algorithm was chosen because it is an online algorithm that does not require a model of the environment and thus it is well suited to dynamic and unpredictable Grid environments. In the negotiation process, each negotiation agent uses a Q-learning algorithm to select the suitable time dependent negotiation tactic introduced before. The classical model of Q-learning consists of:

V [γr0 + γ 2 r1 + ... + γ n rn ] = V

n X

γ i ri

(7)

i=0

where α is the learning rate which determines the rate of change of the estimation and maxa0 Q(s0 , a0 ) is the value of the action that maximizes the Q function at state s. In this paper, we use a ²-greedy [24] function that selects the action with the highest Q(s, a) value. Using this approach, the learning agent behaves greedy most of the time, but every once in a while, with a small probability ², it selects an action at random, uniformly, independently of the action-value estimates. In order to use the Q-Learning algorithm, we have to identify the possible negotiation states and actions. For the job users, the states will be identified according to the number of available resources, the current remaining negotiation time and the type of the negotiation opponent. For the resource provider, the negotiation states can be identified considering the remaining negotiation time and the types of negotiation opponent. In order to identify the negotiation states, we divide the negotiation time into beginning part and ending part evenly. We can identify types of the negotiation opponents when negotiation time t is in the range of [3, T ] where T is the negotiation deadline of the negotiation party. For the job users, j j if (2 ∗ Ot+1 − Otj − Ot+2 ) > 0, then the resource provider use conceder strategy; if it is less than or equal 0, the resource provider use the tough or linear strategy, where Otj is the offer that job user received from the provider at time t. The number of available resources will be decided as high or low according to one specified threshold value. As introduced before, we use the time dependent negotiation tactic to create the next offer. The actions for the negotiation parties are to select the proper parameters α to produce the next offer. Using the Q-Learning algorithms, the procedure of the adaptive negotiation algorithms is as follows: • Initialize Q(s, a) arbitrarily; Specify the terminal states (i.e., agreement reached, deadline reached). • Identify the current state s according to the parameters and get the reward signal r. • Choose action a according to the ²-greedy policy, which means that it will choose the suitable α and generate the next offer using time dependent negotiation tactic.

Job Agent

Job User Thread1

Resource Provider Agent

Resource

Thread2

Resource Provider Agent

Resource

it will be created and committed. Of course, in a real life scenario the job agent might actually exploit several offers and decide the best offer. For simplicity, we restricted our examination to first available agreements. If one negotiation thread is successfully negotiated, all of the other negotiation threads will be terminated. The agreement can then be used by provisioning and execution service to actually start a job on the local resource management system. This concurrent model will be extended in our future research work. IV. E VALUATION

Fig. 1.

System model

Terminate when terminal states are reached. The reward functions of negotiation agents can be quite different. If both of the job users and the resource providers want to create the agreement as soon as possible, the reward scheme for the job user is to reduce the weighted sum of the difference of the expected wait time and the expected cost between the offers of the job user and the resource provider; the reward scheme for the resource provider is to reduce the difference of the price offer between the offers of the job user and the resource provider. If they want to get higher utility and do not care whether they can create the agreement or not, they can use the opposite reward schemes. We just assume that the former is the positive reward scheme, while the later is the negative reward scheme. These schemes will be evaluated using simulations. •

C. Concurrent Bilateral Negotiation Model As mentioned above, in the Grid environment, a number of resources will typically be available which are capable of fulfilling the job constraints after the resource discovery phase. These constraints include: e.g., the required number of CPU nodes, the memory capacity. The user or a corresponding scheduling component will contact different resource providers and initiate the negotiation process for the actual resource allocation. The negotiation relationship is of the ”one to many” type, which can be treated as a concurrent bilateral model. As shown in Figure 1, two concurrent negotiation threads are running. The negotiation process is started by the user, more precisely by the job agent, who contacts different resource providers and begins the negotiation process. In the concurrent negotiation threads in which a single user is involved, the reservation value of the negotiation issues and preferences are the same. However, the user may adopt different strategies with respect to different negotiation opponents. Furthermore, they might change the negotiation strategies during the negotiation process. Because these negotiation threads are executed concurrently, it is very difficult to predict whether the user might achieve a better offer from another negotiation thread if there is already a suitable offer that could be committed to an agreement. In our model, we assumed that once an agreement is available,

Discrete event simulation has been used to evaluate the proposed negotiation model. Currently, there is no real data from Grid computing environments that include suitable information for negotiation models. However, high performance computing is still the typical application scenario for Grid technology. For this scenario, workload traces are available which were recorded on actual machine installations [26]. Therefore, our first evaluations are based on such traces. However, negotiation information are not included in this data as none of the real systems supported negotiation models yet. To this end, the missing information can only be modeled based on first assumptions. In the following the simulation configuration is described and the simulation results are analyzed. A. Simulation Configuration At the beginning of the negotiation, the negotiation parties will always make the offers which are most favorable to themselves. So we assume initial values of 0 for kja of all the negotiation parties. For performance analysis we assume a negotiation interval of 1s between every negotiation round. In the following we describe the models of the users and the resource providers. In order to evaluate the learning-based negotiation algorithms, we will compare simulation results in different simulation cases. 1) User Model: In our simulation we consider parallel batch jobs in an online scenario. That means, jobs are submitted from users over time and they are not known in advance. Typically, users will behave quite differently in the negotiation process. For our simulation, we assume two different kinds of user objectives: time-optimization and cost-optimization. The actual behavior of real users will be investigated in future research work. Below are the parameters of the user modeling which have been applied for the simulation. • Negotiation span is uniformly distributed in [0, 30]s. • Maximum price of the different job user is uniformly distributed in [4.0, 9.0]. • Acceptable waiting time for the job users are uniformly distributed in [0, 36000]s. • For the tough negotiator, β value is uniformly distributed in [0.02, 0.2]. • For the conceder negotiator, β value is uniformly distributed in [20, 40]. • Weights of time and price for the time-optimization are 0.8 and 0.2, while the weights of the time and price for the cost-optimization are 0.2 and 0.8.

2) Resource Provider: For the local resource management system an FCFS scheduling strategy with backfilling [27] is adopted which is common for parallel computers. There is no preemption allowed, which means that once a job is started, it will run to completion. To this end, in this evaluation we do not consider the co-allocation of resources from different providers. The resources are all homogeneous and only differ in the number of available CPU nodes at each site. Different resource providers have different policies and different negotiation strategies. It is assumed that users will contact resource providers, which can fulfill their hard constraints and requirements, in a round robin fashion. The simulated hardware configurations of the resource providers are consistent with actual configurations of the systems from which the real traces are originated. In this paper, we present results for traces from the Cornell Theory Center [26] which had 512 CPU nodes. In our simulation we assumed a Grid scenario with 6 different machines (parallel computer or cluster with a given set of CPU nodes) and therefore 6 resource providers. However, to stay consistent with the available workload from the CTC traces, the total number of nodes for all simulated machines is again 512 nodes. The number of nodes on each machine and the negotiation parameters for each resource provider are given below. • The numbers of CPU nodes for the machines are {384,64,16,16,16,16}. • Their different maximum prices per CPU time are {8.2,8.0,7.5,7.6,7.4,7.5}. • Their different minimum prices per CPU time are {2.4,2.3,2.0,1.95,1.90,1.80}. • Negotiation deadlines of different resource providers are all 30s, which means that usually the resource provider will not opt out of the negotiation once the negotiation thread is created. • For the tough negotiator, β value is {32, 35, 34, 38, 40, 40}. • For the conceding negotiator, β value is {0.03, 0.05, 0.04, 0.10, 0.05, 0.06} B. Evaluation Remarks Without a reference benchmark for negotiation-based Grid scenarios, it is difficult to compare and analyze the quantitative and qualitative output of such a scheduling model. In the following we provide some first evaluation remarks which give some qualitative information about the performance of the model. The actual quality will have to be verified with better workload models and real implementations. • Comparison between the negotiation result and the reference point [21], which is the middle of the zone of possible agreement of user and resource provider: [Cjmax , Sjmin ]. The reference point is computed by the following function: Cjmax + Sjmin (9) 2 The rate of successfully created agreement for all jobs. Ujref =

•

The negotiation overhead to create the agreement measured by the time taken to create the agreement. In our case, we use the final negotiation rounds which represents the required number of messages exchanged. The actual network overhead will depend on the actual network speed for this message exchange. • In computational Grids, the users will concern about the response time and waiting time of a job, while for the resource providers the utilization and the profit will probably be the main objectives. We also compare these criteria to get some feedback about the feasibility of the negotiation model. 1) Simulation Results: We used the first 5000 jobs from the CTC workload traces [26] to do our simulation. We use the settings and configurations as defined earlier to compare the negotiation results in different simulation cases. We compare the on average required number of negotiation rounds for the successful creation of an agreement, and the rate of successfully created agreements in comparison to the total number of job requests. Other criteria are the average weighted response time (AWRT), the average weighted waiting time (AWWT), the average price difference between the agreement price (AP) and the reference price (RP). For the weight in AWRT and AWWT, the job resource consumption is used [28]. This weight prevents any favor of jobs with high or low resource consumption over each other. The parameters used in Formula 8 are as follows: ² is 0.2; learning rate α is 0.5; discount rate γ is 0.8; We assume that in the beginning of the negotiation, both of them use the tough behavior; if the negotiation agent does not use the learning algorithms, it will stick to the tough behavior. Due to the space limitations, we only show some typical simulation results. The following simulation cases are considered. Case 1: Both of them use learning algorithms with positive reward scheme; Case 2: Both of them use learning algorithms with negative reward scheme and want to get higher utilities; Case 3: The job users do not use learning algorithm, while the resource providers adopt the learning algorithm with positive award scheme; Case 4: The resource providers do not use learning algorithms, while the job users use the learning algorithms with positive reward scheme. Case 5: The resource providers use the learning algorithms with positive reward scheme, while the job users adopt the learning algorithm and use the opposite reward scheme. The success rate of negotiations in these 5 cases are 85.92%, 54.10%, 42.40%, 27.38%, 42% respectively. Figure 2 shows a selection of results. R1 to R6 stands for the resource 1 to resource 6. The AWWT and the AWRT is comparable and in the same range as for Grid models which do not use negotiation models but conventional queuing systems. That means, the presented model is feasible for real Grid infrastructure as it does not lead to any drawbacks in the performance results. However, the negotiated waiting time of the jobs will be guaranteed by the resource providers which is the anticipated quality of service level and can be seen as a major asset of such an approach. In Case 1, the number of •

The comparison of average utilization rate of different resources

0.7 R1 R2 R3 R4 R5 R6

The comparison of average utility

0.6

0.5

0.4

0.3

0.2

0.1

0

1

2

3

4

5

50 R1 R2 R3 R4 R5 R6

45

40

35

30

25

20

15

10

5

0

1

2

3

4

5

4

The comparison of AWRT

3.5

3

2.5

2

1.5

1

0.5

The comparison of negotiation rounds in different cases

0

1

2

3

4

16

R1 R2 R3 R4 R5 R6

12

10

8

6

4

2

Fig. 2.

1

2

3

5000

4000

3000

2000

1000

0

14

0

4

R1 R2 R3 R4 R5 R6

6000

5

20

18

7000

The comparison of AWWT in different cases

R1 R2 R3 R4 R5 R6

4

5

The comparison of average difference between the AP and RP

4.5

x 10

1

2

3

4

5

3 R1 R2 R3 R4 R5 R6

2

1

0

−1

−2

−3

1

2

3

4

5

Comparison between different negotiation cases including the results for the individual six resources.

successfully created agreements and the resource utilization is the highest; the required number of negotiation rounds is lowest. Using the learning algorithms with positive reward scheme, the negotiation parties can create agreements with low negotiation rounds. In Case 2, we can see that although the creation rate is less than Case 1, it is still higher than Case 3 and Case 4. In Case 3, the job users stick to the original tough behavior and they can obtain service with lower price and get the highest utility. In Case 4, the resource providers stick to the original behavior, so the job users have to pay a very high price and the creation rate is low. The job users get the lowest utility. If both of the resource providers and the job users stick to the tough behaviors, the creation rate is only 1.88%, therefore, it is very difficult for them to create the agreements.

From these simulations, we can see that learning-based negotiation strategies are quite flexible and can actually be used in the dynamically changing Grid infrastructure. The negotiation parties can use learning-based negotiation strategies with different reward schemes depending on different objectives and preferences of the negotiation parties. If a negotiation party wants to create agreement as soon as possible, it can use a positive reward scheme; while a negotiation party wants to get higher utility and does not care if the process leads to an agreement creation, it can use a negative reward scheme. Of course, the resource provider can get higher price if he sticks to the tough behavior, but this will lose many chances of creating agreements, therefore it can not obtain higher utilization rate. The same applies the user, if he sticks to a tough behavior, he may not obtain services from resource providers even if it

would yield a higher utility. V. C ONCLUSIONS AND F UTURE W ORK In this paper, we proposed and evaluated a strategic negotiation model in the Grid scenario which includes negotiation protocol, utility functions or preference relationships and learning-based negotiation strategies. This model is evaluated using discrete event based simulation. The results show that it can be applied in the practical use in automatic job scheduling. The current research in Grid computing shows that there is a trend for future resource management systems to include automatic management features for quality of service and cost consideration. The inclusion of these features requires negotiation and agreement support. However, this negotiation cannot be expected to be manually conducted by the users or resource providers themselves. Therefore, support for automatic negotiation is expected to become a key component for Grid systems. Here, the foundation of protocols as well as strategies are yet missing. Learning-based negotiation strategies offer a possibility for such a scenario. As we can see from our experiments, the user can now obtain guaranteed quality of service and reliable agreements for Grid jobs by applying the presented negotiation strategies. In our scenario, the expected waiting time is guaranteed by the resource provider. The first simulation results back the assumption that the model is practically useable in the Grid scheduling environment. The presented results can be seen as first steps in analyzing the features and requirements for automatic negotiation strategies. However, the actual evaluation of the obtained service quality is difficult to obtain as there is no valid user job and preference workload model for Grids available which takes economic functionality into account. The presented results indicate that the negotiation overhead in terms of exchanged messages is manageable for practical application. The obtained agreement results can also be considered to be good enough for real world scenarios. In the current work, once an agreement is created, it will be committed by both sides and not be violated. Future work will include further investigation of the strategies considering re-negotiations and cancelation of made agreements under penalty payments. Moreover, the authors work on scheduling frameworks which can include such negotiation protocols. Here, input to the GGF activities of the GRAAP and OGSARSS working group as well as the GSA research group are foreseen. Overall, it is expected that a Grid market model based on different negotiation strategies will emerge from the practical application. These will include pricing models as well as workload models. R EFERENCES [1] I. Foster, C. Kesselman, and S. Tuecke, “The anatomy of the Grid: Enabling scalable virtual organizations,” Lecture Notes in Computer Science, vol. 2150, 2001. [2] I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, 2003. [3] “The w3c web services architecture working wroup public draft,” 02 2004, http://www.w3.org/TR/ws-arch/.

[4] K. Czajkowski, I.Foster, and C.Kesselman, “Agreement-based resource management,” Proceedings of the IEEE, vol. 93, no. 3, pp. 631–643, 03 2005. [5] I. Foster, N. R. Jennings, and C. Kesselman, “Brain meets brawn: Why grid and agents need each other,” in AAMAS ’04: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems. Washington, DC, USA: IEEE Computer Society, 2004, pp. 8–15. [6] “Web services agreement specification,” 07 2005, https://forge.gridforum.org/projects/graap-wg. [7] N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, C. Sierra, and M. Wooldridge, “Automated negotiation: Prospects, methods and challenges,” Int. J. of Group Decision and Negotiation 10 (2) 199-215., vol. 2, no. 10, pp. 199–215, 2001. [8] “Grid scheduling architecture research group,” 07 2005, https://forge.gridforum.org/projects/gsa-rg. [9] “The open grid services architecture, version 1.0,” 01 2005, https://forge.gridforum.org/projects/ogsa-wg. [10] A. R. Lomuscio, M. Wooldridge, and N. R. Jennings, “A classification scheme for negotiation in electronic commerce,” International Journal of Group Decision and Negotiation, vol. 12, no. 1, pp. 31–56, 2003. [11] T. D. Nguyen and N. R. Jennings, “A heuristic model of concurrent bi-lateral negotiations in incomplete information settings,” in Proc. 18th Int. Joint Conf. on AI. IEEE Computer Society, 2003, acapulco , Mexico , 1467-1469. [12] J. Li and R. Yahyapour, “Negotiation strategies for grid scheduling,” submitted to the First International Conference on Grid and Pervasive Computing 2006. [13] R. Buyya, “Economic-based distributed resource management and scheduling for grid computing,” Ph.D. dissertation, Monash University, Melbourne, Australia, 2002. [14] C. Ernemann and R. Yahyapour, ”Grid Resource Management - State of the Art and Future Trends”. Kluwer Academic Publishers, 2003, ch. ”Applying Economic Scheduling Methods to Grid Environments”, pp. 491–506. [15] R. Wolski, J. S. Plank, J. Brevik, and T. Bryan, “Analyzing market-based resource allocation strategies for the computational grid,” International Journal of High Performance Computing Applications, vol. 15, no. 3, pp. 258–281, Fall 2001. [Online]. Available: http://www.cs.utk.edu/ plank/plank/papers/IJHPCA01.html [16] C. Dumitrescu and I. T. Foster, “GRUBER: A grid resource usage SLA broker.” in Euro-Par, 2005, pp. 465–474. [17] S. Kraus., Strategic Negotiation in Multi-Agent Environments. MIT Press, Cambridge, USA,, 2001. [18] A. Rubinstein, “Perfect equilibrium in a bargaining model,” Econometrica, vol. 50, pp. 97–110, 1982. [19] K. Kurowski, J. Nabrzyski, A. Oleksiak, and J. Weglarz, ”Grid Resource Management - State of the Art and Future Trends”. Kluwer Academic Publishers, 2003, ch. ”Multicriteria Aspects of Grid Resource Management”, pp. 271–295. [20] P. Faratin, “Automated service negotiation between autonomous computational agents,” Ph.D. dissertation, Department of Electronic Engineering, Queen Mary College, University of London, UK, 2000. [21] H. Raiffa, The Art and Science of Negotiation. Harvard Universtiy Press, 1982. [22] D. Pruitt, Negotiation Behavior. Academic Press, 1981. [23] L. Kaelbling, L. Littman, and A. Moore, “Reinforcement learning: a survey,” Journal of Artificial Intelligence Research, vol. 4, no. 1, pp. 237–285, 1996. [24] R. S. Sutton and A. Barto, Reinforcement Learning An Introduction. MIT Press/Bradford Books, 1998. [25] C. Watkins, “Learning from delayed rewards,” Ph.D. dissertation, Cambridge Univ., Cambridge England, 1989. [26] “Standard workload format,” 07 2005, http://www.cs.huji.ac.il/labs/parallel/workload/index.html. [27] D. Lifka, “The ANL/IBM SP scheduling system,” in Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph, Eds. Springer-Verlag, 1995, pp. 295–303, lect. Notes Comput. Sci. vol. 949. [28] U. Schwiegelshohn and R. Yahyapour, “Analysis of first-come-firstserve parallel job scheduling.” in Proceedings of the 9th SIAM Symposium on Discrete Algorithms, January 1998, pp. 629–638.