Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing
A Formal Model of Dynamic Resource Allocation in Grid Computing Environment Leila Ismail
Bruce Mills
Alain Hennebelle
United Arab Emirates University College of Information Techology P.O.Box 17555, Al-Ain, UAE
United Arab Emirates University College of Information Technology P.O.Box 17555, Al-Ain, UAE
Laboratoire d’Informatique Fondamentale de Lille (LIFL) - UMR 59655 Villeneuve d’Ascq Cdex, France
Email:
[email protected]
Email:
[email protected]
Email:
[email protected]
Abstract—The Grid computing environment is proposed as a way to enhance the efficiency of the underlying system by maximizing utilization of distributed resources. Dynamic resource allocation algorithms are used to dynamically allocate suitable resources to users’ tasks. The efficiency of these algorithms is then vital for both the usability of the Grid system and fulfilling its objective. In this paper, we define a formal model of a Grid system suitable for a formal analysis of resource allocation algorithms in Grid computing. We also discuss the use of information which is available to resource allocation algorithm developers, such as researchers and students, who do not have access to an internal Grid to compute the efficiency of their algorithms.
I. I NTRODUCTION The widespread use of the Internet and the availability of high-speed networks and powerful computers at low-cost have led to the introduction of Grid computing [1] [2] [3] [4] [5]. This new field has emerged, distinguished from conventional distributed computing by its focus on large-scale resource sharing. Its main objective is to take care of available resources and turn the underlying infrastructure into a powerful enterprize for Grid applications [6]. To achieve its goal of efficient resource sharing, efficient and effective resource allocation algorithms are essentials. The efficiency of the resource allocation algorithms can be affected by many factors, such as the number of tasks arriving in the system, the available resources, the network characteristics, etc. To study the effectiveness of the resource allocation algorithms, there is a need for a formal model to represent the Grid resource allocation problem and to identify and analyze the impact of the different Grid parameters whose changes would have an impact on the efficiency of the algorithms. This model is essential for comparing different algorithms. Having a formal model to compute the efficiency of a resource allocation algorithm is particularly important, as algorithms developers, such as researchers and students, may not have access to a real Grid for computing the efficiency of their algorithms. In this paper, we propose a formal model for understanding and measuring the efficiency of a Grid resource allocation algorithm. Our formal model involves a set of resources connected together, and a set of tasks which have resources requirements. The tasks are assumed to be scheduled to run 978-0-7695-3263-9/08 $25.00 © 2008 IEEE DOI 10.1109/SNPD.2008.136
on these resources according to some resource allocation algorithm. Very little is assumed about the details of the algorithm, so the resulting conclusions are quite broad. We provide a means for analyzing the efficiency of arbitrary resource allocation P algorithms. In particular, we provide the equation T a ¯ = α ri τi . Except for a ¯, this equation uses parameters that can be measured from outside a Grid. a ¯ is P the average load of the Grid. ri τi is the sum of the load (the resources requirements) and time computed by running each task separately on a dedicated reference machine. T is the total time taken by a run when all the tasks are passed to the Grid according to an allocation algorithm. According to the model, for each grid there is an α such that this equation is true independent of the number of tasks, their requirements and their distribution. Consequently, the ratio of a ¯s of the different algorithms can compare between the different allocation algorithms even if α is not known. As a step towards validating the equation and its applicability, we modified our implementation of a Grid test-bed [7] to include a First Come First Served tasks scheduling algorithm and a resource allocation algorithm based on the first fit algorithm. The first fit algorithm has been chosen because, in the literature, it is considered as a base for comparing with other allocation algorithms. Then, we conducted our experiments using our testbed on a cluster of interconnected machines. Our initial experimental results are encouraging. They showed that the equation holds. The paper is organized as follows: in section II, we present the resource allocation problem in a Grid computing environment. Section III presents our overall approach describing the different actors and functions of our model. Our formal model and resolutions are presented in section IV. The experiments conducted and their measurements and evaluation analysis are described in section V. After a presentation of related work in section VI, we conclude in section VII. II. RESOURCE ALLOCATION PROBLEM Grid computing is a form of autonomic parallel and distributed computing [8] [9] which involves coordinating and sharing of distributed and autonomous computing application, data storage, and/or network resources autonomously and
679 685
efficiently to satisfy users’ quality of services as per a Service Level Agreement (SLA). Resource allocation is one of the important system services that have to be available to achieve the objectives from a Grid computing.
Fig. 1.
Architecture Overview of a Grid
Figure 1 shows the resource allocation core services within the Grid core middleware. Resource allocation mechanisms are responsible for the allocation of users’ applications to Grid execution hosts according to an SLA. An SLA describes an application’s requirements in terms of resources to ensure quality of services to the application. Dynamic allocations are required where resources are discovered and allocated to applications. The need for dynamic resource allocation mechanisms is highlighted in the requirements of the Open Grid Services Architecture [10], a standard definition for Grid computing architecture. Furthermore, the resource allocations mechanisms should not lead to resources underutilization [10]. As shown in Figure 1, resource allocation involves two Grid system services which are: resource monitoring and scheduling. Resource monitoring refers to recording and reporting information about each resource’s capabilities, usage and future reservations. A resource can be a CPU, disk, memory, network bandwidth, etc. The information on a resource is then retrieved by the scheduler which decides on the allocation of users’ applications to appropriate resources. The resource allocation problem is not new to the computer and engineering sciences. The Grid computing resource allocation problem is only a special case of traditional parallel computing resource allocation problem addressed in traditional parallel and distributed systems, such as symmetric multiprocessor machines (SMP), massively parallel processors machines (MMP) and cluster of workstations (COW) [11]. In the traditional parallel computing, the parallel processing allocation algorithms mainly concentrated on finding an efficient algorithm to parallelize a job into tasks which are al-
located to a set of processors whenever a processor is free. The parallel problem has then concentrated on reducing the complexity of the parallel algorithms for more efficiency. However, the Grid computing assumes tasks come from independent users and therefore are already independent. Consequently, dividing a task into jobs is not a major concern here. While in traditional parallel computing, and in most cases, the processors are homogeneous and static [12], simplifying a lot the underlying models, the Grid is characterized by a highly heterogeneous, networked, and distributed dynamic resources. First, resources can be added and removed from the pool of resources as resources owners may wish to integrate or remove their resources from the pool of available resources. Second, the public use of the resources contributes to the dynamic capabilities of the resources that should be rediscovered dynamically before any decision of allocation can be made. Therefore the efficiency of the allocation mechanisms under these conditions is a major concern. [13] has proposed a dynamic allocation model, using linear programming techniques, for a group of tasks on heterogeneous processors with different task-type-based suitability. The model is based on the suitability of task execution, measured on average weighted instruction times, on a processor. These models are not directly applicable to a Grid environment. Furthermore, the Grid allocation algorithms should consider the heterogeneity of a large-scale network connecting Grid resources. They should consider the network delays due to network utilization. While in parallel processing, different topologies have been studied to find the most efficient topology [15], the network was not considered to be a major issue. III. ALLOCATION EFFICIENCY MODEL ACTORS AND FUNCTIONS The formal model we define in this paper is based on our previous experience [7] in implementing a testbed for resource allocation in a Grid environment. We define the actors of the model as follows:
686 680
•
Task. A task is a user application or service which is characterized by its requirements in terms of an SLA. The OGF recommends the use of SLAs to represent users’ requirements (section 2.1 of [19]). In our model, the SLA is represented by a vector of required performance metrics, such as the CPU (cycles per second), memory size (Kbytes), Disk (Kbytes), etc. In addition, a task may require to execute for a maximum duration in the system. The relationship between the resources requirements, the duration of execution time, and the computational power of Grid execution nodes is analyzed throughout the model. The model does not take into account the possible migration of a task caused by a violation in the SLA. Once a task is submitted to an execution node, it will stay there until it finishes execution if it has a known duration. A task is represented by the following formula: T = ((r1 , r2 , r3 ), τ )
(1)
•
•
•
where r1, r2 and r3 represent resources requirements. τ represents the execution time duration in the system. During this time, these resources are being used by the task. Resource. A resource is a unit of structuring the computational power of the Grid. A Grid is a collection of resources. For our experiments, we mainly consider the CPU resource usage. But, the model is generic for all resources required by a task. Controller. Every resource allocation algorithm must define a controller. A controller is an entity which is responsible to read the available tasks from its queue of tasks and allocate them to suitable resources using a resource allocation algorithm. The controller has to read information about the resources’ actual capacities provided by the execution nodes (owners of the resources) before making any allocation decision. In a large-scale distributed network, we can have a controller per a local area network or per domain. Assuming the existence of a controller, this paper provides methods for analyzing its performance. Computing node. Each computing node is represented by a set of resources. An execution node must provide resources information to the controller about resources load, configuration, etc. We model an execution node by a vector of resources, for instance: EN = (r1, r2, r3) (2) where r1, r2, and r3 represent the different resources of the execution node (EN).
The different functions of the model are defined as follows: •
•
•
Task creation. Whenever a task is created, a request is sent to the controller. The request includes the requirements of the task in terms of resources. The task will be stored in the controller’s queue until it is allocated to a suitable execution node by the controller. Tasks arrivals and departures. Tasks arrivals and departures are the events of interests to our model. We assume that in any finite interval, only a number of finite number of events occur. Therefore, the tasks arrivals and departures are point processes. On a completion of a task executing on an execution node, some resources become available. The execution node sends an update to the controller about resources availability. Not all the tasks have the same duration of execution on a node. A task which arrives into a Grid may starve. Therefore, starvation should be considered while analyzing the efficiency of resource allocation algorithms. However, our model only considers limited starvation. Dynamic task requirements changes. The task requirements may vary over time (section 3.2.2 of [19]). This is the case of a task service which has a peak service during certain hours of a day. Our model takes into consideration the resources which are consumed during the lifetime of a task, whether they are dynamically changing or not.
IV. GRID FORMAL MODEL In this section, we define a Grid global behavior. After analyzing the relationship between a task load and the power of the machine on which it is executing, we derive the expected resource use efficiency which represents the efficiency of an allocation algorithm. A. Global Grid Behaviour Model Some principles are universally true of Grid systems. Machines are connected in a network. Each machine may generate tasks (perhaps from human actions) and these tasks are allocated to be run on some machine. It is possible to state at each time which tasks are executing on which machines. This is the principle of task localisation. It is universally assumed, but usually not stated. Some patterns of task execution do not occur. Tasks clash. They might clash through resources: a machine with 3Mb of memory cannot execute at the same time two tasks that each require 2Mb. They might clash through security policy: a certified task refusing to execute concurrently with an uncertified task. Cross machine constraints exist: one task executing on one machine, only if a related task is executing on a different machine. Consider a grid as a system of machines in which tasks appear, disappear, and move around from machine to machine. The migration of a task is the result of its own actions, the action of other tasks, and the action of the operating system on each machine. Regardless of whether the machines are cooperating or competing, there is law to the migration of the tasks. This law is the conceptual global grid controler. Given the states of all the tasks, what happens next depends on parts of the states of all the machines, this, taken together, is the state of the global grid controler. Our analysis concentrates on this global behaviour, independent of its implementation. Our model separates the effect of the global behaviour from the means of achieving it. Speaking of a central controller is a metaphor that abstracts the Grid algorithm from each machine. Our model does not require that there really is a physical central controller machine with global network control. B. Power and efficiency The time required for a task to complete on a lone machine may depend on the machine and the other tasks running. The inverse of this time is the computational power provided to the task under the given conditions. But, one machine might interpret Java and compile C#, and another compile Java and interpret C#. The relative power presented to two specified tasks depends on the machine. For a given task arrival history, presented to a Grid, there is a given task departure history. Some time passes between the arrival of the first task, and the departure of the last task to leave. The shorter the time the more powerful the Grid. The inverse of the time measures the Grid power for that task arrival history.
687 681
C. The Fundamental Model: Relationship between Task Load and Machine Power For a simple model, each task is a self-contained machinecode block; the instructions in which execute in a definite sequence. Computational power is machine cycles per second; for machines that vary only in the memory and clock rate, this is well defined. If for any two tasks the ratio of time taken is independent of the machine then the ratio of time taken to time for a reference task can replace the task machine-cycle count. Each task consistently presents a computational load (energy), and each While a task runs it has an allocation of resources R(t) at each time t. With an otherwise isolated task asking for a percentage of the CPU, the constraint is implicitly that a certain number of machine cycles is required to complete the operation. As long as at least this number of cycles is dedicated to this task, Rit will complete correctly. This is an energy-like −∞ contstraint: 0 R(t)dt ≥ L. Memory use, however, may be required to be enough at each time for the task to run. ∀t : R(t) ≥ L(t). This is a power-like constraint. Constraints on task allocation can be complex, tasks might require not just a total memory-time product, but that memory never drops below a given minimum. However it occurs, some memory-time is allocated. Our derivation is about the relation between machine resource efficiency and Grid resource efficiency, regardless of how the tasks are allocated. Each resource is measured by a non-negative real number. The capacity of a machine is a tupple of non-negative real numbers showing its capacity to provide each resource. The requirement of each task is an analogous tupple of real numbers. Let Rir be the requirements of task i for resource r. Let Ami be between 0 and 1 to indicate what fraction of the tasks requirements machine m is giving task i. Let Cmr be the capacity of machine m for resource r. Then at all times in the system, 0 ≤ AR ≤ C, where ≤ is an element-wise comparison. These matrices are functions of time. The task requirement is L(A) > R, where L is a linear operator possibly involving integrals. For example a memory requirement might be that memory supplied is always greater than 1Mb, while a cpu power requirement would be that the integral of the power supplied equals the total cpu energy required (eg total machine cycles). A machine provides memory, bandwidth, printers, power, etc. The quantity is a non negative real number. A task might need a resource, such as security certification, without using it up, or use it exclusively, such as memory. Instantaneous task requirements might be unaffected by speed up, or increase linearly: memory versus bandwidth. (Memory load might be a memory-power integral, while communications load is the bandwidth integral, no power scaling). Many task requirements can be modeled this way. The need to run on a certain type of processor, for example. Each processor of the given type has a resource with value 1 for its type. A task that needs this processor type uses this resource
very briefly at the start of its run, and then returns it. If this pulse of requirement is an impulse of infinitesimal length, then the chance of a clash over this resource is zero. It is more difficult if the type can change over time. For example, a machine not running a program written by software house X. D. Power scaling Let τ be how far through a task is (0 ≤ τ ≤ 1) and R(τ ) the resource used at this fraction of completion. Let p be the cpu-power. Then the scaling law is: At time t resource use is R(p × t). (3) Although natural, the scaling law is not automatically respected. An adaptable task runs faster when given more memory. But, not much software adapts. In practice scaling with computational power actually used by (not power provided to) the task does occur. One limitation is that, if a task is bound by external interactions (perhaps network communications), then allocating more computational power will not speed it up. Commercial pressure maintains approximately constant time ratios. If one machine is relatively better at a common task, then the manufacturers of other machines will adjust their machines to spread the ability over all common tasks. (1) Time ratios under identical conditions depend only on the tasks. Exceptions include graphics and maths coprocessor machines. (2) Software will scale all performance with computational power. Exception, when code is limited by communication bandwidth. (3) Software cannot scale performance with any other resource. Possible exceptions, memorytime trade off in adaptable code. For a single task presenting load L, and a machine of power P, the time taken is L/P seconds to complete. E. Expected Resource Use Efficiency In this section, we derive a formula for computing the expected resource use efficiency for a Grid resource allocation algorithm. This efficiency is computed from externally measurable properties of the Grid. A single processor tackling a stream of tasks can run one after the other and get one hundred percent machine use – on average this is as fast as tasks can expect to go through, a single task gains beyond the first come first served algorithm only to the detriment of other tasks. A Grid computer, however, might allocate tasks to an inappropriate machine, overloading one, and underloading another, thus the power of the Grid is not always used to capacity. The total nominal power of the Grid is the sum of the individual powers, the efficiency is the ratio of the the power achieved to the nominal power. In a task-resource allocation algorithm, the task allocations are deterministic in principle, but a stochastic model can give simple approximation to the behaviour, allowing analysis and synthesis. The deterministic detailed state of all the machines is replaced by a probability density over a space of simpler states. Let us first take a look at the mean resource allocation per task. For a sequence of n tasks and a total resource
688 682
allocated of a, the mean resource per task is a/n. Weight this mean by the time spent in the Grid. Alternatively, for the resources a(t) allocated in the system at time t, and the number of tasks in the system n(t), the allocation per task is a(t)/n(t) at time t, and the temporal mean of this is the mean allocation per task. These two ideas are EI(a/n) versus EI(a/n). It would be nice if these were equal for all tasks arrivals and departures, but they are not. We need some conditions under which they are equal. We found that we require: 1) task allocation is ergodic (ergodicity principle), 2) expected execution time is finite (starvation is limited but not necessarily zero) (liveness principle), and 3) temporal means exist (temporal mean principle). We consider these conditions to be weak enough to generalize our results. Let ai (t) be the resources allocated to task i at time t. The allocated resources is a positive indefinite function of all time. When the task is not in the Grid, its allocation function will be 0. A task also has an execution indicator function. It is 0 when the task is not on execution queue, and 1 when it is. The support of the execution function is a superset of the support of the allocation function. The execution function embodies the assumption that it is possible to tell when the task is running, or not. Let N (t1 , t2 ) be the number of tasks that whose execution function is non zero at some time between t1 and t2 . In the physical analogy, ai (t) is power, the energy allocated to a task over an interval is the intergral of ai (t) over that interval. Thus, given specific allocation functions, the temporal mean energy allocated to tasks from time 0 to T is defined as follows. P∞ R t ai (τ )dτ 0 m(t) = i=1N (0,t) The temporal mean M resource allocation is limt→∞ m(t). For example, if ai (t) is CPU-cycles per second, then the integral is cpu-cycles used, and the temporal mean is the cpucycles that an ‘average’ task uses to complete its operation. Finite Task Density Principle: Generically, the order of the sum and the integral above might change the result. However, in a practical Grid the number of tasks is always finite, in fact is bounded above by a finite number. Thus the summation is really a finite sum, and does not tend to an infinite sum. Temporal Mean Existence Principle: The temporal mean M does not have to exist, other than tending to infinity, the value m(t) might oscilate for ever while remaining finite. The principle of the existence of the temporal mean is that M is defined, and finite, for all possible allocation sequences. The tasks are randomly generated. Each outcome of the sample space is a countable infinity of allocation functions (the precise execution function is not important, to simplify, it is assumed to have minimal closure of support). Not all allocation functions have finite support. So, the expression the mean integral allocation might not be well defined. However, it is assumed that non finite support is statistically insignificant, in the sense that the following limit is finite. RT A = limT →∞ EI 0 ai (t)dt
Ergodic Principle: The value A is the ensemble mean integral allocation to the task. Although there is clearly a strong connection between A and M , they might be different numbers. Effectively, A is the mean (from the probability) temporal mean. The ergodic principle states that the temporal mean is almost (measure zero) always equal to the ensemble mean. Liveness Principle: While tasks taking an infinite amount of time in the Grid are allowed, they are assumed statistically insignificant. That is, the expected length of the execution interval of each task is finite. The exact definition is given by replacing ai in the above formula for A, by the execution function. By limiting the analysis of the Grid to a finite window, this principle is made automatically correct in any practical circumstances. The finite window could be plus or minus one million years, so it covers most situations to be considered. Of course, this is not a good liveness principle to prove about a system, as many tasks may starve, or take prohibatively long to complete. This liveness principle has been selected as the weakest one that must be assumed for the derivation of the Grid efficiency technique to be correct. Or, at least, a simple such one that is always true in practice. Principle of machine power: This principle states informally, that it is possible to allocate a single number, the power, to a machine, in practice a ratio so that it is consistently possible to say that machine-x is, for example, twice as powerful as the standard machine. More precisely, if two tasks are put on one machine, the total load presented might be r1 and r2 , and on another machine s1 and s2 . It is assumed that r1 s2 = s1 r2 . The principle of machine power states that this is true for any two tasks. Thus, tasks present a consistent relative load in the Grid. For example, suppose the resource is cpu time. A task will take less time on a faster machine, but still be twice as fast as some other task, no matter the speed. This principle is less certain that the others. Taking the resource as clock time, if one task allocates a lot of memory, and is put on a machine that has little, thus causing it to use virtual memory on a hard disk, then it may slow down disproportionately. Similarly, the existence of co-processors, graphics or mathematics, could also make the relative load inconsistent. A safer resource is the cpu cycles actually used by the task. This is not changed by the speed of the cpu. However, it may be changed by the details of an optimising compiler. If the resource is taken as percent of cpu power (cycles per second at each time), then if the test machine runs at a different speed, and real time is measured rather than cpu cycles, a (consistent) ratio will exist between the measured task total load on the test and on the Grid machines. Thus, the correct load to use in the Grid is α times the load measured on the test machine. This α measures the power of the Grid machines, in proportion to the power of the standard reference machine. Theorem. Under the three P principles given above, the following formula holds a ¯ T = r i τi
689 683
Proof. Lets look at time behaviour. For a given single run, look at all the tasks that start between time 0 and time T . Task i takes τi time, and ri resources to run, where τi and ri are the amount of time and resources that a task would take if it was run on a dedicated machine. Let N be the number of tasks in this particular run. So, T , ri , τi , and N describe the particular run (ignoring the arrival times of the tasks), and they are definite constant numbers for each run. Let n(t) be a function giving the number of tasks on the machine at a particular time. Let a(t) be the total amount of resources allocated at any time t. The total amount of task×time used can be computed from thePtasks, or from R T n. N τ = n(t)dt = n ¯T (5) i i=1 0 where n ¯ is the average number of tasks in the system between 0 and T . Similarly, the total amount of resource×time is computed as P R RT N 1 T ¯T (6) i=1 ri τi = 0 a(t)dt = ( T 0 a(t)dt) × T = a where a ¯ is the average resource usage between 0 and T . So using equations (5) and (6) above, we obtain the following: N N X X a ¯ r i τi = τi (7) n ¯ i=1 i=1 P a ¯ T = ri τi (8) In this equation the righthand side is computed from measurements made on the test machine. And the lefthand side from measurements made on the Grid. According to the model this should be true independent of the number of tasks, their requirements and their distribution. Most of the principles of the model, for example that there are a finite number of tasks on the Grid at any time, are clearly true from common understanding of Grids. The weak point of the theory is the principle of machine power. If we cannot say that one machine, for example, is always twice as powerful as another, then the model breaks down. Given detailed access to the Grid machines, the ratio PT a¯
sufficiently fast as the time goes to infinity, then the influence of a task, does not extend beyond a certain time with any high probability. This feels like another characterization of an ergodic process, and it seems physically reasonable for machinePtasks. P P P ri = na¯¯ , then ri τi = ( ri )( τi ) (8) If N1 In this context there are actually several senses in which we can look for ergodicity, since we have the discrete sequence of task parameters (ri , τi ) and the temporal evolution of the system as a whole. a ¯ If we take the limit T → ∞, then n¯ , is a constant. So, P N EI na¯¯ N1 i=1 τi (9) P N (10) is na¯¯ EI N1 i=1 τi is na¯¯ EI(τ ) (11) On the assumption that expected value of the statistical average is the same as the expected value. Which is true in the ergodic case. Call resource×time the load. Conclusion: the expected temporal mean resource usage per task is the ratio of the expected load to expected time per task. IE(rτ ) a ¯ (12) n ¯ = IE(τ ) Checking the units this seems to be resource/task = resource*time/time, but in fact, the τ on the bottom is really task*time, but with the value of task being 1, check the origin of the term in the derivation. a ¯ is a figure of merit for algorithms, the bigger average load is the faster the tasks are being passed through. PN Since, using equation (7), T = f a1¯ i=1 ri τi , where PN ¯ the i=1 ri τi is the constant test load. So the bigger is a smaller is T . As a test that means to pick a random time, and test the load. Do that for different algorithms. The alternative would require keeping track of individual tasks over a period of time to see how long before they left the system, which is a very non local measurement.
ri τi
can be computed. This determines the value of α, that is the relative power of the test machine. The principle of machine power states that this number should be (roughly) constant. And so these measurements can be used to calibrate, and to validate the model. If the spread of measured relative powers is small enough then the model is valid. ) Theorem. na¯¯ = IEIE(rτ (τ ) Proof. For a system to be non ergodic, it must be possible for the influence (correlation) of events to extend indefinitely. That is, for any amount of time specified you can find correlations between statistics of the system that extend over that amount of time. Physically, it is like saying, we know that the presence of a task now, using resources, can effect the allocation for other tasks, even if those tasks do not communicate, the fact of the resource usage makes them interact. But, eventually, assuming the that probability of tasks which stay in the system for a long time goes to zero
V. EVALUATION We developed tasks processing and execution tools to verify the validity of the model in our experimental environment. Our hardware environment consists of a set of interconnected machines deployed in a local area network. Our main objective is to show that the α for which the equation is true, is constant over all thePexperiments. We then have validated our equation N Ta ¯ = α i=1 ri τi that we derived from the model. The resource used in these experiments is the CPU usage. We run batches of random tasks on a Grid system. The set of tasks is generated randomly, but we always use the same seed, so that we can reproduce the same task generation in different configurations. Then we measure the time T for a batch of tasks to terminate and the a ¯ which is the average load on the machines. In a separate experiment, these tasks are run sequentially one at a time on one computing node; a reference test machine: the duration of each task and their resource usage
690 684
TABLE I H ARDWARE C ONFIGURATION
requirement PN(CPU cycles) are recorded in order to calculate the sum i=1 ri τi .
Controller
Computing Node
Number of Cpus
Dual processor, dual core
Dual processor, dual core
Cpu Type
AMD Opteron processor 252
Intel Xeon CPU 3.06GHz
Cpu Mhz
2600
3056.911
Total Memory
2 GBytes
4 GBytes
Operating System
Red Hat ES release 4
Red Hat ES release 2.1
A. Experimental Environment We used our software implementation which consists of a Grid controller and different Grid services running on the computing nodes. The Grid controller, on one side, has an interface to accept task requests. On the other side, it maintains an updated view of the resource consumption status of the different computing nodes. The tasks are generated by an application on a separate system, the characteristics of which are not important for the purpose of our evaluation.
TABLE PIIN M EASURED rτ i=1 i i
B. Controller Software The controller implements the algorithm that places tasks on computing nodes: it uses the SLA parameters of the task and the status of the resource usage on the computing nodes. We use a first fit algorithm to place tasks on machines. For each task we look at the list of machines in sequence, and the task is placed on the first one where enough resources are available.
Batch
PN
rτ i=1 i i
100 tasks
339035
200 tasks
679635
C. Computing Node Software
F. Experiments
The software on the computing nodes offers an interface to start a given task. It also maintains the exact status of the computing nodes resources: CPU, memory and disk usage. It implements an interface to query this status, which is used by the controller on a regular basis.
To compute α, we made two series of measurements for two tasks batches: 100 and 200 tasks. Each task has a resource requirement. In our experiment, the resource requirement is a percentage of CPU. We first ran each task one at a time on a single machine (dedicated reference test machine), where no PNother tasks were running. This was to calculate the sum i=1 ri τi , where τi is the time (seconds) required to run the task, and ri is the resource requirement. This measured sum is reported in table II. Then the same batch of tasks was submitted to the Grid controller in 4 different Grid configurations: with respectively 1, 2, 5 and 10 computing nodes. For each run we recorded the total time needed to finish the batch (Figures 2 and 3), and for each machine the average load observed during the duration of the batch. The average load of the Grid is then computed from the average load on each machine (Figure 4). We can therefore calculate α, which represents the ratio of the Grid power to the power of the dedicated reference test machine: α = PNa¯T . In our experimental environment, the power
D. Task Simulation The tasks are simulated with a single program that accepts parameters, implementing different behaviors. Namely the task has a average CPU usage (expressed in % of a CPU), this CPU usage is maintained during the life of the task. It has also memory and disk usage parameters. All these parameters and the actual amount of work (number of cycles) are randomly generated by the task-submission application. This task simulation application is mono-threaded. The simulation task simulation software can be configured to stress any of the resources: CPU, memory and disk. For the purpose of our experiments, we deliberately oriented the generated load to be CPU intensive. This makes the whole batch of tasks mainly CPU-bounded. We chose this option because Grid systems are often used in this CPU intensive context. Moreover, the dynamic nature of the CPU usage produces a more realistic and meaningful context for testing the formal model. Whereas a memory and disk loads have a more static influence on the needed time to finish a batch of tasks, as introduced by our formal model. We used constant inter arrival times for the tasks. The parameters of a task (CPU usage, memory usage, disk usage) are distributed uniformly between [0, max] (see with Alain)
i=1
ri τi
ratio is nearly constant 1.8 (Figure 5). Considering that our cluster is homogeneous, the power ratio was also measured as follows: We ran a simple task in two configurations, without any Grid software running. We measured the time it took the task to complete (Time1). Then we ran four instances of the same task in parallel and recorded the duration for each task and compute the average (Time2). Results are summarized in Table III. We found that the ration is nearly 1.8. This is the power ratio α in our experimental environment. VI. RELATED WORK
E. Hardware Environment Our hardware environment consists of a controller node and 10 computing nodes as described in Table I.
Dynamic resource allocation in a Grid environment is challenging due to the dynamicity of large-scale shared distributed
691 685
Fig. 4. Fig. 2.
Cluster Computing Node Average CPU Usage
Total Duration for a Run of 100 Tasks
Fig. 5. Fig. 3.
The Value of Power Ratio α
Total Duration for a Run of 200 Tasks
heterogeneous resources. Most research work in Grid computing deal with resource allocation algorithms. These works are divided into: economicbased algorithms or non-economic-based. In the economicbased approaches, resources are allocated according to a pricing agreement reached between the clients and the service providers [14] [15] [16]. In these algorithms tasks are generally prioritized according to their bids. A proportional shared protocol can also be used, where a task priority is computed based to its bid and the total bids in the system. In the non-economic-based approaches, resources are allocated according to applications’ requirements of resources using different allocation algorithms such as round robin, loadbalance-based scheduling [17], etc. Currently the efficiency of the resource allocation algorithms are tested via software simulation of a Grid computing environment. In this work, we aim to provide a formal model for computing the efficiency of resource allocation. Therefore, we formally analyze the impact of an arbitrary resource allocation to a Grid environment. We believe that our model is a step forward in formally studying the performance of different scheduling algorithms in a Grid environment. VII. CONCLUSION AND PERSPECTIVES In this paper, we presented a formal model of Grid environment. This allows us to state and analyze the different
parameters that affect the efficiency of allocation algorithms in a Grid, which was one of our objectives. Our model devised a relationship between resource use by the tasks and their duration within the system. This relationship helps resource allocation algorithms developers, such as researchers and students, who do not have access to an internal Grid to analyze the efficiency of their algorithms. Since ri and τi are measured from our test machine, and T is the total time between submitting the first task, and the termination of the last task to complete, all these numbers are available to the experimenter, without internal access to the Grid. So, from the model, the relative efficiency of algorithms can be determined by computing the Grid average load using the formula P ri τi a ¯=α T for each algorithm, and examining the ratios of these numbers. To validate this relationship, we conducted experiments which measure the performance of a first fit allocation mechanism in our experimental Grid system that we have implemented and that we have access to. Our model helps researchers and students to test their allocation algorithms from a machine they control connected to a Grid that they do not control, but may request services from. The global Grid controller is the combination of the algorithms on the researchers/students’ machines, and those algorithms used by the other authorities on the Grid. For one specific algorithm, the computed a ¯ depends on the details of the tasks. And, without the calibration, a ¯ is
692 686
TABLE III S CALABILITY
EFFICIENCY OF COMPUTING NODES
Time1
Time2
Ratio
35.0
64.2
1.829
36.120
68.3
1.89
uninteresting. However, if the exact same run of tasks is submitted according to two distinct algorithms, and the ratio of the measured a ¯ computed, then the result should not depend on the nature of the tasks. In this case, the model is calibrated on the Grid without any requirement for internal access to the Grid machines. While the efficiency of the algorithms cannot be determined, the relative efficiency can, and this is enough to choose between algorithms. And now estimates of the expected performance on other task runs can be made. Our plan is to conduct experimental validation of our model on public Grid. This could be done directly on the uncontrolled Grid. The a ¯s are computed for different allocation algorithms, but the same tasks, using the model. The ratio of the a ¯ should be constant, independent of the tasks.
[13] Leslie Jill Miller. ”A heterogeneous multiprocessor design and the distributed scheduling of its task group workload”. International Conference on Computer Architecture. Proceedings of the 9th annual symposium on Computer Architecture, pp. 283 - 290, 1982 [14] Michal Feldman, Kevin Laiy, and Li Zhangy. ”A PriceAnticipating Resource Allocation Mechanism for Distributed Shared Clusters”. ACM Conference on Electronic Commerce 2005 [15] J. Gomoluch and M. Schroeder. ”Performance Evaluation of Marketbased Resource Allocation for Grid Computing”. Concurrency and Computation: Practice and Experience, pp. 1-6, 2004 [16] Gomoluch and M. Schroeder. ”Market-Based Resource Allocation for Grid Computing: A Model and Simulation”. 1st Int’l Workshop on Middleware for Grid Computing (MGC ‘03), 2003 [17] Tao Wang, Xing-she Zhou, Qiu-rang Liu, Zhi-yi Yang, and Yun-lan Wang. ”An Adaptive Resource Scheduling Algorithm for Computational Grid. Proceedings of the 2006 IEEE Asia-Pacific Conference on Services Computing (APSCC’06), 2006 [18] Open Grid Forum (OGF). http://www.ogf.org/ [19] Karl Czajkowski, Stephen Pickles, Jim Pruyne, Volker Sander. ”Usage Scenarios for a Grid Resource Allocation Agreement Protocol”. Global Grid Forum, 06 February 2003
R EFERENCES [1] Ian Foster. ”What is the Grid? A Three Point Checklist”. Argonne National Laboratory and University of Chicago”. 20 July 2002 [2] Ian Foster and Carl Kesselman. ”The Grid: Blueprint for a Future Computing Infrastructure”. Morgan Kaufmann, 1998 [3] Ahmar Abbas. ”Grid Computing: A Practical Guide to Technology and Applications”. Charles River Media, 2004, ISBN 1584502762 [4] Fran Berman, Geoffrey Fox, Anthony J.G. Hey. ”Grid Computing: Making The Global Infrastructure a Reality”. Wiley Series in Communications, Networking and Distributed System, 2003, ISBN 0470853190 [5] Mark Baker, Rajkumar Buyya, and Domenico Laforenza. ”Grids and grid technologies for wide-area distributed computing”. International Journal of Software: Practice and Experience (SPE), 32(15):1437–1466, December 2002 [6] I. Foster, C. Kesselman, and S. Tuecke. ”The Anatomy of the Grid: Enabling Scalable Virtual Organizations”. International J. Supercomputer Applications, 15(3), 2001 [7] Leila Ismail. ”Dynamic Resource Allocation Mechanisms for Grid Computing”. IEEE International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities (TridentCom2007), 2007 [8] Jeffrey O. Kephar, David M. Chess. ”The Vision of Autonomic Computing”. IEEE Computer Society, pages 41-50, 2003 [9] Ken Kennedy, Mark Mazina, John Mellor-Crummey, Keith Cooper, Linda Torczon, Fran Berman, Andrew Chien, Holly Dail, Otto Sievert, Dave Angulo, Ian Foster, Dennis Gannon, Lennart Johnsson, Carl Kesselman, Ruth Aydt, Daniel Reed, Jack Dongarra, Sathish Vadhiyar and Rich Wolski. ”Toward a Framework for Preparing and Executing Adaptive Grid Programs”. Globus Alliance Research [10] I. Foster, Argonne and U.Chicago (Editor), H. Kishimoto, Fujitsu (Editor), A. Savva, Fujitsu (Editor), D. Berry, NeSC, A. Djaoui, CCLRCRAL A. Grimshaw, UVa, B. Horn, IBM, F. Maciel, Hitachi, F. Siebenlist, ANL, R. Subramaniam, Intel, J. Treadwell, HP, J. Von Reich, HP. ”The Open Grid Services Architecture, Version 1.0”. GFD-I.030. http://forge.gridforum.org/projects/ogsa-wg [11] Allen B. Downey. ”A parallel workload model and its implications for processor allocation”. Proceedings of the 6th International Symposium on High Performance Distributed Computing (HPDC’97), ISBN:0-81868117-9, page 112, 1997 [12] Ed Andert. ”A Simulation of Dynamic Task Allocation in a Distributed Computer System”. Proceedings of the 19th conference on Winter simulation. Pages: 768 - 776, 1987, ISBN: 0-911801-32-4
693 687
