Controlling datacenter power consumption while ... - IEEE Xplore

2014 IEEE 3rd International Conference on Cloud Networking (CloudNet)

Controlling datacenter power consumption while maintaining temperature and QoS levels Sergio Nesmachnow

Cristian Perfumo

Íñigo Goiri

Universidad de la República, Uruguay Email: [email protected]

CSIRO Energy, Australia Email: [email protected]

Rutgers University, USA Email: [email protected]

Abstract—The large amount of energy used by datacenters impacts both energy cost and the electricity grid. These issues can be mitigated by dynamically adjusting the power demand of datacenters. However, conflicting objectives have to be considered: workload and cooling can be dynamically reduced, but with a potential impact on quality of service or excursions beyond acceptable temperature bands. In this paper we use a multiobjective evolutionary algorithm to explore the trade-offs between power consumption, temperature, and quality of service when both servers and cooling are controlled holistically in a datacenter. Keywords—scheduling; power management; datacenters

I.

I NTRODUCTION

Datacenters account for approximately 1.5% of world energy usage [1], and their electricity bills represent 5% of the total ownership cost [2]. Thus, owners and operators are highly interested in “greener” energy-efficient datacenters that integrate renewable generation [3]. There is also a growing interest in dynamically controlling the operation of datacenters based on one or multiple energy-related signals [3][4][5]. We call this technique energy-aware datacenter control.

The experimental evaluation, performed considering a set of realistic workloads and hardware scenarios, suggests that the proposed approach is a useful alternative for power management in datacenters. When compared against a business-asusual (BAU) scenario, the proposed MOEA is able to compute solutions with up to 75% improvement on power tracking and 83% on the temperature values, with very low degradation (below 10%) in the QoS metrics. The paper is organized as follows. Section II reviews the related work about control and energy-aware scheduling in datacenters. Our model and control strategy for energyaware datacenters is described in Section III, including a description of the multi-objective optimization problem solved. The proposed MOEA for energy control is described in Section IV. The experimental evaluation is reported and discussed in Section V. Finally, Section VI presents the conclusions and formulates the main lines for future work. II.

R ELATED WORK

A. Control and energy aware datacenters

Adjusting datacenter power dynamically allows reducing electricity costs by shifting operation to periods of cheaper energy. Furthermore, if the datacenter is partially powered by intermittent renewable energy such as solar or wind, an energy-aware controller can allocate tasks around available energy, storage, cooling [3][4]. Datacenters can also participate in the energy market as ancillary service providers that can respond to a ramped down (or, occasionally up) signals during critical times, thus reducing the pressure for upgrades in power generation, transmission, and distribution infrastructure. It is estimated that each $1 invested on demand side management offsets $2 spent in supply side improvements [6]. Indirectly, these savings lead to a reduction of greenhouse emissions.

GreenSlot [7] considers job allocation for highperformance computing (HPC) applications in a datacenter powered by solar panels, using job information (nodes per job, deadline, estimated runtime) to schedule jobs when solar energy is available. GreenHadoop [8] also considers green generation plus grid energy prices to allocate MapReduce jobs using heuristics to predict the energy requirements GreenSwitch [3] also considers energy storage (i.e., batteries and net metering) and uses a holistic approach for managing energy sources and workloads to minimize the total electricalrelated costs. This approach is evaluated on Parasol, a solar-powered micro-datacenter [3]. In contrast, we propose to adapt the power consumption of a datacenter (including cooling) to the requirements of the power grid.

This paper presents a multi-objective approach to regulate power consumption taking into account the interplay between computational load and cooling infrastructure. This control mechanism allows the datacenter to follow a power reference profile while minimizing impact on quality of service (QoS) and temperature. The proposed controller uses a multiobjective evolutionary algorithm (MOEA) to solve the problem, which provides multiple trade-off solutions, helping the datacenter planner to explore different options for controlling the performance and energy consumption. A specific energyaware backfilling algorithm is used to schedule tasks according to the power constraints and QoS defined by task deadlines.

An alternative to shifting load is to trade off energy and QoS. Krioukov et al. [9] presented an energy-aware scheduler for a datacenter to follow wind generation, which can both use less power by degrading the quality of the responses (cached pages instead of dynamically-generated ones) and delay requests with slack. Fast control of power demand enables datacenters as providers of ancillary services. Aikema et al. [10] presented simulation results for a datacenter participating in the New York ancillary service market. A selective approach is applied in [11], where participation as an ancillary services provider is determined by expected profits (the compensation is weighed against the SLA penalties).

978-1-4799-2730-2/14/$31.00 ©2014 IEEE

242


The most similar existing work to that presented in this paper is [4], where, based on predicted power output and datacenter load, non-critical jobs are scheduled so that the solarpowered datacenter achieves a net-zero operational energy on a daily basis. The main differences between [4] and the present paper are: (a) we look at fine time granularity (minutes), which broadens the applications of datacenter energy management, for example by enabling ancillary service participation, and (b) we approach the problem from a multi-objective point of view, thus considering a range of trade-off solutions between temperature violations, QoS and power.

Fig. 1.

Diagram of the inputs and outputs of the datacenter.

B. Energy aware scheduling Two optimization approaches are applied for energy-aware scheduling: i) the independent approach assumes energy and performance as independent; and ii) the simultaneous approach optimizes performance and energy at the same time, modeling the problem as a multi-objective optimization. In this article, we follow the simultaneous approach. Khan and Ahmad [12] applied a cooperative game theory approach for scheduling independent jobs, simultaneously minimizing makespan and energy on a grid system. Lee and Zomaya [13] studied several heuristics to minimize the weighted sum of makespan and energy using a makespan conservative local search to slightly modify scheduling decisions when they do not increase energy consumption, in order to escape from local optima. Kim et al. [14] studied the deadline constrained scheduling problem in ad-hoc grids with limitedcharge batteries, proposing a resource manager to exploit the task heterogeneity while managing energy. Our previous work [15] introduced an energy consumption model for multi-core computing systems based on the energy the system requires when all, some and no cores are being used (MIN-MAX mode). We simultaneously optimize makespan and energy consumption when executing tasks on a cluster of multi-core computers. Using the same approach, Iturriaga et al. [16] showed that a parallel multi-objective local search based on Pareto dominance outperforms deterministic heuristics based on the traditional Min-Min strategy. We also apply the MIN-MAX mode in this present article. Recently, Dorronsoro et al. [17] presented a two-level strategy for scheduling large parallel workloads in multicore distributed systems, minimizing both the total computation time and the energy consumption of solutions. The approach combines a higher-level (i.e., between distributed datacenters) and a lower-level (i.e., within each datacenter) scheduler. Good-performance schedules were computed by using scheduling heuristics accounting for both problem objectives in the higher level, and ad-hoc scheduling techniques for multicore infrastructures in the lower level. We adapt a lowlevel scheduler for the problem we consider in this work. III.

C ONTROL APPROACH TO ADAPT DATACENTER POWER

A. Data center model Figure 1 shows the schema of our datacenter model, which is based on micro-datacenter Parasol [3]. Red solid arrows are the control inputs, red dotted arrows represent external disturbances and blue solid arrows represent controllable variables.

243

The control signals (inputs) are variables that we can manipulate to alter the behavior of the datacenter. These signals, which we denote with lower-case letters, are: 1) HVAC control (ck ): the operation of the HVAC system. We consider a datacenter equipped with both free and conventional cooling infrastructure, therefore ck is a shortcut name which actually comprises a set of signals, namely: AC compressor state (binary), free cooling fan speed (%), and free cooling damper state (binary). 2) Schedule (sk ): shape the IT power consumption, considering the number of servers running and load constraints. The controllable variables (outputs) are those variables that we want to control via manipulation of the control signals. We denote them with capital letters: 1) Quality of service (QoSk ): depends on user– and system– related metrics. Those metrics are computed using a specific scheduling strategy applied in the datacenter. 2) Internal temperature (Tk ): thermostat reading of the datacenter (o C) (there could be more than one thermostat) 3) Cooling power (Ck ): the power of the AC plus the power of the free cooling fan (kW). 4) IT power (Ik ): the power of the servers, switches and all IT equipment in the datacenter (kW). 5) Total power (Pk ): the total power used by the datacenter (Pk = Ck + Ik ). The disturbances are variables that affect the datacenter behavior but we have no control over: external temperature.

B. The optimization problem We want to control the datacenter so that its total power demand Pk and temperature Tk follow as closely as possible a desired reference demand and temperature profiles Rk and Tref , while minimizing the impact on QoS. The datacenter executes n tasks in the simulation period (K steps). Each user submission requests execution before a deadline D(i) for task i, and the scheduler executes the task according to the availability of computing resources and energy consumption. Each task i finishes at time F T (i), and QoS is evaluated according the deadline satisfaction/violation.


Formally, we want to minimize: K X k=1

K X

|Pk − Rk |/ max(Rk ) K X

(1)

|Tref − Tk |

(2)

max(0, F T (i) − D(i))

(3)

k=1

i=1

Pareto front. A Pareto-based evolutionary search leads to the first goal, while the second one is accomplished by using specific techniques also used in multimodal function optimization (sharing, crowding, etc.). In this paper we use the Non-dominated Sorting Genetic Algorithm, version II (NSGAII) [20], a popular MOEA that have successfully been applied in many application areas. B. The proposed resolution approach

Objective (1) and (2) specify that we want to track the reference power and temperature as closely as possible. Objective (3) represents the total time of the deadline violations. Our datacenter comprises two subsystems: HVAC (cooling) and IT. IT power Ik is calculated as Ik = Skmax +Skidle +Sksleep where Skmax , Skidle and Sksleep are the total powers of all servers that are executing, idle and sleep at time k respectively. We consider two cooling modes: air conditioning and free cooling. In air conditioning mode, a conventional direct expansion computer room air conditioner (CRAC) is used. The compressor state of the CRAC dominates its power consumption, and can only take two values: on/off. In normal operation, the CRAC regulates temperature by cycling the compressor to maintain the temperature within a specified hysteresis band. In free cooling mode, the CRAC is turned off and outside air is blown into the datacenter by a fan. Formally (CompressorPWR is constant, FanPWR is between 0 and the maximum fan power):  CompressorPWR if in AC mode, compressor ON Ck = 0 if in AC mode, compressor OFF  FanPWR if in free cooling mode, The value of Ck and the cooling mode directly affect the temperature Tk in the datacenter. Based on two months of operation data from Parasol [3], we identified an AutoRegressive eXogenous (ARX) temperature model [18]:

Solution encoding: Each solution represents the amount of cooling power and server power to be used at each time step, encoded as an integer vector of 2K elements. The first K elements represent the cooling power and the second K elements represent server power. The server power is encoded directly as Watts, whereas the cooling power is encoded as an integer value representing three states: (a) 1–100: free cooling mode is applied, and the value represents the fan speed as a percentage of its maximum; (b) 101–200: the air conditioning unit is assumed to be operating, and (c) 201–300: neither air conditioning nor free cooling are in operation. A graphical representation of the solution encoding is shown in Figure 2.

Fig. 2.

Solution encoding.

Optimization functions: The functions to optimize correspond to the ones defined in Section III-B. No modifications are performed to encode them in NSGA-II.

(4)

Evolutionary operators: We use a three-point crossover (points p1 ,p2 , and p3 ); p1 is determined randomly in (1,K), p2 is K and p3 is K + p1 . This way, we ensure that portions representing the same time interval for both cooling and server power move together from parents to offspring.

where ek is the ARX error, A(q) is a second degree polynomial, B(q) is a vector of six first degree polynomials, and uk is a vector representing the six inputs at time k. These inputs are the air conditioning state, fan speed, outside temperature, server load, free cooling damper state and temperature setpoint.

Mutation is applied to each gene with probability pM . For a cooling power gene (position 1 to K), we replace its value v with mod(v + rand()×MAX HVAC,MAX HVAC)). For the other genes, we redefine them with a random value between 0 and the maximum server power (i.e. all servers turned on).

A(q)Tk = B(q)uk + ek

IV.

A N EVOLUTIONARY APPROACH FOR ENERGY- AWARE

C. Energy and QoS-aware scheduling

DATACENTER PLANNING

A. Evolutionary algorithms and NSGA-II Evolutionary algorithms (EAs) are non-deterministic methods that emulate the evolution of species in nature to solve optimization, search, and learning problems [19]. MOEAs [20] have been applied to solve hard optimization problems, obtaining successful results when solving real-life problems in many research areas. Unlike many traditional methods for multiobjective optimization, MOEAs are able to find a set with various trade-off solutions in a single execution, since they work with a population of tentative solutions. MOEAs must be designed taking into account two goals: i) to approximate the Pareto front and ii) to maintain diversity instead of converging to a reduced section of the

244

In order to model realistic task planning in the simulated datacenter, we apply a heuristic energy-aware offline QoS scheduler, based on the method introduced in our previous work [17], but adapted to the specific features of the problem addressed in this article. We assume an error-free estimation for tasks execution times and that tasks’ arrival times are known before the scheduling phase. The proposed method (Best Fit Hole—BFH) applies a backfilling-oriented technique to work with computing resources that are available in certain periods of time (holes). BFH is based on filling holes that are left after a given policy for sleeping/shutting down holes is applied to reduce the energy consumption: tasks are first sorted according to their arrival times, and then assigned to computing resources


to fill their existing holes. If a task can fit into more than one hole without violating the task deadline, the one that “best fits” the task (i.e., the hole that minimizes the difference between the hole duration and task execution time) is selected. Holes within each machine are processed according to their finishing times. A specific logic is included to deal with deferrable tasks: when no hole is available to execute a task, BFH assigns it to the machine that provides the minimum finishing time for that task. The rationale behind this strategy is to use available holes and spare unoccupied large holes and empty machines for upcoming tasks with potential larger execution times. V.

E XPERIMENTAL ANALYSIS

This section reports the experimental analysis of the proposed MOEA for datacenter control. A. Problem instances Problem instances are defined by a workload, a hardware scenario, and a reference power profile. Workloads are sets of tasks. Each task is defined by its arrival time (when received by the front-end in the datacenter), estimated execution time, and deadline (a time by which the user wants the task to have finished). Tasks arrival times follow a Poisson distribution, and durations follow a normal distribution, with average and standard deviations following a typical cluster, according to traces from Cluster FING [21] and the Parallel Workload Archive. Deadlines model different SLA between the provider and the users, according to three slack factors (sf ): tight (sf 30%), which represent the user tolerance considering task duration and arrival time. We also consider deferrable workloads, where 25% of the tasks are allowed to end after the deadline without having a negative impact in the QoS perceived by the user. We study three different workload dimensions: low operation (50 tasks in 150 time steps), normal (75 tasks in 150 time steps), and full steam (100 tasks in 150 time steps). Each time step represents 30 seconds. Scenarios and hardware. The scenarios simulated in this paper consider a time horizon of 75 minutes (150 time steps). We assume a datacenter with average utilization of 50%, to allow for enough spare capacity to shift tasks (utilization values as low as 15-20% have been reported [5]). We assume 64 Atom-based servers in the datacenter, mimicking Parasol [3]. The power consumption of each server is 30W, 22W and 3W at full operation, idle and sleep respectively. Power profiles. We consider three reference power profiles to follow (percentages represent a fraction of the maximum datacenter power, i.e., servers plus cooling): 1) Profile A: 20% during 25 time steps, 80% during 25 time steps and 20% during 25 time steps. This scenario aims to assess how well the system can respond to step changes (both up and down) in power profile. 2) Profile B: 50% during 15 time steps, 80 % during 10 time steps, 20% during 20 time steps, 80% during 10 time steps and 50% during 20 time steps. This is a situation where it is known in advance that demand will have to be dropped in the near future (e.g. a short-term forecast indicates

245

that renewable generation will drop) and we decide to compensate by increasing power demand before and after the drop (either by precooling or by shifting tasks). 3) Profile C: 80% during 25 time steps, then a linear ramp decreasing to 20% during the course of 25 time steps, and then 20% during 25 time steps. This scenario is designed to test how the control responds to ramp changes, which are a very common type of power change in the electricity market (as thermal generators usually can only ramp up or down output, due to their thermodynamic properties). Outside temperature is considered constant at 25o C and the initial inside temperature is assumed 26.5o C. B. Parameter setting After performing an initial study on the trade-off between results quality and execution time of the proposed MOEA, we decided to use a fixed-effort stopping criterion of 500 generations, to solve the problem in realistic execution times. The MOEA parameters were studied using three problem instances, including: population size (#pop, candidate values {50,75,100}), and the probabilities for crossover (pC , candidate values {0.6,0.75,0.9}) and mutation (pM , candidate values {0.01,0.05,0.1}). The best results were obtained using the parameter setting #pop = 75, pC = 0.9, and pM = 0.01. C. Results and discussion All the reported results were computed in 15 independent executions of the proposed MOEA for each problem instance. Best and trade-off results. Table I reports the average improvements on the problem objectives (power P , temperature T , and number of deadlines violations dv) for deferrable/nondeferrable workloads and the three power profiles studied, when comparing with a BAU strategy that does not apply a energy-aware control. The BAU strategy represents a conventional datacenter operation, assuming that all the servers can be utilized at anytime and maintaining the temperature within a 1.5 C band around the desired level, using the AC system. All improvement results are averaged by problem dimension and SLA type. The best improvements obtained for each problem class and dimension are marked in bold. We analyze the best results computed for each objective (best power, best temperature, and best QoS solution). This analysis is useful in case the datacenter planner is mainly interested in prioritizing a specific objective. We also analyze the best trade-off solutions defined as the nearest to the (normalized) ideal objective vector [20] for each problem instance, which corresponds to an ideal solution that equally weights power, temperature, and QoS. The results in Table I demonstrate that both the best power and the best QoS solutions compute the best improvements regarding these metrics, but they have a significant impact on the other objectives. Interestingly, the solution that best follows the temperature profile is able to compute significant improvements on both P and T , having a reasonable number of deadline violations. Furthermore, the best compromise solutions have significant improvements on the energy consumption (between 38-56%), temperature (between 4-58%), while accounting for only about 10% of deadline violations.


TABLE I.

AVERAGE IMPROVEMENTS AND Q O S RESULTS OVER THE BAU STRATEGY ( PER WORKLOAD DIMENSION AND TYPE )

best power solution type non-deferrable deferrable tasks E T dv E T dv 50 70.5% -52.5% 8.5 70.8% -64.3% 8.9 75 71.7% -63.6% 14.7 71.8% -82.9% 15.3 100 74.9% -78.0% 15.2 74.2% -81.2% 17.7 type non-deferrable tasks E T dv 50 52.4% 2.9% 8.1 75 54.6% -24.0% 19.8 100 53.2% -38.6% 23.8

deferrable E T dv 50.7% -22.1% 10.1 52.6% -18.0% 21.6 52.6% -33.8% 20.5

type non-deferrable tasks E T 50 57.5% -144.8% 75 56.1% -140.3% 100 58.6% -153.3%

deferrable E T dv 55.7% -139.9% 7.4 54.8% -146.4% 12.0 58.5% -149.5% 11.5

dv 4.6 6.1 7.0

best temperature solution best QoS Power profile A non-deferrable deferrable non-deferrable E T dv E T dv E T dv 22.5% 81.8% 8.6 20.0% 83.1% 11.7 53.2% -15.8% 0.0 19.2% 82.5% 15.4 23.3% 82.3% 17.7 54.3% -95.8% 0.0 21.1% 82.3% 21.7 19.9% 82.5% 17.1 55.7% -41.2% 0.2 Power profile B non-deferrable deferrable non-deferrable E T dv E T dv E T dv 18.2% 81.8% 10.2 10.3% 82.1% 6.3 27.6% 18.4% 0.1 13.1% 82.6% 15.4 10.9% 81.6% 15.4 15.4% 36.0% 0.0 11.3% 81.9% 15.5 10.8% 82.6% 18.7 33.3% -20.2% 0.9 Power profile C non-deferrable deferrable non-deferrable E T dv E T dv E T dv 11.3% 81.9% 8.3 12.6% 81.3% 8.1 32.9% -50.2% 0.7 11.0% 82.1% 9.3 12.0% 81.4% 14.1 30.9% -85.5% 1.1 18.7% 82.9% 16.2 11.7% 81.1% 7.2 28.3% -8.2% 0.6

The results also indicate that no significantdifferences on the objective function values are obtained when considering deferrable tasks using the proposed scheduler. The reported results indicate that our algorithm is a promising candidate for designing a datacenter controller to decides the most appropriate trade-off between objectives according to the current conditions (e.g., during short periods of very high electricity price, it might be useful to drop power demand at the expense of QoS and temperature). Illustrative Pareto Fronts. Figure 3 presents examples of the Pareto fronts computed by the proposed MOEA for three different (representative) problem instances. The figures show that a good coverage of trade-off solutions is obtained by the proposed MOEA, which appropriately samples the region of (equally-weighted) best compromise solutions for the problem. Solution analysis. Figure 4 presents four solutions from the Pareto front obtained by the algorithm for a problem instance with 100 tasks and power profile A. Figure 4(a) shows that the power closely follows the step-changing reference power, enabling the datacenter to reduce electricity costs, maximize renewable utilization, or participate in the electricity market. For example, if the electricity price was to change from $0.4/kWh during peak time to $0.23/kWh during off-peak time (the case for Australia) the solution in Figure 4(a) would reduce the energy cost in the electricity bill of the datacenter by 16.5%. Figure 4(b) shows that datacenter temperature is tightly maintained within less than 0.5 C from the reference. This solution is 81% better than the BAU solution with respect to temperature. Figure 4(c) shows the best solution in terms of QoS (zero violated deadlines). We see that at the beginning of the simulation most servers are active, which ensures that deadline violations are minimized. Finally, Figure 4(d) presents the trade-off solution from the Pareto front which is closest to the ideal vector. We see that this solution follows the power reference in general lines while maintaining the temperature deviation from the reference at less than 1 C at all times. VI.

This article has presented a model and an optimization algorithm for the problem of operating a datacenter taking into account power demand, temperature and QoS. Satisfactory solutions are found when only one objective is considered (extremes of the Pareto front) as well as interesting trade-off solutions which are clearly superior to the BAU strategy.

246

best trade-off solution

deferrable E T 41.3% -17.1% 47.2% -45.7% 53.8% -60.7%

non-deferrable deferrable dv E T dv E T dv 0.4 52.1% 38.5% 6.2 55.1% 28.4% 5.9 0.5 56.8% 16.1% 10.1 56.7% 23.0% 11.6 0.1 54.9% 28.1% 10.3 58.0% 9.5% 11.1

deferrable E T 41.2% 34.9% 32.3% 29.2% 24.0% 8.2%

non-deferrable deferrable dv E T dv E T dv 0.6 38.3% 58.4% 4.1 38.2% 49.1% 4.9 0.0 44.8% 43.5% 12.0 40.1% 46.8% 11.4 0.5 40.5% 29.8% 11.9 39.3% 46.8% 12.1

deferrable E T 36.1% -29.2% 40.8% -98.9% 36.9% -44.2%

non-deferrable deferrable dv E T dv E T 0.0 41.0% 21.6% 3.5 39.8% 23.9% 0.0 39.6% 24.1% 4.4 35.8% 25.3% 0.4 45.2% 4.8% 8.6 42.6% 11.4%

dv 5.0 10.0 5.1

We identify two clear lines for future work. First, more decision variables will be incorporated to extend control flexibility, such as dynamic voltage and frequency scaling (DVFS) and battery charge/discharge. Second, a model predictive control (MPC) will be implemented to dynamically regulate the operation of the datacenter. MPC is a well-known control framework which iteratively updates the state of the system and solves the optimization problem as more information arrives. R EFERENCES [1] [2]

[3]

[4]

[5]

[6] [7]

[8]

[9]

[10] [11]

C ONCLUSIONS AND FUTURE WORK

solution

[12]

[13]

J. Koomey, “Growth in Data Center Electricity Use 2005 to 2010,” 2011, Analytic Press. L. A. Barroso and U. Hölzle, “The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines,” Synthesis Lectures on Computer Architecture, vol. 4, no. 1, pp. 1–108, 2009. Í. Goiri, W. Katsak, K. Le, T. D. Nguyen, and R. Bianchini, “Parasol and GreenSwitch: Managing Datacenters Powered by Renewable Energy,” in Proc. of the 18th Int. Conf. on Architectural Support for Programming Languages and Operating Systems, 2013, pp. 51–64. Z. Liu, Y. Chen, C. Bash, A. Wierman, D. Gmach, Z. Wang, M. Marwah, and C. Hyser, “Renewable and Cooling Aware Workload Management for Sustainable Data Centers,” in ACM SIGMETRICS Performance Evaluation Review, vol. 40. ACM, 2012, pp. 175–186. R. Wang, N. Kandasamy, C. Nwankpa, and D. R. Kaeli, “Datacenters as Controllable Load Resources in the Electricity Market,” in IEEE 33rd Int. Conf. on Distributed Computing Systems, 2013, pp. 176–185. World Energy Outlook. Paris, France: International Energy Agency, Organisation for Economic Co-operation and Development, 2006. Í. Goiri, K. Le, M. E. Haque, R. Beauchea, T. D. Nguyen, J. Guitart, J. Torres, and R. Bianchini, “GreenSlot: Scheduling Energy Consumption in Green Datacenters,” in Proc. of Int. Conf. for High Performance Computing, Networking, Storage and Analysis, 2011. Í. Goiri, K. Le, T. D. Nguyen, J. Guitart, J. Torres, and R. Bianchini, “GreenHadoop: Leveraging Green Energy in Data-processing Frameworks,” in Proc. of the 7th European Conf. on Computer Systems, 2012. A. Krioukov, S. Alspaugh, P. Mohan, S. Dawson, D. Culler, and R. Katz, “Design and Evaluation of an Energy Agile Computing Cluster,” U. of Calif., Berkeley, Tech. Rep. UCB/EECS-2012-13, 2012. D. Aikema, R. Simmonds, and H. Zareipour, “Datacenters in the Ancillary Services Market,” in Int. Green Computing Conf., 2012. M. Ghamkhari and H. Mohsenian-Rad, “Data Centers to Offer Ancillary Services,” in 3rd Int. Conf. on Smart Grid Communications, 2012. S. Khan and I. Ahmad, “A Cooperative Game Theoretical Technique for Joint Optimization of Energy Consumption and Response Time in Computational Grids,” IEEE Trans. Parallel Distrib. Syst., vol. 20, pp. 346–360, 2009. Y. Lee and A. Zomaya, “Energy Conscious Scheduling for Distributed Computing Systems under Different Operating Conditions,” IEEE Trans. Parallel Distrib. Syst., vol. 22, pp. 1374–1381, 2011.

3500

3000

3000

2500

800

2500 2000 1500 1000

QoS (deadline violations)

700 QoS (deadline violations)

QoS (deadline violations)


2000 1500 1000

500

500

0 10

0 20

300 200

25 30

250

40

150

200 50

50

45

0

power profile

temperature profile

(a) 75 tasks, power profile A

150 100

40

50

0

200

35

100

100 60

power profile

40

150

50

50 0

power profile

temperature profile

(b) 75 tasks, power profile B

temperature profile

(c) 75 tasks, power profile C

Sample Pareto fronts computed by the proposed MOEA for representative problem instances. 4000

4000

power (W)

power (W)

actual power reference power

3000 2000 1000 0

0

50

100

power (W)

power (W)

cooling power server power

1000

0

50

100

0

50

26 25 actual temperature reference temperature

24

2000 1000

0

50

0

50

100

150

actual temperature reference temperature

0

50

150

(b) Best temperature solution 4000 actual power reference power

3000

power (W)

power (W)

2000 1000 0

50

100


3000 2000 1000 0

150

0

50

100

150

3000 cooling power server power

2000

power (W)

power (W)

100 time (30 second intervals)

3000

1000

0

50

100


2000 1000 0

150

0

50

100

150

100

150

27 temperature (C)

27.5 temperature (C)

150

26 25.5

4000

actual temperature reference temperature

27 26.5 26 25.5

100

26.5

(a) Best power solution

0

150


time (30 second intervals)

0

100

27 temperature (C)

temperature (C)

1000

0

150

27

23

2000

3000

2000

0


3000

0

150

3000

0

50

100

150

26.5 26 actual temperature reference temperature

25.5 25

0

time (30 second intervals)

50 time (30 second intervals)

(c) Best QoS solution

(d) Best trade-off solution (nearest to ideal vector)

The three extremes in the Pareto front and a selected trade-off solution for the power profile A and a 100-task workload.

[14]

J.-K. Kim, H. Siegel, A. Maciejewski, and R. Eigenmann, “Dynamic resource management in energy constrained heterogeneous computing systems using voltage scaling,” IEEE Trans. Parallel Distrib. Syst., vol. 19, pp. 1445–1457, 2008.

[15]

S. Nesmachnow, B. Dorronsoro, J. E. Pecero, and P. Bouvry, “EnergyAware Scheduling on Multicore Heterogeneous Grid Computing Systems,” Journal of Grid Computing, vol. 11, no. 4, pp. 653–680, 2013.

[16]

S. Iturriaga, S. Nesmachnow, B. Dorronsoro, and P. Bouvry, “Energy Efficient Scheduling in Heterogeneous Systems with a Parallel Multiobjective Local Search,” Computing and Informatics Journal, vol. 32, no. 2, pp. 273–294, 2013.

[17]

400

0 20

30

30

Fig. 4.

500

100

20

Fig. 3.

600

B. Dorronsoro, S. Nesmachnow, J. Taheri, A. Zomaya, E.-G. Talbi, and

247

[18] [19] [20] [21]

P. Bouvry, “A Hierarchical Approach for Energy-Efficient Scheduling of Large Workloads in Multicore Distributed Systems,” Sust. Computing, 2014. L. Lennart, “System Identification: Theory for the User,” 1999. T. Bäck, D. Fogel, and Z. Michalewicz, Eds., Handbook of evolutionary computation. Oxford University Press, 1997. K. Deb, Multi-Objective Optimization using Evolutionary Algorithms. J. Wiley & Sons, Chichester, 2001. S. Nesmachnow, “Computación Cient´ıfica de Alto Desempeño en la Facultad de Ingenier´ıa, Universidad de la República,” Revista de la Asociación de Ingenieros del Uruguay, vol. 61, pp. 12–15, 2010.