A Simulated Annealing Algorithm for Energy Efficient Virtual Machine ...

This is the author’s version of a work that was submitted/accepted for publication in the following source: Wu, Yongqiang, Tang, Maolin, & Fraser, Warren L. (2012) A simulated annealing algorithm for energy efficient virtual machine placement. In IEEE International Conference on Systems, Man, Cybernetics, 14-17 October 2012, COEX, Seoul. (In Press) This file was downloaded from: http://eprints.qut.edu.au/53749/

c Copyright 2012 IEEE

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Notice: Changes introduced as a result of publishing processes such as copy-editing and formatting may not be reflected in this document. For a definitive version of this work, please refer to the published source:

A Simulated Annealing Algorithm for Energy Efficient Virtual Machine Placement Yongqiang Wu, Maolin Tang and Warren Fraser Queensland University of Technology 2 George Street, Brisbane, QLD 4001, Australia {yongqiang.wu}@connect.qut.edu.au,{m.tang;w.fraser}@qut.edu.au

Abstract—Improving energy efficiency has become increasingly important in data centers in recent years to reduce the rapidly growing tremendous amounts of electricity consumption. The power dissipation of the physical servers is the root cause of power usage of other systems, such as cooling systems. Many efforts have been made to make data centers more energy efficient. One of them is to minimize the total power consumption of these servers in a data center through virtual machine consolidation, which is implemented by virtual machine placement. The placement problem is often modeled as a bin packing problem. Due to the NP-hard nature of the problem, heuristic solutions such as First Fit and Best Fit algorithms have been often used and have generally good results. However, their performance leaves room for further improvement. In this paper we propose a Simulated Annealing based algorithm, which aims at further improvement from any feasible placement. This is the first published attempt of using SA to solve the VM placement problem to optimize the power consumption. Experimental results show that this SA algorithm can generate better results, saving up to 25 percentage more energy than First Fit Decreasing in an acceptable time frame. Index Terms—simulated annealing; virtual machine placement; energy efficiency;

I. I NTRODUCTION Energy saving in data centers has become an increasingly urgent problem recently due to their consuming tremendous amounts of electricity and the steady increasing trend. The energy dissipation of the physical servers is the root cause of other power usage like that of cooling systems and the uninterruptable power supply. Virtualization technology like VMware [1] and Xen [2], which consolidate multiple logic servers into a smaller number of physical machines, have been used to make better use of hardware and save energy. Instead of having its own dedicated physical host, each logic server runs on the Virtual Machine (VM) hosted on the Physical Machine (PM). Besides the static consolidation, the live VM migration technique has also been applied to further reduce energy consumption by migrating VMs to fewer physical nodes when workloads of the VMs are lighter. In both static and dynamic assignment decisions, the algorithms assigning the VMs to the PMs are the key issue, which may affect the optimized objective significantly by different placement decisions. However, this is a well known NP-hard problem[3]. Therefore, heuristic solutions are usually used to solve this type of problem, as shown in the literature [4] [5] [3] [6].

In this paper, we propose a Simulated Annealing Virtual Machine Placement (SAVMP) algorithm, which is based on Simulated Annealing theory. Simulated Annealing (SA) was proposed by Kirkpatrick et al [7] to be a general approach of solving some NP complete optimization problems in a heuristic way. The theory is based on the insightful analysis of connection between statistical mechanics and multivariate or combinatorial optimization. The effectiveness of SA comes from its extension of two basic heuristic techniques: 1) divide and conquer approach; 2) iterative improvement scheme. It separates large changes and small changes by allowing large changes in the objective function at high temperatures while deferring small changes to low temperatures. It iteratively moves the configuration by small steps from one state to another while preventing from being stuck by allowing large changes at high temperatures. To evaluate the performance of SAVMP, we compare the test results of it with those of First Fit Decreasing and a multi-start random searching approach. Experimental results show that SAVMP has the better performance at a cost of more time than FFD, outperforming FFD by 0-25% in terms of energy savings; SAVMP is also better than the multi-start random searching, which only performs well in small sized problems and is not able to generate better results than FFD when the problem size is larger. The major contributions of this paper are: firstly we propose a SA algorithm to solve the energy-efficient VM placement problem; secondly we design and perform plenties of tests to demonstrate the efficiency of the proposed algorthim in addressing the targeted problem. II. R ELATED W ORK First Fit Decreasing (FFD), Best Fit (BF) for Bin Packing problems are two well known types of the heuristic methods. FFD has low complexity, good scalability and generally good results in practice [6]. However, its drawback is the over simplicity that may overlook the large amount of combinations among which there probably be significantly better results. Furthermore, it sorts the bins and balls only in one dimension which may compromise its performance when the characteristics of other dimensions disrupt the ordering and undermine the fundamental optimization logic. BF has poor performance in providing the global optimization, because it only aims at finding the best placement for the current ‘ball’ to be put into a

‘bin’. This local optimization strategy makes it perform poorly in the global VM assignment solution. However it has been used often in dynamic placement problems when only one or several VMs need to be relocated at a time like in the works [8] [9]. Genetic algorithms are optimization techniques inspired from natural evolution processes [10], and believed to be able to generate a promising result after a certain number of generations, but there have not been very successful attempts in tackling the VM machine placement problem so far to our best knowledge. Many researchers have proposed solutions to increase the energy efficiency of the server clusters based on virtual machine technologies in data centers [4] [5] [3]. Most of them make use of the ordering algorithms to generate the VM placement. For example, ‘pMapper’ uses FFD to generate an initial assignment of VMs and later re-assign VMs picked from PMs which violate resource constraints using Best Fit [4]; in MFR algorithm [11], the prediction techniques are applied to deal with the time varied demand pro-actively and FFD is applied according to the predicted demand. Our work differs from theirs in that they use the FFD as the basic algorithm to come up solutions to meet the resource requirement of the dynamic VM workloads while conserving energy, but we are proposing an algorithm potential to replace FFD or combine with FFD to generate a better VM placement.

(

if vi is assigned to pj 1≤i≤N,1≤j≤M otherwise (2) For a given assignment V P , the CP U utilization of pj can be calculated by

vpij =

1, 0,

PN

i=1

uj =

pjcpu

M X

... .. . ...

 vp1M  ..  . vpN M

ej

(5)

j=1

subject to:

To formulate this problem we are dealing with in the mathematical form, let’s define N number of virtual machines M number of physical machines vi the number i virtual machine pj the number j physical machine vpij the binary value representing whether virtual machine vi is assigned to physical node pj V the set of N virtual machines, namely v1 , v2 , ..., vN P the set of M physical machines, namely p1 , p2 , ..., pM uj the percentage of CPU utilization of pj ej the energy consumption of pj ejmax the energy consumption of pj when uj =100% ejidle the energy consumption of pj when uj =0% i vcpu the CPU demand of vi i vmem the RAM demand of vi i vnet the Network Bandwidth capacity of vi j pcpu the CPU capacity of pj pjmem the RAM capacity of pj pinet the Network Bandwidth capacity of pj V P the assignment of V to P , represented by vp11  ..  . vpN 1

(3)

The energy consumption of pj can be computed by the following equation [9]: ( PN 0, if i=1 vpij = 0 ej = uj (ejmax − ejidle ) ∗ 100 + ejidle , otherwise (4) PN When i=1 vpij = 0, it means that no virtual machine is assigned to pj , so it can be turned off and consumes no energy. The objective of the research is to find an assignment V P that minimizes

III. P ROBLEM F ORMULATION



i vcpu ∗ vpij

∀i,

M X

vpij = 1

(6)

i vcpu ∗ vpij ≤ pjcpu

(7)

i vmem ∗ vpij ≤ pjmem

(8)

j=1

∀j,

N X i=1

∀j,

N X i=1

∀j,

N X

i vnet ∗ vpij ≤ pjnet

(9)

i=1

Constraint 6 means one virtual machine can only be assigned and should be assigned to one physical machine; constraints 7, 8 and 9 requires that the total CPU, memory, or network resources assigned to the VMs of one PM cannot exceed the respective capacities of the host. Here we implicitly assume that the overall resources on the Physical Machines (PMs) are enough to accommodate the guest virtual machines, so we can ensure every guest VM can have a host to run on. If we traversed all the combinations of VMs with PMs, the complexity would be at least N M , so it is not feasible to use an optimal algorithm to tackle the problem as the search space increases exponentially when N or M increases. Therefore, we are going to deal with this problem via a heuristic way.

(1)

where 2

not be stuck in a local optimization. The deivationEngergy is decided by the temperature. We define it as: tc (11) deivationEngergy = max(ejmax ) ∗ t0 where tc is the current temperature and t0 is the initial temperature. Hence at a high temperature, the energy increase allowed by the next state can be relatively large, but when the temperature is low, the energy of accepted states will become stable.

IV. A LGORITHM DESCRIPTION As the authors of SA suggested, there are basically four components in the SA algorithm: 1) configuration of the system; 2) a generator of the new configuration; 3) the objective function for the optimization problem; 4) a schedule of the temperatures and length of times to evolve [7]. In SAVMP, the four components are clearly defined to follow this methodology. A. Configuration of the problem The assignment of VMs V to PMs P is the configuration. Making use of constraint that one VM can only assigned to one PM, and to reduce the number of variables and the searching space, we use an integer array to represent the assignment rather than the matrix as in Equation (1). V M = {vm1 , vm2 , . . . , vmi , . . . , vmn }

D. Temperature Scheduling The temperature scheduling is a very important aspect in the simulation annealing methodology, otherwise the quick quenching can lead to results far from optimal [7]. How many iterations are allowed at each temperature and the range of temperature, for example, are the issues of temperature scheduling. However, the designing of the temperature scheduling is not detailed in the SA methodology, but explained as through an empirical way. We tried several settings and selected the best ones as the parameters of the scheduling as follows. The temperature starts from 1000 degree and reduces gradually to 0 by 5 degrees each time, at which new temperature there are 10, 000 ∗ N times of evolutions.

(10)

The index of the array is the VM number, and the value indexed by the VM number in the array is the PM number to which the VM is assigned. For example, we have 10 VMs indexed from 1 to 10 and 3 PMs numbered from 1 to 3. V M = {1, 2, 1, 3, 2, 1, 3, 2, 1, 2} means that vm1 , vm3 , vm6 and vm9 are assigned to pm1 ; vm5 , vm8 and vm10 are assigned to pm2 ;vm4 and vm7 are assigned to pm3 .

E. SAVMP alogrithm pseudocode and complexity analysis

B. Generator of new configurations

The main steps of Simulated Annealing VM placement algorithm are quite straightforward as described in Algorithm 1 below. As the energy calculation needs C ∗ M times of computation, where C is a const, the complexity is 1, 000 ∗ 10, 000 ∗ N ∗ C ∗ M , which can be expressed as O(N ∗ M ).

The aim of the algorithm is to search better configurations in term of objective function, but the search space is very large. Which way is the economical path toward better solutions? SA searches in the neighborhood states for better results. At every evolution step, the configuration needs to be changed into a new neighborhood state, so the configurations change slowly moving toward better configurations. We define a new neighborhood state as the new VM assignment array acquired by picking a small random number (from 1 to 3 in this implementation) of VMs and changing their host machines to randomly selected PMs or swapping a pair of VM assignment. These two types of rearrangement of VMs to create new configurations reflect the basic operations of doing bin-packing to get a better result.

V. E XPERIMENT AND E VALUATION A. Methodology To test the performance of SAVMP, we design a series of tests with different numbers of VMs to test how the proposed algorithm performs under different problem sizes. For the same numbers of VMs, PM sizes are also varied to test the impact of the container sizes on the performance of SAVMP. In addition to this, we compare the energy cost of the VM placement decided by SAVMP with that by FFD (the upper boundary) and the low boudary, which is calculated with fewer constraints taken into account. Specifically, VM numbers are 20, 50, 100 or 200. To show the size of their containers, the PMs, we introduce the Capacity Index: PM j j=1 pcpu /M (12) CI = PN i i=1 max (vcpu )/N

C. Acceptance criteria of new configurations While generating new configurations provides the candidate configurations, the acceptance criteria decide which configuration becomes the next new state. In the proposed algorithm, if a new configuration is feasible, namely satisfying all constraints specified by Equation (6,7,8,9), and has lower energy consumption than the previous state, it will be accepted as the new state. To be simple, during the energy computation, if a solution is found to be infeasible, the energy value will be assigned a huge number, which will not pass the acceptance criteria. Therefore the energy comparison is the only operation in the acceptance decision. When the neighborhood searching gets into a stalemate, we allow the new state to cost more energy than the previous one by a certain amount, called deivationEngergy in our algorithm, so the algorithm would

i where max (vcpu ) is the maximum allowed CPU allocation for vi . For example, if an internal or external customer asks for a VM with 1000 MIPS ( million instructions per second) i of CPU, the max (vcpu ) of this VM will be 1000 in our problem formulation. When CI is 1, that means the number of PMs in the problem is the same as the number of VMs, and CI = 5 means the PM number is only one fifth of that of VMs, because one PM can host 5 VMs averagely. In the

3

generating the power numbers as priori would not affect the SAVMP’s ability of finding the better results. For each physical server, the energy of idle status is set to 70% of the maximum power, namely,

Algorithm 1: Simulated Annealing Virtual Machine Placement Algorithm Input : VM requirements, PM capacities Output: assignment of VMs to PMs 1 Generated Initial Assignment using FFD 2 T emperature ← 1000 3 while Temperature > 0 do 4 Compute the deviationEnergy allowed by the current temperature 5 for i ← 1 to 10000 ∗ N do 6 Create a random neighbor placement 7 Calculate the incremental energy compared to previous placement 8 if incremental energy < 0 then 9 Set the placement as the new state 10 Save the placement if it costs the least energy so far 11 else 12 if evolution in stalemate and incremental energy < deviationEnergy then 13 Set the placement as the new state 14 else 15 16 17 18 19

ejidle = ejmax ∗ 0.7

As to the initial placement, our proposed SAVMP does not require a special state. In the following tests, we choose to start from the placement results generated by FFD, although SAVMP can achieve a comparable result from a random feasible placement at the cost of more time according to our preliminary tests. For every configuration of VMs and PMs, we run 10 times and average the percentage of the energy saving compared to FFD and the time to reach the best placement among all the evolutions in order to have a statistical view of performance of the SAVMP because it does not generate the exactly same result every time as FFD does for the same problem. All the tests are run on a laptop with a 2.2G Hz dual core processor. As the best possible solution is not available to be compared with, we calculate the low boundary of energy consumption to see how far the SAVMP result is from the low boundary (LB). The LB is calculated by filling the most energy efficient PMs with CPU requirement of VMs, and otherwise finding a PM with the lowest energy cost to accommodate the left CPU requirement. This energy value is the low boundary because it ignores the constraints 7,8,9 and arrange the requirement in the most energy efficient PMs. If an algorithm find a solution with the energy value very close to low boundary, we can say it is a very good result. However, it cannot be concluded that an algorithm performs badly when its energy cost is far from the LB, because the constraints may be the reason for the poor energy efficiency. In such case we need other ways to analyze the placement results. For example the table listing the resource utilization for all PMs may be very useful to check whether the VMs are assigned in a energy efficient way.

end end end Decrease T emperature end

following experiment, for every VM size, CI values iterate in the set [1, 2, 3, 4, 5, 10] to form a new VM placement problem. We consider CPU and Memory requirements to simulate the two major resource constraints in data centers when relocating VMs. The VM CPU requirement is generated randomly from the value 0 to 2000. This is to simulate the resource usage of the VMs at the data center at a sampling time. The PM sizes vary from 1000 to 3000 multiplied by the CI (capacity index), simulating the heterogeneous host environment. The requirement of memory is generated in the same way as CPU, but the two requirements are most probably not the same value for the same VM because of the random nature. In this simulation test setup, each PM’s maximum energy ej is generated by the equation: ej = (1 − log(pjcpu /1000) ∗ 0.4) ∗ E ∗ (pjcpu /1000)

(14)

B. Results of two-dimension packing Table 1 lists the testing results of FFD, SAVMP and the low boundary energy on 24 problems with different VM and PM numbers. Each problem has one row in the table. In the columns, ‘VM’, ‘PM’ show how many VMs and PMs are in the problems respectively; CI is the capacity index, which is defined in Equation(12); ‘Energy by FFD’ is the energy cost if the VMs’ assignment to PMs is decided by the FFD algorithm; ‘Energy by SA’, in which SA is the short format for SAVPM for saving space, is the energy cost of the placement generated by SAVPM; ‘Energy Saved’ is calculated by (1 − Energy by SA/Energy by F F D) ∗ 100%; ‘SA from LB’ is how many more percent of energy used by SAVMP than the low boundary;‘Time’ is the time cost for SAVPM to find the ‘best’ solution. ‘Energy saved by Random Search’ column is the energy saving effects achieved by a Random Search (RS) method. In this method, a PM is picked randomly for each VM. If the chosen PM does not satisfy the constraints, it will try another randomly chosen PM until it finds a suitable one

(13)

In above equation, E is a const, set to 100 (Watt) in our test, representing the base energy which is consumed by the smallest PM when pjcpu = 1000. When the CPU capability is 10 times of the smallest PM, the energy cost is only 6 times as much. This is to simulate a data center environment where the physical servers have been procured through the years and have increasing computing power and more energy efficient with time [12]. Surely the PM’s maximum energy can be obtainted by more accurate power calculation tools provided by hardware vendors or by measuring tools, but this way of 4

or gives up searching and returns no feasible solution after the maximum attempts have been used. The same process repeats until all the VMs have their assigned PMs. To be fair, the number of iterations of RS is the same as SAVMP. The Random Searching’s performance is close to SAVMP when there are only 20 VMs. However, when the VM number increases, the energy saving improvement becomes less or even zero. This is especially obvious when the number is greater than 50. When the VM number is 100 or 200, the Random Search can only reduce the energy cost by a small percentage to FFD when CI is 5 or 10. In all tests but the one of which the VM number is 20 and CI is 5, the Random Search performs worse than SAVMP. The random searching results show that the SAVMP performs better than FFD not only because it involves random searching method to explore more assignment combinations than FFD, but also because the Simulation Annealing approach includes a evolutionary search more than random attempts. The distances of SAVMP from the Low Boundary are averagely within 10 percent, which says the simulated annealing algorithm has achieved very good energy saving results. The improvement effect by SAVMP for two-constraint problem is more significant than that for only CPU constraint. When the VM number becomes larger, the improvement tends to be less because the searching space grows bigger. The relationship of the improvement and CI is not straightforward. It has two contradictory effects on the improvement by SAVMP. For one thing, the larger the CI is, the smaller the searching space becomes. The smaller searching space favors SAVMP but FFD also performs better than when CI is smaller. From the table, we can find another benefit of using SAVMP in the test of which the VM number is 20 and CI is 10. FFD or Random Search cannot find a feasible solution for this test, but SAVMP can find the feasible assignment successfully. It is probably impossible to determine the minimum energy usage among all the possible assignment, but we can use the resource utilization on the PMs resulted from the VM assignment besides the low boundary to examine and compare the energy saving effect of different algorithms. In Table 2, the utilization of CPU and memory has been listed for each PM for the test of 200 VMs with CI set to 10 applying FFD and SAVMP respectively. It is clear from the table that SAVMP achieve higher resource utilization with 2 less PMs turned on than FFD, leaving very small room for to be improved if there is any.

algorithm would be more suitable as proposed in related literature.[13][4] When the problem size grows larger, the time needed to find the ‘best’ solution can be very long. One option is to stop searching within reasonable time limit and used the suboptimum results which can be very close to the ‘best’ result. On the other hand, to reduce the long computation time, we can adopt the ’devide and conqure’ strategy by decomposing the large problem into smaller sub problems and merging results back to the original problem to improve the computing efficiency like the method used by Jian [14]. We have proposed SAVMP - a new VM placement algorithm based on Simulated Annealing theory to improve energy efficiency in data centers, and have evaluated it by experiments. This is the first published attempt of using SA based algorithms to solve the VM placement problem for optimizing energy efficiency. FFD and Random Searching were selected for comparation in evaluating the performance of the SA algorithm. Experiments with different problem configurations have been done. In the experiments, the number of VMs varied from 20 to 200, the sizes of the PMs were from one to 10 times of maximum possible requirement of VMs, and resource constraints were CPU combined with memory two dimensions. Experimental results have shown that SAVMP can generate better VM assignment than FFD, with 0-25% more energy saving than FFD. How much improvement it can make from the FFD depends largely on the room FFD leaves for improvement and the searching time allowed. SAVMP is able to generate VM placement that costs the energy very close to the low boundary, averagely within 10% from the low boundary in the tests. ACKNOWLEDGMENT This research is partially funded by Division of Technology, Information and Learning Support, Queensland University of Technology. During the research process, Don Caruana, Adrian Yarrow and Mohammad Hassan generously shared their expertise on data center management and energy consumption containment. R EFERENCES [1] VMware. [Online]. Available: http://www.vmware.com [2] xen.org. [Online]. Available: http://www.xen.org [3] M. Stillwell, D. Schanzenbach, F. Vivien, and H. Casanova, “Resource allocation algorithms for virtualized service hosting platforms,” Journal of Parallel and Distributed Computing, vol. 70, no. 9, pp. 962–974, 2010. [4] A. Verma, P. Ahuja, and A. Neogi, pMapper: Power and Migration Cost Aware Application Placement in Virtualized Systems Middleware 2008, ser. Lecture Notes in Computer Science. Springer Berlin / Heidelberg, 2008, vol. 5346, pp. 243–264. [5] A. Verma, G. Dasgupta, T. K. Nayak, P. De, and R. Kothari, “Server workload analysis for power minimization using consolidation,” pp. 28– 28, 2009. [6] A. C.-C. Yao, “New algorithms for bin packing,” J. ACM, vol. 27, no. 2, pp. 207–227, 1980. [7] S. Kirkpatrick, J. C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science, vol. 220, p. 10, 1983.

VI. D ISCUSSION AND C ONCLUSION The proposed SAVMP provides another way of solving the VM placement problem in addtion to the ordering algorithm and constraint programming. As demonstrated above, SAVMP can be very suitable in solving the static VM placement problem. However, similar to the limitation of FFD, SAVMP might not be applied in a simple way periodically to do the reshuffling of the VMs due to reasons like the migraiton cost when being used in the dynamic VM consolidation. Instead an incremental way or a two-stage way of applying the placement 5

TABLE I S IMULATION TEST RESULTS OF SAVMP ON DIFFERENT NUMBERS OF VM S AND VARIED C APACITY I NDEXES WITH CPU AND M EMORY CONSTRAINTS

VM (#) 20 20 20 20 20 20 50 50 50 50 50 50 100 100 100 100 100 100 200 200 200 200 200 200 a

PM (#) 20 10 6 5 4 2 50 25 16 12 10 5 100 50 33 25 20 10 200 100 66 50 40 20

CI 1 2 3 4 5 10 1 2 3 4 5 10 1 2 3 4 5 10 1 2 3 4 5 10

Energy by FFD (watt) 2090.7 1682.7 1688.9 1800.3 1627.7 N/Aa 4834.7 4256.4 3856.6 3365.2 3304.5 3057.3 9404.8 8159.5 7494.9 6835.5 6704.9 5247.0 18373.2 15944.7 14606.7 13442.6 12375.4 9675.6

Energy by SA (watt) 1774.1 1445.5 1429.2 1341.6 1339.3 1219.0 4310.9 3616.8 3298.8 3006.5 2840.8 2578.2 8871.7 7448.1 6813.5 6116.3 5777.0 4513.0 17379.2 14514.2 12999.5 12253.2 11446.7 8941.3

Time (sec) 3.20 1.54 1.24 0.52 1.32 0.94 28.92 7.45 1.73 0.72 0.64 1.09 131.53 44.51 25.86 7.78 2.04 1.85 1250.81 208.13 215.01 22.45 17.41 23.50

Energy Saved by SA (%) 15.79 14.13 15.37 25.48 16.84 N/Aa 10.19 15.15 14.47 10.67 14.04 15.68 5.41 8.78 9.08 10.53 13.85 14.00 5.46 8.29 10.14 8.85 7.50 7.59

Energy Saved by RS (%) 9.23 8.35 13.91 24.45 17.40 N/Aa 0.17 5.81 7.02 3.82 11.43 15.65 0 0 0.57 0 6.37 7.41 0 0 0 0 1.66 5.61

LB 1422.7 1278.4 1423.5 1196.7 1296.1 1102.0 4062.3 3470.1 3119.6 2710.9 2487.7 2275.4 8461.6 7296.2 6726.5 5774.6 5405.2 3903.6 16446.3 14120.1 12824.6 11538.8 10558.2 8462.3

SA from LB (%) 19.80 11.56 0.40 10.80 3.23 9.60 5.77 4.06 5.43 9.83 12.43 11.75 4.62 2.04 1.28 5.59 6.44 13.50 5.37 2.72 1.35 5.83 7.76 5.36

For VM number=2 and CI=10 with CPU and memory constraints, FFD cannot provide a feasible solution, so the “Energy Saved” value is not applicable. TABLE II PM RESOURCE UTILIZATION C OMPARISON BETWEEN PLACEMENT BY FFD AND SAVMP ON 200 VM S WITH CAPACITY INDEX = 10

PM Index 1 2 3 4 5 6 7 8 9 10 11 .. . 20

Size 30000 30000 27000 27000 25000 25000 24000 24000 23000 23000 20000 .. . 10000

CPU Utilization (%) 99.83 99.83 99.81 99.81 99.80 99.80 80.00 51.25 18.04 0.43 0.00 0.00 0.00

FFD Memory Utilization (%) 54.00 58.67 75.93 57.04 83.20 94.00 99.58 99.58 99.57 13.91 0.00 0.00 0.00

[8] R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. F. D. Rose, and R. Buyya, “Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms,” Softw. Pract. Exper., vol. 41, no. 1, pp. 23–50, 2011. [9] A. Beloglazov and R. Buyya, “Energy efficient resource management in virtualized cloud data centers,” in Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, ser. CCGRID ’10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 826–831. [Online]. Available: http://dx.doi.org/10.1109/CCGRID.2010.46 [10] D. K. Alain Hertz, “A framework for the description of evolutionary algorithms,” European Journal Of Operational Research, vol. 126, no. 1, pp. 1–12, 2000. [11] N. Bobroff, A. Kochut, and K. Beaty, “Dynamic placement of virtual machines for managing sla violations,” in Integrated Network Management, 2007. IM ’07. 10th IFIP/IEEE International Symposium on, pp. 119–128.

SAVMP CPU Utilization (%) Memory 100.00 100.00 100.00 100.00 99.80 99.80 74.79 73.33 0.00 0.00 0.00 0.00 0.00

Utilization (%) 79.00 96.67 82.96 78.15 99.20 98.40 85.00 91.25 0.00 0.00 0.00 0.00 0.00

[12] M. Poess and R. O. Nambiar, “Energy cost, the key challenge of today’s data centers: a power consumption analysis of tpc-c results,” Proc. VLDB Endow., vol. 1, no. 2, pp. 1229–1240, 2008. [13] F. Hermenier, X. Lorca, J.-M. Menaud, G. Muller, and J. Lawall, “Entropy: a consolidation manager for clusters,” in Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, ser. VEE ’09. New York, NY, USA: ACM, 2009, pp. 41–50. [Online]. Available: http://doi.acm.org/10.1145/1508293.1508300 [14] J. xiong Dong, A. Krzyzak, and C. Suen, “Fast svm training algorithm with decomposition on very large data sets,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, no. 4, pp. 603 –618, april 2005.

6