2016 5th International Workshop on Green and Sustainable Software
Adaptive Virtual Machine Migration Mechanism for Energy Efficiency A. Aleti
[email protected]
S. Sohrabi, A. Tang, I. Moser {ssohrabi}, {atang}, {imoser} @swin.edu.au
Monash University Melbourne, Australia
Swinburne University of Technology Melbourne, Australia
ABSTRACT
2013, in the United States alone, data centers were responsible for using almost 91 billion kilowatt-hours of electricity [2] and it is estimated to be 140 billion kilowatt-hours by 2020 [2]. The goal of avoiding a large carbon footprint has spurned new research into energy-aware Cloud management systems. Managing the workload in the Cloud is the task of a hypervisor or virtual machine monitor. This Cloud management software is responsible, among other things, for allocating tasks to virtual machines (VMs) and VMs to host servers. Allocating workloads to suitable hardware avoids overloading, which leads to increased energy consumption. Optimizing the scheduling of tasks is a research topic that has been addressed frequently from an energy saving perspective [3], [4], [5]. Static policies as well as learning heuristics have been proposed for scheduling optimization. Energy-aware schedulers aim to minimize the energy usage by relying on the information available at the time of scheduling. In many cases, the resource requirements of a task are not known beforehand. If information is available, it may not be precise, and load imbalances may still arise. Most load balancing approaches proposed in the literature address the problem of scheduling alone and do not reschedule tasks when hosts become overloaded [3]. Some approaches provide scheduling as well as re-scheduling [4]. In this work, we propose a Bayesian Belief Network (BBN) that learns which VMs should be shifted when a host becomes overloaded. At the time of rescheduling, more information is available regarding the resource requirements of a VM than at the time the VM is initially deployed. Having this information about the VMs to reschedule helps make better rescheduling decisions. The proposed Bayesian based heuristic distinguishes between VMs by their CPU usage, and the conditional probabilities of migrating a VM with a certain CPU usage are maintained based on feedback drawn from the difference in processing power the VM has received before and after migration. The approach is evaluated experimentally by benchmarking against two competitive state-of-the-art heuristics. This paper is organized as follows. First, a review of related literature is provided in section 2. Then, the motivation of this study is explained. In section 5 the process of VM migration is defined and is later used in section 6 to describe the proposed VM migration heuristic. The evaluation of the proposed heuristic and the comparison with the existing policies are in section 7. Conclusions and future research directions are in section 8.
Cloud systems have become a popular platform for business applications due to the flexibility in usage and payment they offer. One of the caveats of Cloud systems is their high energy consumption. Minimizing energy consumption while maintaining a high service level has become a relevant optimization task for Cloud providers. Opportunities for energy savings arise when server hosts are overloaded, which also entails unnecessary delays. To address the problem, researchers have devised strategies how to choose the server host to deploy an application to and how to choose a running application for migration when a host has been identified as overloaded. In this work, we introduce a Bayesian Belief Network which learns over time which of the virtual machines are best removed from a host that has been identified as overloaded. The probabilistic choice is made among virtual machines that are grouped by their degree of CPU usage. Given the feedback in the form of the computing resources saved, the system learns which virtual machine profiles should be shifted for best performance. This strategy compares favourably to two existing methods for load balancing.
Keywords Cloud computing; VM migration; Bayesian Network
1.
INTRODUCTION
Cloud systems are convenient and flexible platforms for software applications, but they consume considerable amounts of energy. The cost of energy has always been a concern for Cloud providers, but more recently, the environmental consequences of non-renewable energy production have become more of a public concern. The proportion of energy consumed by data centers was estimated to be as high as 1.1 - 1.5% already in 2011[1]. Since then, many more Clouds have been established. In Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected].
Greens2016 2016, Austin, Texas, USA c 2016 ACM. ISBN 978-1-4503-4161-5/16/05. . . $15.00
DOI: http://dx.doi.org/10.1145/2896967.2896969
8
2.
RELATED WORK
oritised applications with short deadlines. In another ACO approach, Lu and Gu [16] define a hotspot as a host whose CPU, RAM or bandwidth have been exceeded. The algorithms learns what degree of host capacity is ideal for a VM that needs shifting. Tang et al. [17] used a peer-to-peer infrastructure to poll many hosts at the same time for available resources while estimating the resource requirements of tasks based on previous usages of the same task. Since the approach specializes in enterprise data centers, the managing software continuously works through the loaded servers, switching off redundant applications or shifting them when necessary. Kusic et al. [18] developed an algorithm that manages applications in small server clusters to ensure SLA levels Gold and Silver while minimizing energy consumption. The algorithm projects the expected demand to avoid later switches and dynamically tunes the number of VMs to allocate to each application as well as the hosts to allocate to the VM. Existing scheduling and re-scheduling heuristics either rely on the existence of information about tasks’ resource requirements or suggest a definite set of rules to apply in order to minimize energy consumption. However, the tasks’ resource requirements are not always known to the hypervisor. Moreover, setting rules without the flexibility of learning might fail in responding to changes in either the system state or the tasks’ requirements. An adaptive re-scheduling mechanism that pursues energy minimization without relying on prior knowledge about tasks’ resource requirements is needed to address the energy efficiency problem in a dynamic way. Such mechanism has learning ability and responds to changes adaptively.
Since the inception of Cloud technologies researchers have devised scheduling algorithms for load balancing. Sometimes a workload is sought that is spread evenly on the available hardware, sometimes it is the virtualization that is preferred with even loads. In some studies, the balanced load itself is the goal of scheduling optimization, in others the CO2 emissions, processing power, fan usage, the makespan, the monetary cost and similar variables are minimized, or service levels are imposed. In most studies, the optimization task comprises only the initial assignment of a VM to a hardware host. Fewer research efforts have been directed at the problem of rescheduling VMs when they are running on an overused host. Among the scheduling policies, Tsafrir, Etsion and Feitelson [6] proposed an first-come-first-serve algorithm with backfilling that uses a combination of kill times and run time estimates – shorter tasks are scheduled before their time if they are estimated not to interfere with the deadlines of other tasks. The errors that necessarily exist in the run time estimates inspired separate kill times, which were found effective. Suresh and Vijayakarthik [7] improved a backfilling algorithm with a technique that places tasks with similar lengths next to each other to ensure they finish at roughly the same time and free up ’adjacent’ computation power that can be used by larger tasks. Traditionally, scheduling policies such as these optimize the make-span. However, Garg et al. [5] proposed a number of policies to find ideal task placements among heterogeneous hosts with diverse workloads in order to minimize energy consumption. Dynamic voltage scaling was devised as a means of adjusting the voltage of hosts depending on the load [8] to save energy. Another energy-aware approach, devised by Lee et al. [9], consolidates VMs onto servers to switch off others that were freed off in the process. Unlike policies, heuristics attempt to learn certain properties of a solution to be able to produce better solutions in the future. Ant Colony Optimization (ACO) has been used for scheduling in Cloud systems a number of times. Zhu, Li and He [10] devised an algorithm where tasks have quality of service parameters related to bandwidth, completion time, reliability and cost. These characteristics determine which of the heterogeneous resources to choose for placement. Over time, the system learns the near-optimal placements. Similarly, Feller, Rilling and Morin [11] adapted ACO to a workload consolidation problem. The system accommodates the same tasks over time and is able to record the highest resource requirements of each task, making them easier to deploy once the system has learned the near-optimal assignment. ACO was also used by Gao et al. [12] to minimise both resource wastage and energy consumption. Genetic Algorithms (GA) have also been applied to multi-objective formulations. Xu and Fortes [13] optimized resource wastage, power consumption and thermal dissipation costs in this way. Among the approaches that tackle the problem of rescheduling, Mi et al. [14] applied GA to adjust the data center to the dynamically changing demands with the goal of minimizing energy consumption. Resource requirement prediction is used to avoid delays caused by migration. Chen and Wang. [15] developed a rescheduling algorithm to maximize the profit of the Cloud provider by giving preference to pri-
3.
MOTIVATION
Beloglazov and Buyya propose energy-aware VM selection policies and report Minimum Migration Time (MMT) as the most energy optimal [4]. MMT selects a VM that can be transferred through the network in the shortest time. The migration time depends on the memory size of the VM and the available bandwidth. In a case where a host with low bandwidth is imbalanced, VMs from such host will be the last to be chosen by MMT. The multi-objective migration heuristic proposed by Sallam and Lee [19] aims to find the energy efficient way of migrating VMs. The objectives are load volume, energy consumption, thermal status, resource wastage and the cost of migration. All objectives are set to be minimized. Although taking loads off an overloaded host solves the imbalance problem, it keeps more hosts in their active status that needed if there is no imbalanced host. The above mentioned approaches suggest a static rule for resolving the imbalance in the system. However, such approaches may not perform well when their rule is not aligned with the resource requests and they do not present learning ability. In the Cloud, with volatile resource status, an adaptive mechanism is expected to i) find the best VM to migrate in terms of energy and ii) have the ability to learn and respond to changes in the resource state. One of the approaches for development of an adaptive migration mechanism can be the application of probabilistic methods. Probabilistic methods base their decision on the likelihood of an event according to the observed values. That is, a certain input is linked to an expected output. The adaptability comes from its ability to update its probabil9
5.1
ities according to the observed values. In a study by Hu et al [20], a probability based scheduling policy, Probability Dependent Priority (PDP), is proposed. They argued that PDP showed significant improvement in performance compared to First Come First Serve (FCFS) and Head-Ofthe-Line (HOL) priority. HOL gives the tasks from ‘a more demanding class‘ a higher priority. They also demonstrate that PDP requires fewer hosts to meet a certain level of SLA. One of the widely used probabilistic approaches is Bayesian Network. Its application includes failure management [21], mobile Cloud computing [22] and scheduling [23]. Bayesian has its potential in statistically estimating the likelihood of selecting a VM for migration to result in a better outcome in terms of energy. Moreover, Bayesian updates its probabilities according to the observed results. Such methodology has the potential to enable the VM migration mechanism to learn and adapt to the changes as they emerge. It gives the hyper-visioning of the Cloud the versatility and adaptivity it requires.
4.
Let M V M = {mvm1 , mvm2 , · · · , mvmn }, where n ∈ N, denote the set of migratable VMs. Each VM is running a list of tasks, Ti . Ti = {ti1 , ti2 , · · · , tip }, for V Mi running p tasks. Each migratable VM has the followings: - Running task, RT: T → N - Processing share (MIPS), PS: M V M → R - Memory, M: M V M → R Also, let H = {h1 , h2 , · · · , hm }, m ∈ N represent the list of candidate destination hosts, each host has the following attributes when hi has a list of VMs running on it as V Mi = {vm1 , vm2 , ...vmq }. - Running VMs, RVM: H → V M → N - Processing units, PU: H → R - Memory capacity, MC: H → R - Network bandwidth, NB: H × H → R
5.2
ENERGY CONSUMPTION
t (f /fmax )
Z
C × fmax × f 2 × t
5.3
VM migration cost
A time and energy penalties are associated with each VM migration. The time penalty comes from the time it takes to migrate the VM, which is the time required to copy itsmemory and CPU status. VM migration time is calculated by dividing VM’s memory capacity by available bandwidth between source and destination hosts [4]. The energy penalty comes from the act of copying the VM memory from source to destination hosts through network links. That is, a VM migration helps easing the imbalance problem but adds to the time and energy. Therefore, an efficient heuristic is expected to perform fewer number of migration and minimizes the energy usage and execution time. Migration time is modeled by Beloglazov and Buyya [4] and energy consumed for a VM migration is modeled by De Maio and Prodan [34].
(1)
5.4
0
Eq. 1 shows that the frequency of the processor f and the length of time t determine the energy consumption E, while fmax is the maximum frequency and C is a constant. The frequency at which the processor runs and the duration depends on the resource requirements of the tasks running on the VMs.
5.
Migrating a VM to another host
VM migration can be defined as M ig = {mig|mig : M V M → H}, where M ig is a set of combinations of mvmi and hj . Because M V M and H are finite sets, M ig is also finite. A migration map can be M igi = {(mvm1 , h1i ), (mvm2 , h2i ), · · · , (mvmn , hni )}, hxi might appear more than once, if is a suitable destination host. Each migration to hxi is a candidate VM migration.
Among all contributors to energy consumption, research has shown [24] the CPU to be the main contributor. Around 90% of the energy expended is used by the CPU, with memory and network energy usage accounting for the remainder. CPU utilization is widely used to relate CPU utilization to energy consumption [25, 3, 26]. This is also confirmed by Deelman et al [27] where they determine the cost of running a scientific workload and report the computational cost (CPU utilization) to outweigh the storage cost. Energy usage consists of static and dynamic energy consumption. Static energy usage is determined by the hardware. However, among existing formula for calculating the dynamic energy consumption we follow popular formulation by Kim, Beloglazov and Buyya [3] (Eq. 1). This formulation has been adopted by CloudSim [28, 29] and therefore been used to measure the total energy consumption in similar studies [?, 4, 19, 31]. where E is the energy consumption, t is the time, f is the frequency of the processor and ; C is a constant. Ed =
VM migration description
VM migration constraints
Let’s C = {cmigratable , cexcludedHosts , cmaxV M s } represent the constraints. cmigratable means not all VMs are migratable. The migration process only chooses VMs that can be migrated. cexcludedHosts excludes hosts that are not suitable for receiving the migrating VM. It is because they are already overloaded and should not be given another VM. According to the VM monitoring software/hyper-visor used, there is a limitation on the maximum number of VMs that can be assigned to a host that is cmaxV M s .
VM MIGRATION
VM migration is referred to the process of moving a VM from a host to another. If VM migration is achieved without stopping the execution in source and destination hosts (apart from a short time for transferring the VM status), it is called live migration [32]. The reasons for migrating VMs include its potential in taking the extra burden off an overloaded host. An overloaded host is more likely to encounter hardware failure [33]. In case of an underloaded host, VM migration provides the chance to migrate the VMs from the underloaded host and switch it off to save energy.
6.
BAYESIAN HEURISTIC FOR VM MIGRATION
The Bayesian heuristic for VM migration (BHVM) uses a Bayesian Network (BN) [35], which is a probabilistic technique for reasoning under uncertainty or lack of information. BN estimates the impact of an action on the outcome. In this study BN estimates the impact a VM migration has on execution time and energy usage. 10
p(R1 )
R1
p(R2 )
R2
R3
p(R3 )
p(R4 )
Start
R4
Update probabilities
∆P U
∆P U
∆P U
∆P U
p(∆P U > 0|R1 )
p(∆P U > 0|R2 )
p(∆P U > 0|R3 )
p(∆P U > 0|R4 )
Imbalance detected
6.2
n X
P Si +
i=1
m X
P Sj
(2)
(3)
j=1
P UP re is the processing units provided to the VMs on source and destination hosts before the migration. Similarly, the total processing units after migration is equal to equation 4 P UP ost =
n−1 X i=1
P Si +
m+1 X
7. P Sj
Heuristic description
EXPERIMENTS AND EVALUATION
In this section, the experiment setup and the results of the comparisons are explained.
(4)
j=1
The difference between the computational resources provided for VMs before and after the migration is ∆P U = P UP ost − P UP re
Record feedback value
The proposed Bayesian, probabilistic mechanism follows figure 2. We term this Bayesian Migration Heuristic (BMH). M V M is the input of migratable VMs. Based on the Bayesian probabilities for each state of the variable, VMs of a state with highest probability are candidate VMs for migrations. mvmi is then selected from that subset and migrated. Then the feedback value is recorded. The system should be monitored to find any imbalance. A host is imbalanced if it is overloaded or underloaded. An overloaded host received resource requests that exceed its capacity. VMs from an underloaded host can be migrated to other hosts without making them overloaded. If an imbalance is detected, the Bayesian probabilities will be updated and the new input of M V M will be presented to the migration heuristic. To calculate the Bayesian probabilities and select VMs P (Ri |+ ), P (+ ) and P (Ri ) should be calculated. P (+ ) and P (Ri ) are prior probabilities. P (+ ) is the number of instances of + per total number of migration instances. P (Ri ) is the number of migration instances where the migrated VM was from Ri divided by the total number of instances. P (Ri |+ ) is then calculated according to algorithm 1. After calculating Bayesian probabilities, M V M will have its members {mvmi , mvm2 , , . . . , mvmn } assigned to a range in R. Then a mvm from a range with the highest probability is chosen for migration.
where p(x2 ) is the prior probability for x2 , modeled as p(Ri ) in this work, and p(x2 ∩ x1 ) is the joint probability of the two variables, p(∆P U > 0 ∩ Ri ). We have modeled the impact on energy consumption as the change in the computational resources (in our case CPU utilization) when a VM is migrated. This is modeled as the node labeled ∆P U in Figure 1. Let us refer to the two hosts that the VM is migrated to and from as destination and source hosts respectively. If we assume that the source host is already running n VMs and the destination host is running m VMs, the total processing units provided to the VMs is equal to equation 3.
P UP re =
Monitoring for imbalance
measured in MIPS (Million Instructions Per Second). ∆P U values greater than zero, indicates that the VM migration has provided VMs with more processing units. If represents the outcome for Bayesian, ∆P U greater than zero is termed + . The conditional probability p(+ | Ri ) estimates the probability of + given a VM with utilization in Ri range is migrated. This has its effect on the frequency of the processor and VMs’ execution time which are the contributing factors in energy usage according to equation 1.
In figure 1 the nodes labeled as R1 , R2 , ...Rk represent the ranges in which the virtual machines’ utilization can be. In this study, the ranges are defined according to the quartiles of the observed utilization value, R = {R1 , R2 , R3 , R4 }, where R1 covers minimum to first quartile, R2 first quartile to second quartile, R3 second quartile to third quartile and R4 is third quartile to maximum. The probabilities associated with each node are the likelihood that a VM from the range Ri is migrated p(Ri ). Directed arcs connecting pairs of nodes represent dependencies between variables. The strength of the dependencies between variables is quantified by conditional probabilities associated with each node. The probability of x1 conditional on x2 is calculated as p(x2 ∩ x1 ) , p(x2 )
Migrate mvmi
Figure 2: Bayesian VM migration Heuristic Flowchart
Bayesian Network
p(x1 | x2 ) =
Bayes probabilities
yes
Figure 1: Bayesian Network representing the ranges as nodes labeled Ri , the probabilities that a virtual machine from a range is migrated p(Ri ), and the change in processing units ∆P U resulting from VM migrations modeled as conditional probabilities.
6.1
MVM
7.1
Experiment setup
To evaluate the proposed Heuristics, CloudSim simulation package [28, 29] is used. The workloads for evaluation are the ones for CoMon project, a monitoring infrastructure for
(5) 11
Data: Obtained results from migrating VMs Result: Conditional probabilities for each instance of VM migration do if VM belongs to Ri then increment counteri ; if this instance has a positive value then increment counteri,+ ; end end end counter
The energy efficiency of our mechanism relates to its short execution time. According to equation 1, minimization of execution time relates to the minimization of energy consumption. Another reason is the effect of the VM migration cost on execution time and energy consumption. Figure 3d illustrates the number of migrations that had to be carried out to ease the imbalance throughout the execution of tasks. BMH had to perform minimum number of migrations, meaning the decision in which VM to migrate has been more effective in solving the imbalance. As explained in section 5.3 fewer migrations translates into less time and energy penalties due to the migration which is another reason for BMH to result in minimization of energy and execution time. To have a complete overview of the effect of heuristics, SLA violation percentage is measured and presented in figure 3c. SLA is violated if the total request for the resource exceeds the resource capacity. SLA violation is the percentage of time that hosts have encounter requests more than what they could provide for VMs. SLA violation is slightly increased by BMH, 0.0005% mean difference with MMT and MOH. The increase of SLA violation is insignificant. The increase in SLA violation is associated with the number of VM migration that are reported in figure 3d. Although BMH performs fewer migrations, it takes longer to choose a VM for migration. The slight increase in the time needed for selection has led to slight increase in the SLA violation yet improved the energy consumption and execution time significantly. BMH showed to minimize energy consumption in the Cloud. It has its strength in the probabilistic aspect of it when it adaptively calculates the likelihood of a type of VM leading to the optimization of energy usage and execution time.
+
i, P (Ri |+ ) = counter i Algorithm 1: Conditional probabilities
PlanetLab [36] that are included in the CloudSim and were previously used as a bench mark in the literature [4, 37]. Ten days are randomly selected in March and April 2011. Therefore the simulation time is over the selected 24 hours time. We base our evaluation on the same chosen dates to enable the reproduction of the results. There are 800 hosts (heterogeneous, 400 HP ProLiant ML110 G4 and 400 HP ProLiant ML110 G5 ) and the maximum number of VMs on a host is limited to 75 [19], CmaxV M s = 75, as it runs Xen [38] hyper-visioning software. The experiments were conducted on an Intel Core i5-2400 CPU machine with 3.10 GHz, 8MB of memory running Windows 7 Enterprise. All heuristics are run when threshold based VM allocation [4] is used and the parameter is .6, .7, .8, .9 or 1. BMH starts with equal probabilities assigned to each variable state, it then re-calculates (updates) its probabilities after every 300 migrations.
7.2
Results
8.
Figure 3 shows the results for BMH compared to two heuristics, MOH and MMT. Multi-Objective Heuristic (MOH) is a multi-objective heuristic that uses the five objectives of the approach suggested by Sallam and Lee [19]. It has load volume, energy consumption, thermal status, resource wastage and migration cost as its objectives. In their heuristic a Static Bayesian Game - Multi-objective Genetic Algorithm is used where each objective is considered to be a player that optimizes its payoff function. Their evaluation is on a scale where 60000 GB of migration is carried out to have the first generation in the GA. However, in the actual workload of the selected benchmark the number of migratable VMs are smaller and the game and GA sections are not applicable. BMH is also compared to the best VM migration selection policy reported by Beloglazov and Buyya [4], Minimum Migration Time (MMT). They calculate the migration time according to the VM’s memory size, M , and the available bandwidth between source and destination hosts, N B. Figure 3b shows total energy consumption by each heuristic. BMH outperforms other policies. The mean execution time of VMs is reported in figure 3a. It shows the time (seconds) it took for each heuristic to run the workloads. The mean execution time is inclusive of the VM selection (heuristic’s) time. This is to consider any potential increase in the complexity of the VM migration process because of probability calculations and updating of the proposed mechanism. Figures 3a and 3b indicate improvements in execution time and energy by BMH.
CONCLUSIONS AND FUTURE WORK
In this work we assumed that in a Cloud system, the workloads of VMs cannot be estimated beforehand and a Cloud centre has to adjust its balance when hosts become overloaded. Hosts are defined as overloaded when the CPU request from the host’s VMs exceed its computational capacity. Ideally, such a hotspot should be resolved in as few migrations as possible due to the overhead in energy consumption caused and the delay incurred. The Bayesian based heuristic proposed -BMH- chooses VMs to migrate by their degree of CPU usage. Since this parameter correlates with the effectiveness of the migration, the elimination of the hotspot, the BMH has been found to outperform two competing methods. Comparing the mean values, BMH uses 24 to 25 kilo watts less energy than the competing heuristics which can be translated to the competing heuristics having approximately 15% higher energy consumption than BMH. In addition to a lower energy consumption, the BMH has the shortest total execution times and the fewest migrations. It appears that the migration of the best suited VMs lead to fewer overall migrations, which, in turn, saves time and energy. Unfortunately the BMH has the highest SLA violation percentage among the three methods. Nevertheless, SLA violation is .0005% higher than the competing heuristics. The BMH does not capture any information about SLA violations. In further work, we will investigate how the service level can be better maintained in this mechanism. 12
0.3
700
0.25
600 500 Kilo Watts
seconds
0.2 0.15
400 300
0.1 200
0.05
100 0
0 MOH
MMT
MOH
BMH
(a) Mean execution time (seconds)
MMT
BMH
(b) Total energy consumption (kilo Watts)
1.40%
140000
1.20%
120000
1.00%
100000
0.80%
80000
0.60%
60000
0.40%
40000
0.20%
20000 0
0.00% MOH
MMT
MOH
BMH
(c) Overall SLA violation (percentage)
MMT
BMH
(d) Total number of migrations
Figure 3: The box plots for BMH in comparison to MOH and MMT
9.
REFERENCES
[1] Jonathan Koomey. Growth in data center electricity use 2005 to 2010. Oakland, CA: Analytics Press. August, 1:2010, 2011. [2] Pierre Delforge. America’s data centers consuming and wasting growing amounts of energy. Natural Resource Defence Councle, 2014. [3] Kyong Hoon Kim, Rajkumar Buyya, and Jong Kim. Power aware scheduling of bag-of-tasks applications with deadline constraints on dvs-enabled clusters. In CCGRID, volume 7, pages 541–548, 2007. [4] Anton Beloglazov and Rajkumar Buyya. Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurrency and Computation: Practice and Experience, 24(13):1397–1420, 2012. [5] Saurabh Kumar Garg, Chee Shin Yeo, Arun Anandasivam, and Rajkumar Buyya. Energy-efficient scheduling of HPC applications in cloud computing environments. arXiv preprint arXiv:0909.1146, 2009. [6] Dan Tsafrir, Yoav Etsion, and Dror G Feitelson. Backfilling using system-generated predictions rather than user runtime estimates. Parallel and Distributed Systems, IEEE Transactions on, 18(6):789–803, 2007. [7] A Suresh and P Vijayakarthick. Improving scheduling of backfill algorithms using balanced spiral method for cloud metascheduler. In Recent Trends in Information Technology (ICRTIT), 2011 International Conference on, pages 624–627. IEEE, 2011. [8] Young Choon Lee and Albert Y Zomaya. Minimizing
[9]
[10]
[11]
[12]
[13]
13
energy consumption for precedence-constrained applications using dynamic voltage scaling. In Cluster Computing and the Grid, 2009. CCGRID’09. 9th IEEE/ACM International Symposium on, pages 92–99. IEEE, 2009. Young Choon Lee and Albert Y Zomaya. Energy efficient utilization of resources in cloud computing systems. The Journal of Supercomputing, 60(2):268–280, 2012. Linan Zhu, Qingshui Li, and Lingna He. Study on cloud computing resource scheduling strategy based on the Ant Colony Optimization Algorithm. IJCSI International Journal of Computer Science Issues, 9(5):1694–0814, 2012. Eugen Feller, Louis Rilling, and Christine Morin. Energy-aware ant colony based workload placement in clouds. In Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing, pages 26–33. IEEE Computer Society, 2011. Yongqiang Gao, Haibing Guan, Zhengwei Qi, Yang Hou, and Liang Liu. A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. Journal of Computer and System Sciences, 2013. Jing Xu and J.A.B. Fortes. Multi-objective virtual machine placement in virtualized data center environments. In Green Computing and Communications (GreenCom), 2010 IEEE/ACM Int’l Conference on Int’l Conference on Cyber, Physical and Social Computing (CPSCom), pages 179–188, Dec 2010.
[14] Haibo Mi, Huaimin Wang, Gang Yin, Yangfan Zhou, Dianxi Shi, and Lin Yuan. Online self-reconfiguration with performance guarantee for energy-efficient large-scale cloud computing data centers. In Services Computing (SCC), 2010 IEEE International Conference on, pages 514–521, July 2010. [15] Yee-Ming Chen and Wen-Chien Wang. An adaptive rescheduling scheme based heuristic algorithm for cloud services applications. In Machine Learning and Cybernetics (ICMLC), 2011 International Conference on, volume 3, pages 961–966, July 2011. [16] Xin Lu and Zilong Gu. A load-adapative cloud resource scheduling model based on ant colony algorithm. In Cloud Computing and Intelligence Systems (CCIS), 2011 IEEE International Conference on, pages 296–300, Sept 2011. [17] Chunqiang Tang, Malgorzata Steinder, Michael Spreitzer, and Giovanni Pacifici. A scalable application placement controller for enterprise data centers. In Proceedings of the 16th International Conference on World Wide Web, WWW ’07, pages 331–340, New York, NY, USA, 2007. ACM. [18] D. Kusic, J.O. Kephart, J.E. Hanson, Nagarajan Kandasamy, and Guofei Jiang. Power and performance management of virtualized computing environments via lookahead control. In Autonomic Computing, 2008. ICAC ’08. International Conference on, pages 3–12, June 2008. [19] Ahmed Sallam and Kenli Li. A multi-objective virtual machine migration policy in cloud systems. The Computer Journal, 2013. [20] Ye Hu, Johnny Wong, Gabriel Iszlai, and Marin Litoiu. Resource provisioning for cloud computing. In Proceedings of the 2009 Conference of the Center for Advanced Studies on Collaborative Research, pages 101–111. IBM Corp., 2009. [21] Qiang Guan, Ziming Zhang, and Song Fu. Ensemble of bayesian predictors and decision trees for proactive failure management in cloud computing systems. Journal of Communications, 7(1):52–61, 2012. [22] Xianbin Wang, Guangjie Han, Xiaojiang Du, and Joel JPC Rodrigues. Mobile cloud computing in 5g: Emerging trends, issues, and challenges [guest editorial]. Network, IEEE, 29(2):4–5, 2015. [23] Wei Wang, Guosun Zeng, Daizhong Tang, and Jing Yao. Cloud-DLS: Dynamic trusted scheduling for cloud computing. Expert Systems with Applications, 39(3):2321–2329, 2012. [24] Feifei Chen, John Grundy, Yun Yang, Jean-Guy Schneider, and Qiang He. Experimental analysis of task-based energy consumption in cloud computing systems. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering, pages 295–306. ACM, 2013. [25] Russ Joseph and Margaret Martonosi. Run-time power estimation in high performance microprocessors. In Proceedings of the 2001 international symposium on Low power electronics and design, pages 135–140. ACM, 2001. [26] Chiyoung Seo, Sam Malek, and Nenad Medvidovic. Component-level energy consumption estimation for distributed java-based software systems. In
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35] [36]
[37]
[38]
14
Component-Based Software Engineering, pages 97–113. Springer, 2008. Ewa Deelman, Gurmeet Singh, Miron Livny, Bruce Berriman, and John Good. The cost of doing science on the cloud: the montage example. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing, page 50. IEEE Press, 2008. Rodrigo N Calheiros, Rajiv Ranjan, C´esar AF De Rose, and Rajkumar Buyya. CloudSim: A novel framework for modeling and simulation of cloud computing infrastructures and services. arXiv preprint arXiv:0903.2525, 2009. Rodrigo N Calheiros, Rajiv Ranjan, Anton Beloglazov, C´esar AF De Rose, and Rajkumar Buyya. CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience, 41(1):23–50, 2011. Gregor Von Laszewski, Lizhe Wang, Andrew J Younge, and Xi He. Power-aware scheduling of virtual machines in dvfs-enabled clusters. In Cluster Computing and Workshops, 2009. CLUSTER’09. IEEE International Conference on, pages 1–10. IEEE, 2009. Nicola Cordeschi, Mohammad Shojafar, and Enzo Baccarelli. Energy-saving self-configuring networked data centers. Computer Networks, 57(17):3479–3491, 2013. Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul, Christian Limpach, Ian Pratt, and Andrew Warfield. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation-Volume 2, pages 273–286. USENIX Association, 2005. Wu-chun Feng, Xizhou Feng, and Rong Ge. Green supercomputing comes of age. IT professional, 10(1):17–23, 2008. Vincenzo De Maio, Radu Prodan, Shajulin Benedict, and Gabor Kecskemeti. Modelling energy consumption of network transfers and virtual machine migration. Future Generation Computer Systems, 56:388–406, 2016. Kevin B Korb and Ann E Nicholson. Bayesian artificial intelligence. CRC press, 2010. KyoungSoo Park and Vivek S Pai. CoMon: a mostly-scalable monitoring system for planetlab. ACM SIGOPS Operating Systems Review, 40(1):65–74, 2006. S Sohrabi and I Moser. The effects of hotspot detection and virtual machine migration policies on energy consumption and service levels in the cloud. 51:2794–2798, 2015. Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. Xen and the art of virtualization. ACM SIGOPS Operating Systems Review, 37(5):164–177, 2003.
