Virtual Machine Based Energy-Efficient Data Center ... - CiteSeerX

2010 IEEE/ACM International Conference on Green Computing and Communications & 2010 IEEE/ACM International Conference on Cyber, Physical and Social Computing

Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing: A Performance Perspective Kejiang Ye, Dawei Huang, Xiaohong Jiang, Huajun Chen, Shuang Wu College of Computer Science, Zhejiang University Hangzhou 310027, China {yekejiang,tossboyhdw,jiangxh,huajunsir,catting}@zju.edu.cn

As reported in [7], the power consumption of worldwide enterprises exceeds $30 billion in 2008. The rated power consumptions of servers have increased by 10 times over the past ten years. What’s more, it’s worth noting that the server management and maintenance costs and electricity and cooling costs in modern data center have exceeded the server equipment costs. Due to the huge energy cost in data center, there is an urgent need of designing and deployment of energy-efficient technologies for building a green data center [8]. Fortunately, The energy consumption problem has already attracted enough attention [9]. Many efforts have been made to improve the energy efficiency of data center from different aspects including processor energy efficiency [10], storage power management [11] and network power management [12]. Recently, with the rapid development of virtualization technology, such as VMware [13], Xen [14], KVM [15], OpenVZ [16], more and more data centers use this technology to build new generation data center architecture to support cloud computing due to the benefits such as improving resource utilization, reducing costs, easing server management. What’s more, server consolidation and live migration of virtual machine are two crucial methods to achieve load balancing and energy saving. Server consolidation which allowing multiple servers running in a single physical server simultaneously is a main approach to achieve better energy efficiency of data center. It is because in doing so, server consolidation allows more physical servers to be turned off via live migrating the virtual machines to other unsaturated physical servers. Although virtual machine technology can improve the energy efficiency in data center, the overheads caused by virtualization and the efficiency of consolidation and migration strategies need to be investigated. In this paper, we first present a virtual machine based energy-efficient data center architecture for cloud computing and discuss the details. Then, we evaluate the potential performance overheads of server consolidation and investigate the consolidation efficiency with different consolidation strategies that will affect the energy efficiency with varying degrees. After that, we investigate the trade-off of the performance degradation and energy saving. Finally, we explore the process of live migration of virtual machine and

Abstract—Virtual machine technology is widely applied to modern data center for cloud computing as a key technology to realize energy-efficient operation of servers. Server consolidation achieves energy efficiency by enabling multiple instantiations of operating systems (OSes) to run simultaneously on a single physical machine. While, live migration of virtual machine can transfer the virtual machine workload from one physical machine to another without interrupting service. However, both the two technologies have their own performance overheads. There is a tradeoff between the performance and energy efficiency. In this paper, we study the energy efficiency from the performance perspective. Firstly, we present a virtual machine based energy-efficient data center architecture for cloud computing. Then we investigate the potential performance overheads caused by server consolidation and live migration of virtual machine technology. Experimental results show that both the two technologies can effectively implement energy-saving goals with little performance overheads. Efficient consolidation and migration strategies can improve the energy efficiency. Keywords-Virtual Machine; Server Consolidation; Live Migration; Energy Efficiency

I. I NTRODUCTION Cloud computing [1] has recently received considerable attention in both academic community and industrial community as a new computing paradigm to provide dynamically scalable and virtualized resource as a service over the Internet. By this means, users will be able to access the resources, such as applications and data, from the cloud anywhere and anytime on demand. Currently, several large infrastructure companies, such Amazon [2], Google [3], Yahoo! [4], Microsoft [5], IBM [6] and Sun, are developing cloud platforms for consumers and enterprises to access the cloud resources through services. Data center is traditional concept that provides powerful computing and storage capacity for crucial areas, such as particle physics, scientific computing and simulation, earth observation and oil prospecting and so on. A data center usually deploys hundreds or thousands of blade servers which are densely packed to maximize the space utilization and management efficiency. As the rapid growth of server quantity and scale, the energy consumed by the data center, which is directly related to the number of hosted servers and their workloads, is becoming a great challenge. 978-0-7695-4331-4/10 $26.00 © 2010 IEEE DOI 10.1109/GreenCom-CPSCom.2010.108

171

related issues. Experimental results show that both the two technologies can effectively implement energy-saving goals with little performance overheads. Efficient consolidation and migration strategies can improve the energy efficiency. The rest of the paper is structured as follows. In Section II, we introduce the background of cloud computing, virtualization and VM-based energy-efficient technologies. In Section III, we present the architecture of VM-based data center for cloud computing. In Section IV, we perform a comprehensive evaluation and analysis on the server consolidation and live migration of virtual machine. Section V presents the related work. Finally we give our conclusion and future work in Section VI.

cloud environment. Pre-copy technology [23, 24] is prevailing live migration technology to perform live migration of virtual machine, which is used in the mainstream virtual machine monitors such as VMware, Xen, KVM and OpenVZ. There are four metrics to quantify the migration algorithms: downtime, total migration time, total data transmitted and Workload’s QoS. In order to minimize the downtime, precopy approach fist copies memory pages to the destination machine recursively while keeping VM service still available. When the applications’ writable working set becomes small and reaches the threshold, the virtual machine is suspended and only CPU state and dirty pages in the last round are sent out to the destination machine. What’s more, a new approach named CR/TR-Motion [25] transfers check-point and execution trance files rather than memory pages in precopy phase. Additionally, memory compression technology is used to reduce the amount of data transmitted in the migration process [26]. What’s more, other methods [27, 28] are also used to optimize the live migration of virtual machine.

II. BACKGROUND & M OTIVATION In this section, we will survey the open source cloud platforms and introduce the live migration of virtual machine. The VM-based energy-efficient mechanism is also presented. A. Open Source Cloud Platforms With the development of cloud computing, there emergence a lot of open source projects for cloud computing. Eucalyptus [17, 18]: is a VM-based private cloud management platform developed by the University of California at Santa Barbara. Eucalyptus provide a flexible architecture allowing the users to make experiments on cloud-related software and infrastructure. It realizes the Amazon cloud platform and mimics the deployment of Amazon’s EC2 and S3 in a private cluster. OpenNebula [19]: is an open-source toolkit to easily build private, public and hybrid cloud. OpenNebula has been designed to be integrated with networking and storage solution to fit into any existing data center. Currently, it both supports Xen and KVM. What’s more, it is the only platform to officially support live migration in its documentation. ECP (Enomaly Elastic Computing Platform) [20]: is a well-suited platform for VM-on-Demand scenarios that provides Infrastructure-on-demand (IaaS) cloud computing services to the customers quickly and easily. However, the limited network management makes it limited to small-scale installations. oVirt [21]: is a virtualization management framework constisting of a small host image, that provides the libvirt service to host virtual machines, and a robust VM management software stack, controlled by a web-based management interface. Nimbus [22]: is an open source toolkit that allows the user to turn cluster into an Infrastructure-as-a-Service (IaaS) cloud.

C. VM-Based Energy-Efficient Technology Various management strategies have been developed to effectively reduce the power consumption from different aspects, however they cannot be directly applied to today’s data centers that rely on virtualization technologies. Virtual machine technology can efficiently manage the server consolidation, and improve the total power efficiency in data center. Nathuji et al. [29] have proposed an online power management to support the isolated and independent operations in virtual machines and coordinate the different power management strategies applied by the virtual machines to the virtualized resource. They use the virtual power to represent the soft versions of the hardware power state, facilitating the deployment of the power management policies. They also implement Virtual Power Management (VPM) state, channels, mechanisms, and rules to map the soft power state to the actual changes of the underlying virtualized resource. Differ from the virtual power method, we improve the virtual machine based energy efficient method at the level of virtual machine management, such as server consolidation and live migration of virtual machine. Virtualization partitions computational resources and allows the sharing of hardware. Many services often need only a small fraction of the available computational resources in a data center server [30]. However, even when run at a low utilization, servers will consume 70% of their maimum power. Such services can be virtualized and run within a virtual machine leading to significant increases in overall energy efficiency. Based on the server utilization, many virtual machines can run on a single physical server (server consolidation). Therefore, less physical servers are needed overall, thus reducing energy wasted for electricity and cooling. Live migration of virtual machine realizes the server consolidation.

B. Live Migration of Virtual Machine There are two migration methods: suspend/resume migration and live migration. Due to the long downtime in suspend/resume migration, live migration is widely used in

172

p

(

#



2

!

1

(

q

$

"

$

r

&

$





s

.

'

&

2

#

6

t



u

(

!







#

2

0

4

$



 t

*



!

+

"



&

%

,



&

"

-

#

$

.

'

&

2

#

#

.

$

)

!

&

$





,

.

$

)

!

&

$





'

"

)



!

$

/

0

7

8







#



"

$

%

&

$







!

&

(

)

$

(

k




l



k



l








k

m

n

o



#

7

'

9

!

&

(

)

$

(

#

:







#



"

$

%

&

$











!



"

.

$

)

!

&

$











!



"

^

_

`

a

b

c

d

^

_

`

a

b

c

d

^

j

_

`

a

b

c

d

j

;

e

c

f

g

_

h

i

e

c

f

g

_

h

i

e

c

f

g

_

h

i

:



















!



"



















!



"

8





























 

















Q

K




?

K

: A

'

(

1

2

!

$

( -

4

"



-

0

(

B

C

D

E

F

G

H

F I

B

M

P

I



@

.

&



&

)

(

0

(



.

3

&



&

)

(

0

(



5

5

J

K

L

B

K

Figure 1.

E

M

N

L

J

O K

L

K

B

E

M

L

N

Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing.

to consolidate several workload servers into a single physical server and switch off idle physical machines to reduce energy consumption with various consolidation strategies. If the consolidation decision is made, Consolidation Controller will respond to the consolidation strategies and switch on/off the physical servers. While in the Migration Management, there is a migration algorithm library that contains a number of migration algorithms, such as Pre-copy migration, Postcopy migration and memory compression based migration, etc. What’s more, the migration decision is also an important issue to answer when and where to migrate the virtual machines. It is an essential module because the migration location and time will greatly influence the overall energy consumption. Similar to Consolidation Management, the Migration Management also has Migration Controller to implement the migration task. Cloud Server Module plays an important role in cloud computing that in which all the resources are provided to the users with services via Internet.

III. VM-BASED E NERGY-E FFICIENT DATA C ENTER A RCHITECTURE In this section, we propose the VM-based energy-efficient data center architecture for cloud computing, as shown in Figure 1. As discussed above, energy efficiency is an importance challenge in today’s data center. To address such challenge, we design a virtual machine based for energy-efficient architecture. Virtualization technology can consolidate computing resources, reduce management complexity and speed the response to business dynamics. The architecture is consisted by four main modules: Virtualization Module, Monitoring Module, Management Module and Cloud Service Module. We are concerned about energy efficiency by focusing on the Energy Management Module and Monitoring Module. Virtualization Module is an abstraction layer that shielding heterogeneous physical resources and provides dynamical, scalable virtual resources to users on demand. Currently, Xen and KVM are widely used in cloud environment. The Monitoring Module is responsible for monitoring both virtual machines and physical machines, including resource utilization, power consumption and virtual machine status, etc. We implement this module by using vTestkit, a performance benchmarking framework for virtualization environments [31]. Management Module implements all the management issues in the data center cloud, including Energy Management Submodule, Security Management Submodule and Deployment Management Submodule, etc. The Energy Management Submodule is a key module in our energy-efficient architecture which is responsible for the energy management including Consolidation Management and Migration Management. In Consolidation Management, firstly we analyze the resource consumption characterization of each workload server which is importance to help advise suitable consolidation strategy. After that, we need to consider how and when

IV. P ERFORMANCE E VALUATION & A NALYSIS In this section, we investigate the VM-based data center architecture for cloud computing from the performance perspective. We first investigate the resource demands characterization of each workload which may affect the consolidation efficiency. Then, we perform the consolidation experiments with various consolidation strategies affecting the energy consumption. Next, the trade-off of the performance degradation and energy saving is presented. At last, we evaluate the live migration of virtual machine focusing on downtime, total migration time, total data transmitted and QoS degradation. A. Experimental Configuration and Benchmarks The resource demands characterization experiment of various workloads and server consolidation experiment are performed on the Dell 2900 PowerEdge server with two 173

Quad-core 64-bit Xeon processors at 2GHz, with a 6MB second level cache. The system has 6GB physical memory. We use Ubuntu 8.10 with kernel version 2.6.27 in domain0, and the version of Xen hypervisor is 3.3.1, which has built-in support for Oprofile. All the virtual machines are configured with 4 vCPU and 1GB memory size. The benchmarks used in the server consolidation are SPECjbb2005 for java server, IOzone for file server, Sysbench for database server and Webbench for web server. While the live migration of virtual machine is performed on three Dell OPTIPLEX 755, with Intel Core2 Quad CPU at 2.4GHz and 2GB physical memory. One is for source migration server, one is for target migration server, and the other is for NFS storage server. The benchmark used in the live migration of virtual machine is SPECjvm2008. The migrated virtual machine is configured with 512MB virtual memory, 1VCPU and 10GB disk. In order to ensure the data precision, each of the showed experiment results was obtained via running benchmarks five times on the same configuration, the highest and lowest values for each test were discarded, and the remaining three values were averaged.

such as core, cache, memory, etc. From this experiment, we observe that there is a performance overhead of server consolidation. However, we can improve the consolidation efficiency with suitable consolidation strategies based on the workloads characterization analysis. C. Resource Demands Characterization of Various Workloads In order to achieve energy efficiency with server consolidation and investigate how different workloads consolidation can affect the energy efficiency, it is necessary to study the characterization of each workload and the consolidation efficiency of various consolidation strategies. Figure 3 illustrates the performance impact of number of VCPU on each workload. We observe that when the VCPU number scales from 1 to 16, the performance of SPECjbb first increases then decreases and reaches the best performance when VCPU number is 5. This result accords to our original expectation when design the experiment. We allocate 2 VCPUs to each virtual machine. When the virtual machines allocated on the single physical platform (Dell 2900 with two Quad-core) are increasing to reach 5, the number of total VCPUs (10) are greater than the physical cores (8), which will definitely lead to core interference issue. If the VCPU number is keeping on growing, then the core interference will be getting more serious and the scheduling overheads will be getting heavier, so that the performance is turned to decreasing with the VCPU increasing. Different from the SPECjbb, the performances of IOzone and Sysbench keep relatively stable while the number of VCPU growing. This is because the IOzone and Sysbench are both I/O intensive but CPU insensitive workloads. The performance for Webbench increases when VCPU number goes from 1 to 5, but becomes steady slow decreasing after that. It is because Webbench needs CPU resource to handle network I/O operations but it is not CPU-intensive workload. On the other hand, in the vMemory scalability experiment as shown in Figure 4, we observe that the performance of IOzone, Sysbench, and Webbench are quite stable, while the performance of SPECjbb remains relatively stable when the vMemory between 768MB and 1792MB, and has a jump when the vMemory changes to 2048MB and keeps stable when the vMemory scales to 4096MB. The performance increase may be a result of the JVM using a large heap. We summarize the resource demand characterization of the four workloads as follows: i) SPECjbb is sensitive to core interference and memory resource, and need careful consideration when consolidating with other servers. ii) IOzone and Sysbench are insensitive to the VCPU number and vMemory resource. iii) While Wenbbench is not only network I/O intensive, but also consumes some CPU resource to process the I/O requirements.

B. Performance Characterization of Server Consolidation

Figure 2. The Performance Overheads of Server Consolidation with Java server, File server, Database server and Web server.

To understand the performance overheads of server consolidation, we have compared the performance running alone and in consolidated mode. As expected, the performance of each workload decreases due to the resource contention from other workloads within consolidation scenario. Figure 2 shows the performance of the workloads running in alone and consolidated mode. We normalized the result to a fraction of the alone mode. We observe that compared with individual mode, the performance of each workload has a degradation in some extend in consolidated mode with SPECjbb losing 10.50% performance, IOzone losing 31.36% performance, Sysbench losing 22.27% performance, and Webbench losing 26.48% performance. The reason for this loss in performance is due to the shared resource contention

174

Figure 3.

Figure 4.

The VCPU resource demands characterization of SPECjbb, IOzone, Sysbench, and Webbench

The vMemory resource demands characterization of SPECjbb, IOzone, Sysbench, and Webbench

Table I P ERFORMANCE C HARACTERIZATION C OMPARISON OF SPEC JBB , IO ZONE , S YSBENCH , AND W EBBENCH WHEN C ONSOLIDATING WITH O THER W ORKLOADS .

SPECjbb (bops) IOzone (Kbytes/sec) Sysbench (s) Webbench (Pages/min)

Individual 32371 193980 45.01 905286

with SPECjbb 26145 113844 56.67 711441

D. Server Consolidation Efficiency with Various Strategies

with IOzone 30012 51103 64.79 762564

with Sysbench 30663 66395 49.56 821360

with Webbench 27941 99823 59.92 507092

CPU intensive, when consolidating two SPECjbb workloads together, there will be a heavy pressure of CPU resource, while leaving other system resources with a low utilization like network bandwidth resource. From this experiment, we also can find that which two kinds of workloads can coexist well, and which two kinds of workloads are mutually exclusive. We see the SPECjbb and Sysbench workloads are most friendly with each other, leading to least performance loss, while IOzone and Sysbench workloads are least suitable to consolidate together. It is because that the SPECjbb consumes a lot of CPU resource while Sysbench consumes a lot of I/O resource and also consumes a small amount of CPU, the consolidation of the two workloads will make the best use of the different aspects of system resources. On other hand, IOzone and Sysbench are both I/O intensive, when consolidating the two workloads, I/O becomes a main performance bottleneck. Besides, we find the IOzone is the most vulnerable to any other workload due to its heavy and slow I/O operations. So it’s better to run the IOzone workload in a stand-alone server.

Although server consolidation can reduce the number of physical machines, different consolidation strategies can affect the workloads’s performance with varying degrees. In this section, we will investigate the performance effects when consolidating every two workloads together, which is useful to understand the isolation between every two workloads, and can give guidance to make appropriate consolidation strategies to achieve the best workloads’ performance and data center energy efficiency. Table I shows the performance characterization comparison of SPECjbb, IOzone, Sysbench, and Webbench when consolidating with other workloads. The individual mode means the performance result of workloads when running in individual virtual machine instead of consolidating with other virtual machines. Obviously, consolidating with other workloads will lead to different degree of performance loss. From the table, we can find that all the workloads have a performance degradation in the consolidation scenario. What’s more, we find the SPECjbb, IOzone, and Webbench achieve the worst performance when consolidating with the same workloads. It is not suitable to consolidate two same workloads in a single platform because of the contention of the same kind resource. For example, the SPECjbb is

E. Trade-off of Performance Degradation and Energy Saving We perform the trade-off experiment to investigate the relationship between the performance degradation and en-

175

RADE - OFF OF

T

about 512MB. The workload’s QoS is also an key metric indicates the migration efficiency. We compare the workload performance before and after migration in Figure 8. As shown in the figure, the workload running in a migrated virtual machine has a certain degree of performance degradation compared with workload running in normal virtual machine. However the performance difference is not very clear. Based on the above analysis, we find that live migration of virtual machine incurs little performance overheads and can be used to as an important way to improve the energy efficiency.

Table II P ERFORMANCE D EGRADATION AND E NERGY S AVING .

Workloads Java Server File Server Database Server Web Server

Performance Decreased (%) 30 30 30 30

VM Number Saved 4 2 2 1

ergy saving. Server consolidation can cause the performance degradation compared with individual running mode. In other words, the sacrifice of performance or the QoS (Quality of Service) of the workloads can exchange for the energy saving. Table II shows the results of the relationship between the performance degradation and energy saving. From the Table, we find when the workloads have 30% performance decrease, various number of physical machines can be shutdown to save the energy. Java server obtains the most obvious effect. It is because the java server is a kind of CPU intensive workloads and can receive a balanced scheduling by the CPU scheduler. While the memory-intensive and network intensive workloads have more stringent demands on corresponding resources.

V. R ELATED W ORK Energy efficiency is a key challenge in data center. Many efforts have been made to improve the energy efficiency of data center. They achieve the goal of energy efficiency from the aspects including processor energy efficiency [10], storage power management [11] and network power management [12]. However, with the deployment of virtualization, modern data center has employed virtualization technology to support cloud computing. Due to the new computing paradigm, traditional energy efficient technology cannot be directly used in today’s data center. Server consolidation and live migration of virtual machine are two main ways to achieve energy efficiency. Apparao et al. [32] has evaluated the server consolidation performance with vConsolidation benchmark with detailed architecture characterization analysis, such core interference and cache interference. A number of work has been done on the algorithm of live migration of virtual machine, such Pre-copy technology [23, 24], Postcopy technology [27], log-based migration technology [25], memory compression based migration technology [26] and whole system migration technology [28]. Nathuji et al. [29] have proposed an online power management to support the isolated and independent operations in virtual machines and coordinate the different power management strategies applied by the virtual machines to the virtualized resource. They use the virtual power to represent the soft versions of the hardware power state, facilitating the deployment of the power management policies. They also implement Virtual Power Management (VPM) state, channels, mechanisms, and rules to map the soft power state to the actual changes of the underlying virtualized resource. Berl et al. [33] have reviewed the challenges and existing methods to achieve the energy-efficient cloud computing. However, they didn’t perform a experimental evaluation. Liu et al. [8] present a GreenCloud based on the VM live migration technology, but leaving server consolidation untouched. Recently, Liao et al. [34] have use the virtual machine technology to optimize the energy in cluster environment. Differ from the above work, we study the virtual machine based energy efficient data center architecture for cloud

F. Live Migration of Virtual Machine From Figure 5 to Figure 8, we show the results of live migration of SEPECjvm2008 workload running in virtual machine from one physical machine to another physical machine. Four performance metrics are measured in the migration process: downtime, total migration time, total data transmitted and workload’s QoS. As shown in Figure 5, most workloads achieve similar downtime about 50ms due to the similar workload characterization with few memory operations. However, it is obvious that the compress workload achieves highest downtime with 114ms. It is because that compress causes large number of compression operations and incurs frequent memory read and write operations leading to considerable dirty pages. Excessive memory operations make the dirty pages and CPU status can not be migrated in the pro-copy process, and need to stop and transfer the remaining data. What’s more, startup is a kind of composite workload and mpegaudio is a workload that processes video decoding. Both the two workloads also involve a number of memory operations thereby affecting downtime. Figure 6 indicates the result of total migration time which shows the similar phenomenon to the downtime. As explained in the above, compress, startup and mpegaudio workloads are memory-intensive and incur rapid pollution of memory pages which eventually affects the total migration time. Figure 7 shows the total data transmitted. It is a important metric that may influence the network throughput. Obviously, the data transmitted maintains at a stable level with

176

Figure 5.

The Downtime of Live Migration of SPECjvm2008 in Xen. Figure 6. The Total Migration Time of Live Migration of SPECjvm2008 in Xen.

Figure 7. The Total Data Transmitted of Live Migration of SPECjvm2008 in Xen.

computing from the performance perspective and perform the efficiency of server consolidation and live migration of virtual machine.

Figure 8.

The QoS of Live Migration of SPECjvm2008 in Xen.

Future work will include design and implement intelligent consolidation and live migration mechanism to improve the energy efficiency. Based on this intelligent decision made by the consolidation and migration mechanism, the data center can maintain energy efficiency automatically.

VI. C ONCLUSION & F UTURE W ORK Cloud computing as a new computing paradigm used to proved dynamically scalable and virtualized resource as a service over the Internet. Today’s data center has widely involved virtualization technology to realize cloud computing. As the rapid growth of server quantity and scale in data center, the energy consumed by data center which is directly related to the number of hosted servers and their workloads, is becoming a great challenge. And it is urgent to design and deploy energy-efficient technologies for data center. In this paper, we firstly present a virtual machine based energy efficient data center architecture for cloud computing. Then, we evaluate the potential performance overheads of server consolidation and investigate the consolidation efficiency with different consolidation strategies that will affect the energy efficiency with varying degrees. After that, we explore the process of live migration of virtual machine and related issues. Experimental results show that both the two technologies can effectively implement energy-saving goals with little performance overheads. Efficient consolidation and migration strategies can improve the energy efficiency.

ACKNOWLEDGMENT This work is funded by the National 973 Basic Research Program of China under grant NO.2007CB310900 and National Natural Science Foundation of China under grant NO. 60970125. R EFERENCES [1] M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, et al., “Above the clouds: A berkeley view of cloud computing,” EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-28, 2009. [2] “Amazon elastic compute http://www.amazon.com/ec2, 2010.

cloud

(ec2),”

[3] “Google app engine,” http://code.google.com/appengine, 2010.

177

[4] B. Cooper, E. Baldeschwieler, R. Fonseca, J. Kistler, P. Narayan, C. Neerdaels, T. Negrin, R. Ramakrishnan, A. Silberstein, U. Srivastava, et al., “Building a Cloud for Yahoo!” Data Engineering, p. 36, 2009.

[20] “Ecp: Enomaly elastic http://www.enomaly.com/, 2010.

computing

[5] “Microsoft azure cloud http://www.microsoft.com/windowsazure, 2010.

[22] “Nimbus,” http://www.nimbusproject.org/, 2010.

platform,”

[21] “ovirt,” http://ovirt.org, 2010.

platform,”

[23] C. Clark, K. Fraser, S. Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield, “Live migration of virtual machines,” in Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation-Volume 2, 2005, p. 286.

[6] “Ibm blue cloud project,” http://www.ibm.com/ibm/cloud, 2010. [7] R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, and X. Zhu, “No power struggles: Coordinated multi-level power management for the data center,” ACM SIGPLAN Notices, vol. 43, no. 3, pp. 48–59, 2008.

[24] M. Nelson, B. Lim, and G. Hutchins, “Fast transparent migration for virtual machines,” in Proceedings of the annual conference on USENIX Annual Technical Conference, 2005, p. 25.

[8] L. Liu, H. Wang, X. Liu, X. Jin, W. He, Q. Wang, and Y. Chen, “GreenCloud: a new architecture for green data center,” in Proceedings of the 6th international conference industry session on Autonomic computing and communications industry session, 2009, pp. 29–38.

[25] H. Liu, H. Jin, X. Liao, L. Hu, and C. Yu, “Live migration of virtual machine based on full system trace and replay,” in Proceedings of the 18th ACM international symposium on High performance distributed computing, 2009, pp. 101–110.

[9] “The green 500 list,” http://www.green500.org/, 2010.

[26] H. Jin, L. Deng, S. Wu, X. Shi, and X. Pan, “Live virtual machine migration with adaptive memory compression,” in Proc. CLUSTER, 2009.

[10] D. Brooks and M. Martonosi, “Dynamic thermal management for high-performance microprocessors,” in The Seventh International Symposium on High-Performance Computer Architecture, 2001, pp. 171–182.

[27] M. Hines and K. Gopalan, “Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning,” in Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, 2009, pp. 51–60.

[11] E. Carrera, E. Pinheiro, and R. Bianchini, “Conserving disk energy in network servers,” in Proceedings of the 17th annual international conference on Supercomputing, 2003, p. 97. and via 5th Im-

[28] Y. Luo, B. Zhang, X. Wang, Z. Wang, Y. Sun, and H. Chen, “Live and incremental whole-system migration of virtual machines using block-bitmap,” in 2008 IEEE International Conference on Cluster Computing, 2008, pp. 99–106.

[13] C. Waldspurger, “Memory resource management in VMware ESX server,” ACM SIGOPS Operating Systems Review, vol. 36, no. SI, p. 194, 2002.

[29] R. Nathuji and K. Schwan, “VirtualPower: coordinated power management in virtualized enterprise systems,” in Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles. ACM, 2007, p. 278.

[14] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield, “Xen and the art of virtualization,” in Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, p. 177.

[30] “Ibm virtualization view: Virtualization can help power efficiency,” http://www03.ibm.com/systems/virtualization/news/view/011607.html, 2010.

[15] A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori, “kvm: the Linux virtual machine monitor,” in Proceedings of the Linux Symposium, vol. 1, 2007, pp. 225–230.

[31] K. Ye, J. Che, X. Jiang, J. Chen, and X. Li, “vTestkit: A Performance Benchmarking Framework for Virtualization Environments,” in Proceedings of fifth ChinaGrid Annual Conference, 2010, pp. 130–136.

[12] S. Nedevschi, L. Popa, G. Iannaccone, S. Ratnasamy, D. Wetherall, “Reducing network energy consumption sleeping and rate-adaptation,” in Proceedings of the USENIX Symposium on Networked Systems Design and plementation, 2008, pp. 323–336.

[16] “Openvz: Server virtualization http://openvz.org, 2010.

open

source

project,”

[32] P. Apparao, R. Iyer, X. Zhang, D. Newell, and T. Adelmeyer, “Characterization & analysis of a server consolidation benchmark,” in Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, 2008, pp. 21–30.

[17] “Eucalyptus cloud platform,” http://www.eucalyptus.com/, 2010. [18] D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and D. Zagorodnov, “The eucalyptus open-source cloud-computing system,” in Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid-Volume 00, 2009, pp. 124–131.

[33] A. Berl, E. Gelenbe, M. Di Girolamo, G. Giuliani, H. De Meer, M. Dang, and K. Pentikousis, “Energy-Efficient Cloud Computing,” The Computer Journal, 2009. [34] X. Liao, L. Hu, and H. Jin, “Energy optimization schemes in cluster with virtual machines,” Cluster Computing, vol. 13, no. 2, pp. 113–126, 2010.

[19] “Opennebula: The open source toolkit for cloud computing,” http://www.opennebula.org/, 2010.

178