Profiling core operations for elasticity in cloud ... - IEEE Xplore

Profiling core operations for elasticity in cloud environments Ernani Azevêdo, Cyrus Dias, Ramide Dantas, Djamel Sadok, Stenio Fernandes

Rhodney Simões, Carlos Kamienski Universidade Federal do ABC (UFABC)

Universidade Federal de Pernambuco (UFPE) Recife, Brazil {ernani, cyrus, ramide, jamel, stenio}@gprt.ufpe. br

Santo André, Brazil {rhodney.simoes, cak}@ufabc.edu.br performance of the hypervisor, is useful for isolating the effect of cloud services and virtualization. This paper shows the results of a variety of performance experiments conducted in a controlled environment for the Infrastructure as a Service (IaaS) model. It is worth mentioning that the level of control differs significantly for public and private clouds. For instance, the EC2 service creates its virtual machines (called instances) in any physical machine available in their infrastructure, according to Amazon policies, regardless of the preferences of the cloud user. As for OpenNebula, the cloud administrator can exert some control on physical servers available for the experiments with virtual machines. Our results show that elasticity may be provided by EC2, OpenNebula, and Xen with response times that comply with the necessary agility for most of cloud elasticity needs, so that the user choice for these systems does not exclusively depend on performance, but on other technical, financial or strategic criteria. All in all, the first conclusion that makes common sense is that the use of Xen, without any middleware abstraction layer, presents the best performance results, at the expense of additional management complexity. OpenNebula makes it easier to manage resources in a private cloud, providing more flexibility but with some performance penalties. Amazon EC2 is a comfortable choice for users with seasonal demand patterns, presenting higher performance variability as well as reduced control over physical resources. Users must choose the most adequate platform for their needs. For example, in the case where budget is a concern but there are plenty of physical servers and processing time (for elasticity operations) and so performance is not a critical factor, the best choice may be OpenNebula. The controller performs automatically a variety of tasks typical of the system administrators in environments where hypervisors are used with no additional cloud support. In case higher control is required over hardware resources or in case response time for elasticity matters, the direct use of the hypervisor (i.e., Xen) may be the best choice. When the organization lacks the expertise needed for managing its own resources or physical servers and expertise are available but budget is not a concern, Amazon EC2 may be the best choice. The rest of the paper is structured as follows. Section II presents background and related work while Section III presents the platforms and tools used in our experiments. Section IV describes the evaluation methodology, whose results are presented in section V. Finally, section VI presents conclusions and future work.

Abstract - Cloud computing provides on-demand allocation and deallocation of resources via the key feature of elasticity. Although there are many choices of public and private clouds, detailed performance analysis for elasticity operations in such environments are not available. This paper compares the performance of core elasticity operations for the Amazon EC2 public cloud service with private clouds controlled by OpenNebula on top of Xen. Both public and private cloud environments are also compared against the Xen hypervisor alone. Our results show that elasticity can be provided by both cloud services with acceptable response times. We identified that EC2 is always faster than OpenNebula and sometimes even faster that our Xen setup. Index Terms – Cloud Computing; Performance Analysis; Elasticity I.

INTRODUCTION

The long lasting goal of providing computing services as utility is becoming a reality with the dissemination of public and private datacenters in the network cloud. Similar to electricity, water, and gas, cloud services make it possible for the consumers (or users) to use cloud resources on demand, and pay for the usage accordingly. This distinctive feature of clouds, called elasticity, allows different resources, such as virtual machines and storage, to be made available quickly in order to meet instantaneous application demands or to be freed when they are no longer needed [6]. For an organization to clearly assess the trade-offs and the effectiveness obtained by hosting services in the cloud, it needs precise technical, financial, and strategic information about its performance. However, even though several public cloud services and private cloud platforms exist, to the best of our knowledge, no relevant performance results are available when it comes to crucial tasks, such as resource allocation and deallocation. We argue that this information is fundamental in order to help users to understand the challenges involved in providing elasticity in public and private clouds. This paper compares the performance of the Amazon EC2 [9] public cloud service, with a private cloud implementation managed by the OpenNebula [10] controller, putting them in perspective by comparing the performance obtained directly from the Xen hypervisor [13]. The evaluation focuses specifically on virtual machine tasks, such as creation and teardown, suspend and resume, and reboot. Since both Amazon EC2 and OpenNebula use Xen internally, a comparison of public and private clouds, together with the

978-1-4673-5163-8/12/$31.00 © 2012 IEEE

43

II.BACKGROUND AND RELATED WORK Cloud computing is a recent paradigm for resource provisioning aimed at providing dynamic scalability and elasticity to the users on an “as a service” basis. Cloud services are charged as utilities, similar to electricity, water, and gas. This model allows IT services to be provided on demand and users only pay for those resources effectively used. A typical user profile who particularly benefits from cloud computing is an entrepreneur of innovative Internet services who does not need to pay up-front for hardware and human resources for running a new service or company. Figure 1 compares the traditional service provisioning model to the cloud model. In the traditional model users need to create and maintain an IT infrastructure, with hardware and connectivity, and need to be directly responsible for service maintenance. On the other hand, the cloud model allows users to focus on their core business, relying on a third-party for keeping the service running adequately with enough resources.

(a)

(b)

Figure 1: Service provisioning models; (a) traditional; (b) cloud.

Clouds may be classified as public, private and hybrid clouds [6], as depicted by Figure 2. Cloud services offered for the general public in a multi-tenant environment are called public clouds. As for private clouds, they are owned and managed by organizations for improving the performance and cost effectiveness of their businesses. Hybrid clouds are a result of excess service demand in private clouds being transferred to public ones. Also, there are three cloud models, typically known as Software as a Service (SaaS; users are endusers), Platform as a Service (PaaS; users are developers) and Infrastructure as a Service (IaaS; users are administrators). This paper deals with IaaS performance analysis.

from each option. However, even though there are some qualitative comparisons available [1] [2], there is no comprehensive performance analysis of VM manipulation operations of private cloud controllers as compared to public clouds as well as to simply (non-cloud) virtualized environments. A comparison of the cloud controllers Eucalyptus1, OpenNebula, and Nimbus 2is presented in [5] with emphasis on their internal structure and the impact of their design on features offered to their users. In [1] a qualitative analysis of Abicloud, Eucalyptus, Nimbus, and OpenNebula is presented. While it presents a list of resources of cloud controllers, in addition to a description of their internal architectures and their distinctive features, no performance results are presented. Also, [2] presents a qualitative comparison of Enomaly ECP, Eucalyptus, OpenNebula, and Virt, with the focus on operating system-like characteristics, such as image manipulation for virtualization and the management of quotas, clusters, and network. In the end, the strengths of each controller are summarized. The authors of [3] undertook a quantitative evaluation of the Amazon EC2 service, using instances of the type small in their experiments. The first experiment set was aimed at measuring the stability of the VM’s performance, where instances were stressed under intense workloads, both in terms of CPU cycles and I/O. The second set dealt with the homogeneity of instance performance, where instances were continually evaluated for 6 hours with an I/O-intensive workload. The last experiment set was aimed at establishing a correlation between CPU and I/O. However, no comparison was made with another cloud platform – public or private – using open source cloud controllers, as we present in this paper. A comparison of the public clouds Amazon EC2, Flexiscale and Rackspace is presented in [4], where the benchmark tool Phoronix Test Suite was used to generate load by running tests which demand high processing. The authors’ goal was to warn users of the real value they obtain for their expenditure on cloud services, primarily on processing resources. A performance analysis is presented in [11], which compares OpenNebula, Eucalyptus, and Amazon EC2. The authors focused on the analysis of the virtual machine creation times for each platform. The particular goal was to find the best configuration for the storage layer of OpenNebula that optimizes batch processing and throughput of web applications. We do not analyze the impact of different configurations of OpenNebula. We took a different research path and investigate the effect that the cloud control layer has on key virtual machine management activities, such as creation and shutdown. III. PLATFORMS

Figure 2: Three types of cloud service provision.

A private cloud needs a specific software platform, here called the cloud controller, for making it possible to automatically manage the resources in a datacenter in order to achieve the benefits of cloud computing. When deciding which cloud controller to adopt, organizations would appreciate to know the expected performance they will obtain 44

Three types of cloud platform were considered for evaluation. Among them, the tools differ in the level with which they interact with the hardware resources and with the user, and the means through which they use the underlying network capabilities. 1 2

Project page: http://www.eucalyptus.com/ Project page: http://www.nimbusproject.org/

2012 IEEE Latin America Conference on Cloud Computing and Commmunications (LatinCloud)

The hypervisor, or virtual machine manager, is the most essential cloud tool. It allows the hardware resources to be sliced and shared among the virtual servers that integrate the cloud. Most of the hypervisors use the libvirt3 virtualization library, which was designed to ease the creation and handling of virtual machines [5]. Cloud controllers are placed in an upper level of the virtual machine provision architecture and represent an abstraction between the commands passed through the front-end and the components underneath (e.g., the hypervisor itself, network infrastructures, storage, and image repositories). Controllers translate an instruction into many control signals that actually enforce the user goal [5]. In addition to that, controllers are able to cluster many servers that host their own virtual machine manager, thus aggregating the management of several virtual machines in one single point. Public cloud platforms, on the other hand, have their own infrastructures, with powerful servers and hypervisors. They offer their virtualization structure as a service to users, usually wrapped in a Web Services API. The rationale behind choosing the three cloud platforms are presented below. 1) Xen Xen [12] is an open source solution for infrastructure virtualization that provides an abstraction layer between hardware and operating system, thus allowing the execution of many operating systems in parallel, hosted in the same physical machine. It has components that schedule the utilization of the hardware resources by the virtual machines. The Xen hypervisor supports paravirtualization [13], where the VM operating system is diverted to act as if it is interfacing directly with the hardware. This feature delivers a significant gain in VM performance compared to the full virtualization approach. Despite being an open solution, many commercial platforms use Xen (or adapted/enhanced versions of it) to power their services, such as Amazon EC2 (section III.B.3). For the experiments, version 4.0, last stable version of Xen to the moment, was used. 2) OpenNebula OpenNebula [10] is a highly centralized, flexible, and customizable cloud controller. It is an adaptable platform for building private and hybrid clouds at existing datacenters. Its interfaces and open source modules enable it to plug in other modules in a straightforward manner, allowing a flexible use of its capabilities. Because of OpenNebula’s high customization ability, the selection of its underlying modules may lead to significant variations in its performance. It has abstraction and customization features, typical of cloud controllers, and it is the most acknowledged controller for users who wish to add custom mechanisms to the cloud environment [5]. Due to a greater support and usage by the cloud computing community, version 2.2 of OpenNebula was chosen to be part of the evaluation. 3) Amazon EC2 Amazon Web Services (AWS) [8] was the public cloud selected for our experiments since Amazon is a major 3

Available at http://libvirt.org/

Infrastructure-as-a-Service (IaaS) provider and has a sufficiently satisfactory reliability4. The Elastic Compute Cloud (Amazon EC2) [9] is the AWS service that handles virtualization, an important part of the AWS platform, and permits the use of Amazon’s large computing infrastructure by the clients. The service is named Elastic because it can dynamically increase and decrease the use of resources by the users, by adding or removing virtual machine instances. The Amazon EC2 service offers pre-defined VM instance configurations to its clients. Amazon uses the concept of EC2 processing units (or ECUs), which determine the equivalent processing power of a 2007 Opteron processor or a 2007 Xeon processor with 1.0 to 1.2 GHz speed. Amazon EC2 uses an improved close version of Xen hypervisor beneath its servers. IV. METHODOLOGY In order to evaluate the performance of the platforms alone (with no concern with the complete environment, once this would lead to organizational issues) with respect to their elastic capabilities we selected five basic virtual-machine manipulation operations common to all platforms, which are described in Table I. The experiments consisted on invoking the operations in each cloud platform and measuring the time they took to execute them. This metric was selected because it is the one which mostly interfere the end user’s experience for the operations. To obtain time measures with statistical significance, 30 iterations were performed for each experiment involving the selected functions, from which the average time was computed. Error bars for a 99% confidence level are shown in the results, but only when the confidence interval is significant. TABLE 1: VIRTUALIZATION FUNCTIONS USED IN EVALUATIONS.

Function Create Teardown

Suspend Resume

Reboot

Description Constructs a virtual machine with hardware specifications defined in a configuration file. Disables the virtual server, as well as disposes of its files, programs and services. Tearing down ends virtual machine life and deallocates its reserved computer resources. Temporarily releases hardware resources assigned to a virtual server, though, data and services are only suspended, not discarded. Restores a suspended virtual machine, with the same data, running services and hardware specifications defined at creation time. Deallocates and reallocates hardware resources for a virtual machine. This function is particularly useful to prevent dilapidation caused by prolonged software usage.

The hardware of our local servers is compatible with the average configuration found in a cloud host, that is, with a virtualization-ready modern processor. In our experiments, we used four computers with the same specifications: Intel Core i7 3.2GHz processor 12GB RAM, 1 TB storage and two Gigabit Ethernet network adapters, one for local connection and one for Internet connectivity. The servers are 4 According to Cloud Harmony (http://cloudharmony.com/status), Amazon EC2 datacenter in São Paulo, assessed by our tests, has an uptime of 100%.


45

geographically separated: two of them were in Recife, northeast Brazil, and the other two were in Santo André, southeast Brazil. All of them were connected to the Brazilian academic backbone network (RNP). Since the cloud platforms evaluated work on top of Linux-based operating systems, all machines were powered by Debian Squeeze x64, a distribution known for its stability. Four different virtual machine configurations were tested. We selected VM configurations (and corresponding images) taken from the Amazon’s EC2 platform (section III.3) that could be instantiated in our local servers without exhausting their resources. The virtual machine configurations used are described in Table II. TABLE 2: VM CONFIGURATIONS USED IN EVALUATION.

Config. micro32 micro64 small32 large64

Processor 2 ECUs 2 ECUs 1 ECU 4 ECUs

Memory 613 MB 613 MB 1,7 GB 7,5 GB

OS Ubuntu 11.10 x86 Ubuntu 11.10 x64 Ubuntu 11.10 x86 Ubuntu 11.10 x64

Two evaluation experiments were performed with the cloud platforms, aiming at measuring the impact of background load in the performance of VM manipulation functions. First we tested the cloud platforms with no other VM being executed in the host machines, which provides the expected performance in the best-case scenario. Then we evaluated a most realistic scenario where server load is about to reach its maximum. For this latter scenario, VMs were instantiated in each host machine up until its resources were near exhaustion (details are provided in sections IV.A and IV.B). CPU intensive processes were executed in the background VMs, which kept CPU utilization at a maximum of the VM’s capacity. Since we cannot control the background load in the Amazon EC2 platform, this latter scenario was tested only with Xen and OpenNebula. The overall measurement script used in the experiments is show in Figure 4, where 30 time measurements are performed in sequence, for a given function, with 60 second intervals between measurements. Specific measurement details for each platform are described in the following sections.

the virtual machine list, measuring the time taken until the expected changes are observed in the list. For the scenario with a background load in the Xen servers, six VMs with 1 ECU and 500 MB of RAM running Ubuntu Server 11.10 x86 were used. In this configuration, each virtual processor is allocated to one physical processor and the physical memory is almost completely used. The Xen commands for creating, tearing down, suspending, resuming, and rebooting a virtual machine are, respectively, xm create, xm shutdown, xm save, xm restore, and xm reboot. B. OpenNebula While OpenNebula is frequently used with graphic front-ends to enhance its abstraction power (e.g., Sunstone 5), in this evaluation we used its command line interface only. In OpenNebula, each invocation triggers the execution of various processes, some from the platform itself, others belonging to the underlying Xen hypervisor. We adapted the experiment script to measure the execution time of the functions by monitoring the status of the OpenNebula servers (command onevm list). Possibly, different configurations of OpenNebula would lead to different results, and for it is a design choice, our setup was made for the sake of simplicity. The evaluation of other configurations was left for future work. The background load used with OpenNebula was the same used with the Xen hypervisor. The OpenNebula commands used in the evaluation were: onevm create, onevm shutdown, onevm suspend, onevm resume, and a combined call of onevm suspend followed by onevm resume to simulate the reboot function6 from the end-user point of view. C.Amazon EC2 For the experiments using Amazon’s EC2 public cloud we performed measurements from machines geographically closer to the EC2 datacenter in Brazil (both the evaluation machines and the EC2 datacenter are located in the Sao Paulo metropolitan area). In order to provide a fair comparison between EC2 and the local experiments with OpenNebula and Xen, the average round-trip time between the EC2 datacenter and the evaluation machines was measured and subtracted from the response times of invoking the EC2 Web Services API. The API functions used in the experiments were: ec2start-instances, ec2-terminate-instances, ec2-stop-instances, ec2-run-instances and ec2-reboot-instances.

V. EXPERIMENTAL RESULTS

Figure 3: Generic code used to evaluate function performance.

A. Xen When a Xen operation is invoked, the changes in environments are reflected in the list of virtual machines (accessed via the command xm list) after the operation is completed. The calls to the Xen operation are executed in the background, making it difficult for a direct assessment of the time taken to execute them. In this case, the script monitors 46

The first results presented in this section are for the scenario with no load in the host machines, i.e., the ones in which the VM being manipulated is present in the host. Figure 4 through Figure 6 show the time consumed by the VM manipulation operations for each platform and VM configuration tested. Please note that the time axis of all plots is in log-scale. Figure 4(a) shows that adding a Cloud controller on the top of Xen yields a significant impact on the performance of VM creation tasks. This is due to the fact that OpenNebula keeps VM images in a separate repository and makes a local copy at each 5

http://opennebula.org/documentation:archives:rel2.2:sunstone Only the most recent versions of OpenNebula implement reboot function, as it appears in libvirt. 6


operation. Although this is suitable to keep consistency, it takes additional time to process the usual tasks. Results show that Xen presents a stable behavior as creation times remain nearly constant for all experiments. CPU and memory requirements for the VM affect the creation time, as can be observed in the large64 scenario tests. In the tests of VM teardown, Figure 4(b) shows that the EC2 platform presents high variability (as denoted by the confidence interval). We performed additional measurements to make sure that results were not influenced by oscillations in the network connectivity. We concluded that such high variability is indeed a characteristic of EC2 for this type of task. One may also notice that the worst VM creation and teardown times in the EC2 platform come from the instances micro32 and micro64. A reasonable explanation for this is that the lowest cost instances might receive low allocation priority as compared to the other two more expensive configurations.

EC2 outperforms the bare Xen hypervisor should be explained by the improvements made to Amazon’s private Xen. 3,666

large64

77,576 96,437 3,459

small32

19,219 23,437

amazon xen

3,510

micro64

7,766 9,156

opennebula

4,589

micro 32

8,239 9,151 1

10

100

(a) 3,675

large64

1,363 75,606

3,809

large64

3,510

1,510

small32

78,438

1,251

small32

40,250

amazon

micro64

1,252

opennebula 7,096

xen

4,060 1,267

opennebula 27,563

4,479

micro 32

1,257 7,098

5,122

micro 32

xen

3,451

1,205

micro64

1

1,210

10

100

27,456 1

10

(b)

100

Figure 5: Response time (in seconds) for VM (a) suspend and (b) resume

(a) 3,502

large64

12,798 5,918 3,394 2,339 5,832

small32

amazon

The results in Figure 6, regarding the reboot times, reveal that OpenNebula had the worst performance as compared to the other platforms. Its performance degraded as the VM configuration required more resources. The reboot time for Amazon EC2 and Xen has low variability with respect to the VM configuration.

xen

3,738

micro64

2,333

opennebula 5,834

large64

3,507 5,926

small32

3,490 4,231

micro64

3,431 4,166

167,668

5,022

micro 32

amazon

17,022

3,618

2,332 5,832

amazon

37,848

1

10

100

(b) Figure 4: Response time (in seconds) for VM (a) create and (b) teardown

Figure 5(b) shows the results for the VM resume task. Amazon’s EC2 and Xen presented a near-constant behavior to resume VMs. In such cases, different configurations of VM have little or no impact on the VM resume times. On the other hand, the OpenNebula controller was sensitive to the VM configuration, with the measured times changing with the VM configuration used. Concerning VM suspension, Figure 5(a) shows that both OpenNebula and Xen are affected by the type of VM configuration, while Amazon EC2 is not. In such case, there is no significant variation in the suspension time for the EC2 as we change the VM configuration. The cases in which

xen opennebula 15,194

3,588 4,231

micro 32

15,127 1

10

100

1000

Figure 6: Response time (in seconds) for virtual machine reboot

For the scenario with background load in the host machines, Xen hypervisor was very sensitive to the lack of available computational resources. All its functions presented decreased performance if compared to clear resources experiments, as it is showed in Figure 7 below, for the suspend function. The VM configuration barely affected the results of the overloaded scenario: the behavior of the chart is shifted from the no-load


47

scenario to higher values in the loaded one. Therefore, the addition of background workload generated an oscillation in response times (denoted by the confidence interval bars in the chart) that was not seen in the clear resources experiment, implying that Xen’s control over the underlying hardware is affected by background workload.

152,52 77,58

large64

34,34 19,22

small32

LOAD CLEAR

15,12

micro64

7,77 14,32

micro 32

8,24 1

10

100

1000

Figure 7: Response times (in seconds) for the suspend function in no-load scenario (CLEAR) and background load (LOAD).

With regard to the OpenNebula platform, the decrease of performance in all functions happens as well, though, it is not as expressive as the Xen case. We observed the general average increase (i.e. all operations for all configurations) for Xen is of 321% while the average increase for OpenNebula is 158%. From Figure 8, one can observe that VM configuration influences the results: while more powerful configuration VMs present a small performance loss, simpler configuration VMs present a bigger performance loss. We also noted that there was a significant increase in the standard deviation of the measured times, indicating that OpenNebula stability is affected by the underlying computers’ workload – an expected behavior for applications operating on higher levels in a machine’s architecture. Due to limitation on the number of pages, only the most representative data are shown.

82,84 78,44

large64

50,63 40,25

small32

LOAD 46,54

micro64

CLEAR

27,56

52,94

micro 32

27,46 1

10

100

Figure 8: Response times (in seconds) for create function in noload scenario (CLEAR) and background load (LOAD)

VI. CONCLUSION AND FUTURE WORK Elasticity is a major feature of Cloud Computing, enabling dynamic changes in the application demand to be accommodated by the infrastructure via allocation of more resources, which can be released when no longer necessary in 48

order to keep computation costs low. This paper evaluates the performance of elasticity operations, aiming at helping organizations to select the most cost-effective option for public, private, or hybrid cloud environments. We measured the time taken to execute such operations in the Amazon EC2 public cloud and compared the results to those obtained from a private setup running the OpenNebula controller and the Xen hypervisor. Our results show that, although all platforms offer acceptable execution times for elasticity operations, technical and financial factors, as well as the kind of applications, may lead to different choices of platform. As future work, we plan to evaluate VM migration functions and other cloud platforms, as well as the performance of the platforms in differently loaded environments and the bundled invocation of the aforementioned operations. Other interesting subjects are the evaluation of the elasticity operations across cloud boundaries, i.e., using resources from different clouds (also known as cloud burst) and the performance of the platforms after aging. REFERENCES [1] Peng, J., Zhang, X., Lei, Z., Zhang, B., Zhang, W., Li, Q. , "Comparison of Several Cloud Computing Platforms", 2nd Intl. Symposium on Information Science and Engineering (ISISE), 2009, pp.23-28,December 2009. [2] Cerbelaud, D., Garg, S., Huylebroeck, J., “Opening the clouds: qualitative overview of the state-of-the-art open source VMbased cloud management platforms”, 10th ACM/IFIP/USENIX International Conference on Middleware, December 2009. [3] Dejun J., Pierre, G., Chi, C., “EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications”, NFPSLAM-SOC, November 2009. [4] Lenk, A., Menzel, M., Lipsky, J., Tai, S., Offermann, P., "What Are You Paying For? Performance Benchmarking for Infrastructure-as-a-Service Offerings," IEEE Cloud Computing (CLOUD), pp.484-491, July 2011. [5] Sempolinski, P., Thain, D., "A Comparison and Critique of Eucalyptus, OpenNebula and Nimbus", 2nd IEEE Cloud Computing Technology and Science (CloudCom), 2010, pp.417426, Febreuary 2011. [6] Armbrust, M. et al., “A View of Cloud Computing”, Communications of the ACM, 53(4), pp. 50-58, 2010. [7] Cordeiro, T. Damalio, D. Pereira, N. Endo, P. Palhares, A. Gonçalves, G. Sadok, D. Kelner, J. Melander, B. Souza, V. Mångs, J.-E., “Open Source Cloud Computing Platforms”, Grid and Cooperative Computing (GCC), 2010 9th International Conference on, pp.366-371,January 2011. [8] Amazon Web Services (2012). Available at http://aws.amazon.com. Last accessed: 11/Sep/2012 [9] Amazon Elastic Compute Cloud (2012). http://aws.amazon.com/ec2/. Last accessed: 11/Sep/2012 [10] OpenNebula Project (2012). “The Open Source Toolkit for Cloud Computing”, Available at http://opennebula.org/. [11] Ueda, Y., Nakatani, T., "Performance variations of two opensource cloud platforms", Workload Characterization (IISWC), 2010 IEEE International Symposium on, pp.1-10, December 2010. [12] Citrix Systems (2010) “Xen Hypervisor”, http://xen.org/products/xenhyp.html. Last accessed: 3/Aug/2012 [13] Barham P., Dragovic B., Fraser K., Hand S., Harris T., Ho A., Neugubauer R., Pratt I., Warfiled A., “Xen and the art of virtualization”, SOSP '03 19th ACM symposium on Operating systems principles, December 2003.
