A Recommendation-as-a-Service Platform for Cloud

CloudAdvisor: A Recommendation-as-a-Service Platform for Cloud Configuration and Pricing Gueyoung Jung, Tridib Mukherjee, Shruti Kunde, Hyunjoo Kim, Naveen Sharma, Frank Goetz Xerox Research Center {gueyoung.jung, tridib.mukherjee2, shruti.kunde, hyunjoo.kim, naveen.sharma, frank.goetz}@xerox.com

Abstract—The proliferation of cloud computing can imply a barrier to cloud users. When deploying their complex workloads into clouds, cloud users are typically overwhelmed by too many technical choices. Moreover, underlying technologies and pricing mechanisms of clouds vary and are not transparent to them. Consequently, it is hard for cloud users to capture the monetary and performance implications of their workload deployments. This paper introduces a cloud recommendation platform, referred to as CloudAdvisor. It allows cloud users to explore various cloud configurations recommended based on user preferences such as budget, performance expectation, and energy saving for given workload. Then, it allows cloud users to compare offered price and performance with other clouds’ offerings for the workload. By providing transparent comparisons, it can also support cloud provider to develop a competitive pricing strategy such as price reduction driven by energy efficiency. We have applied the proposed platform for recommendation from a real data center and some external clouds. Keywords-recommendation, cloud configuration, pricing, cloud comparison

I. I NTRODUCTION As cloud computing has become popular, the number of cloud providers has proliferated in the cloud marketplace recently [1]. Those cloud providers are continuously adding new hardware and software artifacts into their cloud infrastructures with various different pricing strategies for their services. In order to achieve cost-efficiency while keeping reasonable performance, many users have started to migrate to clouds by deploying their complex workloads such as web transactions and big-data analytics into those cloud infrastructures. In this trend, cloud users typically face with a practical challenge posed by uncertainty on the potential monetary and performance achievements from their workload deployments into a cloud. One immediate barrier to cloud users is identifying the right cloud resources and software technologies (i.e., cloud configuration) for their workloads among various different cloud configurations, and figuring out the cost-efficiency from their configurations. Meanwhile, the main challenge to cloud providers is identifying competitive pricing strategies while meeting the cloud user’s performance requirements. Nevertheless, underlying technologies and pricing strategies of clouds vary, and it is not trivial for both cloud users and providers to make a right decision. Specifically, the implication of choosing a cloud configuration is not transparent to cloud users [2], [3]. Cloud interfaces are typically presented to cloud users as a number

of technical choices, where users need to input specific cloud configuration details for their workloads including resource requirements (e.g., the type and number of VMs, storage size, and required network bandwidth) and software (e.g., OS, platforms, and databases). Such technical choices are usually unknown to naive users. Hence, workload deployments are initially either over-provisioned, leading to cost overhead, or under-provisioned, impacting on performance. Given the large spectrum of cloud users, who may not be aware of the intricacies of such cloud configuration details, a complimentary simplified requisition interface with a method for estimation of cloud configuration is imperative. Such an interface can be used to further enable pricing strategy references to cloud providers allowing negotiations with cloud users. In this situation, a couple of simple methodologies can be considered, but neither of them is sufficient. First, one may find a cloud configuration for her workload using trial-andtest method by deploying it into each cloud configuration and measuring performance and price. However, this will be very expensive and time consuming since different workloads and configurations have different performance characteristics [4], and the workload deployment is typically complicated [5]. Second, one can refer to existing cloud comparison services such as CloudHarmony [1], CloudVertical [6], and ComparetheCloud [7] that provide basic comparisons of clouds to potential cloud users. However, these are rudimentary comparisons, such as which VM type has the fastest CPU, disk IO, or memory sub-system separately [1], [7] or providing prices of different VM types [6]. This information is not sufficient for cloud users who try to deploy complex workloads that typically need multiple VMs. This is because resource subsystems are usually inter-dependent to deal with a workload. Some sub-systems can be bottlenecks for certain amount of loads, and the bottleneck is migrated between resource subsystems as load changes [8]. Additionally, some tradeoffs between user preference metrics such as performance, budget, and energy consumption should be considered for given workloads, but it is difficult to address these tradeoffs only using such rudimentary comparisons. In this paper, we introduce CloudAdvisor as a Recommendation-as-a-Service platform designed to recommend an optimal cloud configuration for given user workload and preferences. It allows cloud users to explore different optimal cloud configurations and corresponding prices recommended based on user preferences such as a

Cloud Configuration CloudAdvisor

Applications Add-ons Platform

Business users IT staff

Virtualization

Price, Performance, Discount, Configuration

Fig. 1.

Infrastructure

Optimization Comparison

Workload Info, Performance, Budget, Greenpoint

Interactive and transparent recommendation and comparison

maximum budget, a throughput expectation, the amount of input load estimated, and a willingness to save energy. For given user workload, it also allows cloud users to compare offered configuration and price with other clouds’ offerings automatically derived using benchmarking-based approximations. Finally, a cloud provider can leverage the transparent comparison for the decision making process on pricing strategy. In order to perform initial evaluation, we have applied CloudAdvisor to one of our internal private data centers and some external public clouds.

Usage

Load

Cost

- Resource Availability - HW & SW dependencies - Workload modeling - Cloud modeling

Performance Greenpoint

Constraint Optimization

Cloud Configuration Details

Compare to Other Cloud Providers Price Performance Price&/&Day&

II. P LATFORM OVERVIEW CloudAdvisor enables interactive and transparent ways for cloud users to explore optimal cloud configurations and comparisons with different clouds based on high-level user preferences. As shown in Figure 1, user preferences can be presented through simple interface where users do not need to enter specific cloud configuration details but provide preferred requisition on workload, performance requirement, budget, and energy level (i.e., Greenpoint, see Section III-A) for building cloud configurations. These preferences are usually well understood to end users such as business analysts and application developers, since metrics used for these preferences (e.g., throughput and cost) are based on servicelevel objectives or service-level agreement (SLA). Moreover, CloudAdvisor supports users to figure out the relation between preferences (metrics) and their goals (e.g., price) by allowing them to interactively explore various different preferences and corresponding results. This is different from existing interfaces used in other cloud providers (e.g., Amazon EC2 and Windows Azure) that typically need configuration details from users. CloudAdvisor can estimate a near optimal cloud configuration with price, performance, and some discount driven by energy-efficiency based on user preferences. These outputs are estimated by a constraint optimization solving method [9] (see Section III-B) that takes into account user’s preferences, resource availability, and dependency of proper hardware and software. Specifically, we adopt a best-first graph search method (based on A* search algorithm [10]) to solve the constraint optimization problem. To provide a concrete Recommendation-as-a-Service to users, CloudAdvisor is also designed for comparing different clouds for user workload and preferences as shown in Figure 1. When a user requests a comparison, it computes near optimal prices and performances that can be offered by other cloud

Optimization Comparison

Fig. 2.

Discount&

Performance&

Our&Offer&

$"12.00""

$"2.00"

2030"/"sec"

Cloud&1&+&2&

$"12.50"

N/A"

2002"/"sec"

Cloud&1&

$"18.20"

N/A"

2240"/"sec"

Cloud&2&

$"27.00"

$"1.50"

2550"/"sec"

Inputs and outputs of CloudAdvisor

providers. Specifically, we adopt a benchmarking-based approximation technique to estimate performance capabilities of target clouds and then, convert the optimization problem into a Knapsack problem [11] with target performance and price. It solves the problem using a greedy algorithm (see Section IV-B). Finally, when user wants to deploy the recommended cloud configuration, CloudAdvisor builds the configuration and then, deploys it into the cloud. This deployment component will be integrated into CloudAdvisor in future. III. I NTERACTIVE E XPLORATION AND R ECOMMENDATION CloudAdvisor provides a user-friendly interface where users do not need to enter specific cloud configuration details but only provide five preferences as shown in Figure 2. It integrates a constraint optimization solving method (i.e., Constraint Optimization module in Figure 2) that precisely estimates detailed cloud configuration and provides corresponding price, performance, and discount as outputs. All these preferences and the constraint optimization method will be described in this section. Optimization Comparison module in the figure will be described in the next section. A. High-level User Preferences CloudAdvisor asks specific user preferences that depend on user workload. User can choose any combination consisting of usage, load, budget (cost), performance, and Greenpoint levels that will guide our system to recommend cloud configuration. As illustrated in Figure 2, user can choose a combination by moving five sliders, and all these values of the combination are abstracted. This abstraction can transparently explain some tradeoff between performance, cost, and energy efficiency for

Discount Estimator

g (Greenpoint)

Greenpoint Estimator

f (cloud_conf.)

Energy demand

cloud users. For example, if user chooses Greenpoint and Cost (i.e., budget) with values, 3 and 4 for Greenpoint and Cost respectively as shown in figure, our system considers “Greenpoint = 3” and “Cost = 4” as constraints and then, attempts to maximize performance level under constraints. Using performance estimations (see Section IV-A), power models, and available resources reported by monitoring systems, CloudAdvisor computes the maximum and minimum actual values (e.g., the maximum throughput and energy consumption of the cloud system) and coverts these values into abstracted values (i.e., levels). User can also change the usage level (i.e., how many hours a cloud configuration will be used per day) and load level (i.e., how much load into the cloud configuration she expects for a certain interval) that directly affect the performance and the cost. 1) Performance and Cost Levels: CloudAdvisor first computes the fraction of available resources with respect to the total allowed resources. Then, it comes up with similar fraction on the performance of available resources. Multiplying these two fractions gives a composite fraction on the performance characteristics of the given workload. It then discretizes the fraction in 10 distinct ranges (i.e., performance level of 1 represent 0 to 0.1 whereas performance level 10 represent 0.9 to 1 of the composite fraction). As an example, when a workload will use 1000 highest-end VMs out of total 1000 VMs and when a highest-end VM’s execution time is 95% with respect to all other types of VMs, then the composite fraction for the workload can be obtained as (1000/1000)0.95 = 0.95, which falls in the range of 0.9 to 1 and hence, the performance level is 10. Similarly, when the service uses 1 low-end VM out of 1000 VMs and when the low-end VM has an execution time of 1% with respect to all other types of VMs, then the performance level becomes 1. Cost levels can be computed with similar way based on resource usage prices (e.g., the price tag of each VM type). 2) Greenpoint: Energy consumption is one of major cost factors for cloud providers. Existing research efforts principally attempt to reduce energy consumption by resource management in virtualization and underlying infrastructure [12], [13], [14], [15]. We instead focus on the service layer to improve transparency on energy demands of user workload. CloudAdvisor allows cloud users to provide their willingness of energy savings as Greenpoint. Greenpoint is a userunderstandable abstract form to describe energy demand level for user workload deployment. CloudAdvisor then converts it into cost saving of cloud provider and offers incentives through dynamic discounts to users for a green configuration selection with acceptable performance level. It can also maximize the profit of cloud provider while providing such discounts as described in [16]. Figure 3 depicts the computation of Greenpoint and discount. Greenpoint Estimator estimates the relative energy consumption level (i.e., Greenpoint) of a workload based on cloud configuration and the power characteristics of the resources. Similar with the performance level, the principal idea is capturing the relative energy consumption of configuration

GreenPoint1

GreenPoint5 Performance Level

Power management system

Fig. 3.

Greenpoint and discount estimations

with respect to all the other workloads and then, apply discretization on the relative energy consumption. The discretized ranges of values are referred to as Greenpoint. To estimate the energy consumption of cloud configuration, we use a linear energy consumption model [17] that shows the energy consumption linearly increases as resource usage increases for a given workload. Such linear model can be obtained with an experimental environment with initial sample data for the given workload. Alternatively, we can also obtain such coarse grained estimates for different servers and VMs from existing benchmark profiles (e.g., available in http://www.spec.org/ and [18] from VMs). Discount Estimator then estimates the discount for user workload with estimated profit from a pricing mechanism of cloud provider that can be formulated with individual users’ resource demands and Greenpoints [16]. B. Optimal Cloud Configuration Internally, CloudAdvisor can map user preferences into corresponding cloud configuration. It can also map a selected performance level to the corresponding Greenpoint (or budget level) and vice versa through the cloud configuration mapping. These mappings depend on some dependencies of underlying available resources at different layers including infrastructure, virtualization, and SW layers. To compute an optimal cloud configuration based on given preferences, CloudAdvisor also considers tradeoffs between Greenpoint, performance and budget levels. We cast this problem into a constraint optimization problem [9]. Our constraint optimization solver basically searches cloud configuration to meet user preferences while considering SW and HW availabilities and their dependencies. 1) Constraints: Besides given user preferences, our constraint optimization solver first determines a set of constraint variables and the discrete value space of each variable that can impact on the cloud configuration. It includes system specific variables such as VM types, virtualization types, guest OSs, and platforms offered by the cloud provider and infrastructure specific variables such as locations and available HW resources used in each location. Then, the value space of each variable is determined. For example, Location = {“Dallas”, “P ittsburgh”, “KualaLumpur”} is set based on our data center locations. V M T ype = {“small”, “medium”, “large”} is set depending on CPU, memory, disk, and network capacities, V M M ix is a set of VM instances instantiated from V M T ype, and GuestOS = {“ubuntu”, “centos”, “windows”} is set based on available

Start

Pruned out Available option Candidate option

1 2

1-D

2-D

Dallas

1-I

1-GP

3 U

X

2-LV

Pittsburgh

1-D

6

Kuala Lumpur

1-I

2-I

C

WS

9

7 C

4

5

WS

K

E

3-LV

1-MV

1-HV

2-HV

1-LV

2-LV

12

3-LV

U

Server Cluster Type

2-GP

10 U

11

82-MV

1-GP

C WS

Virtualization Type VM Mix

X

K

E

1-MV

2-MV

1-HV

1-LV

Location

Specialization

OSs for VM instances. Based on variables and their respective value sets, there are certain rule sets that impose constraints on what values the variables can feasibly have in a cloud configuration to be built. For variables representing user preferences, rule sets represent user selections on these variables. For variables pertaining to system and infrastructure specific parameters, rule sets represent the dependencies of HW (e.g., server clusters) on specific location, virtualization technology on specific HW, and SW on guest OSs and platforms as shown in Figure 4. For rule sets representing dependencies, we use a notation convention A → B, where B is the dependent variable and A is the variable on which the dependent variable depends on. The notation denotes the dependency relationship as “A implies B”. Apart from these, there are rule sets that capture the availability of resources. For example, the number of VM instances is dependent on the number of HW resources available and VM types. The following list provides example rule sets in detail. Note that one can extend these dependencies based on her or his cloud infrastructure. HW dependencies: This includes any dependency between HW and location (i.e., situations where in a particular data center, a certain server cluster is used). The general form of the dependency is HW Dependence(location): [location ∈ Location] → [hw ∈ HW ], where location is a specific value in the value space of variable Location, and hw is the subset of value space of variable HW . For example, if a couple of server clusters are deployed at the “Dallas” data center, then the rule set for HW dependency can be HW Dependence(“Dallas”): [“Dallas”] → [{“cluster(5, 40, 160, 160)”, “cluster(8, 128, 512, 640)”}], where numbers in parenthesis represent total resource capacities (e.g., the number of available servers, CPU cores, memory size, disk size) in the data center. OS and Virtualization dependencies: This includes dependencies of guest OS and hypervisor types on the server cluster and can be denoted generally as [vm ∈ V M T ype] → [os ∈ GuestOS] and [hw ∈ HW ] → [h ∈ V irtualizationT ype], following similar convention as mentioned in HW dependency. For example, if “ubuntu” and “windows” are the GuestOS, all VM types are applicable, and “KVM” and “Xen” are hypervisors deployed on a server cluster, the corresponding OS dependency can be given as [{“small”, “medium”, “large”}] → [{“ubuntu”, “windows”}] and virtualization dependency is as [“cluster(5, 40, 160, 160)”] → [{“KV M ”, “Xen”}]. Resource availabilities and VM dependency: This represents the type and number of VM instances that can be hosted in a server cluster. It is derived from the availability of CPU cores and memory size in the server cluster as [vms ∈ V M M ix] → [hw ∈ HW ]. CloudAdvisor generates various combinations of VM types offline as templates that can be packed into the server cluster based on the cumulated capacities of VM instances. For example, if a cluster with 3 servers, 24 cores and 96 GB RAM is available in “Dallas” data center and the small VM type (2 cores and 1GB RAM), the medium (2 cores, 4GB RAM), and the large (4 cores, 16 GB

Guest OS

2-HV

13

H

M

D

GP

Lib/ Platform/ Middleware

End

Greenpoint (or performance level)

Fig. 4.

An example dependencies and search

RAM) are applicable to this server cluster, then some possible templates are 12, 12, and 6 homogeneous setups respectively using VM instances from the same VM types. These rule sets can be updated once the availability of the server cluster changes. This template generation can be an overhead, but we have observed that the availability is not changed often in our server clusters so that an offline process works in real world environments. CloudAdvisor encodes all these dependency and availability constraints into a dependency graph as shown in Figure 4. The dependency graph is represented as a fat hierarchical tree with the specialization degree (depth) and sorted values by performance level, Greenpoint, or price level (width). 2) Constraints Optimization Solver: Given dependency graph, we cast the constraints optimization problem into a graph search problem and then, adopt A* graph search algorithm [10] with admissible heuristic Greenpoint, budget, or performance level. The search for configuration starts from location and then goes deep (i.e., more specialized) based on dependency rule set (i.e., first it decides the best location and then best server cluster in the location and so on). Figure 4 illustrates our search approach when user provides a performance level preference for her workload. In this case, the search aims to maximize Greenpoint (i.e., minimize energy consumption) while meeting the given performance level. At start point (i.e., “Start” in the figure), our algorithm generates all possible locations (e.g., “Dallas”, “Pittsburgh”, and “Kuala Lumpur” in the figure), and applies dependency rule set (e.g., “performance level = 8”). In this figure, it prunes out “Kuala Lumpur” because the location cannot meet “performance level = 8” even though we utilize all currently available resources in the location. The algorithm records the possible options (e.g., “Dallas” and “Pittsburgh”) in a queue. Then, it determines which location has the maximal Greenpoint by sorting the options with admissible heuristic Greenpoint values. Each admissible heuristic Greenpoint value indicates the approximated best Greenpoint at the current node (e.g., current location). In other words, it indicates the approximated shortest path from the current node to “End” node. In Figure 4, our algorithm chooses “Dallas” as the next starting point for

Generate candidate nodes at the next specialization step Prune nodes using dependency rules from candidate nodes Prune nodes, which cannot meet the given performance level

No candidate nodes No

Exit and start Yes the search with less performance level

Compute Greenpoint of each node (admissible heuristic Greenpoint) and insert the node into queue

Characterize the performance of target workload

Benchmark VMs of target clouds

Generate the base capability vector (a knapsack)

Generate target capability vectors of VMs

Pack knapsack with cumulative capability values of the cheapest combination of VMs No

Knapsack is full & have minimum price Yes

Return the cloud configuration (combined VMs) with the minimum price

Sort nodes in queue with Greenpoint Choose the node having the maximal Greenpoint from queue

Fig. 6. No

Fig. 5.

Selected node is “End”

Yes

Exit

Search procedure for the maximal Greenpoint

the search because its admissible heuristic Greenpoint value is the current best. Then, it generates all possible server cluster types in the location for further search with pruning nodes until it reaches a certain point (e.g., step 4 in the figure). The algorithm stores still “Pittsburgh” in the queue as a potential candidate. When this node is selected in the sorted queue, it starts the search back from Location specialization level (e.g., step 5 in the figure). This is because it turns out the “Pittsburgh” is possibly better than the current option. The search is over once it reaches “End” since the current path is possibly better than the best efforts in other options by the definition admissible heuristic Greenpoint. Finally, it returns the path as a specialized configuration and price based on resources to be used in the configuration for the workload. Figure 5 summarizes our constraints optimization solver. When a Greenpoint is given, the goal is to maximize the performance level (i.e., maximize application throughput) while meeting Greenpoint. We can estimate the maximal performance level at each chosen node at each specialization step as admissible heuristic performance level. Then, we can apply our algorithm as described in this section with different pruning and sorting criteria. Similarly, when a budget level is given, our algorithm can deal with the tradeoff between performance level and budget level using our constraints optimization solver. Due to the limitation of space, we omit this tradeoff in this paper. IV. C LOUD C OMPARISON It can be a big challenge for cloud users to identify an optimal cloud configuration from external public clouds. This is caused by a fact that most cloud providers in the market hide their infrastructure configuration details such as resource availability, the structure of physical servers, storages, and network switches, and how to manage their VMs, but they only show a list of VM configurations and their prices [2], [3]. Hence, we should consider such clouds as black or gray boxes to the cloud users.

Identifying optimal cloud configuration of target cloud

With performance (e.g., throughput) and price obtained from aforementioned approach as goals of comparison, CloudAdvisor estimates a cloud configuration of each cloud, which meets the throughput goal with a best price, or which meets the price goal with a best throughput. We define such best price or throughput as the capability of the cloud. Figure 6 outlines the procedure of identifying an optimal cloud configuration for the best price. The overall approach is (a) characterizing the performance of the target workload in terms of resource usage pattern in the white-box testbed, (b) based on the resource usage pattern, estimating a base capability vector, each of which elements represents the required performance capability value of each resource sub-system (e.g., CPU, memory, network, and disk I/O) to meet a given target throughput, (c) capturing the performance characteristics of the target clouds using a benchmark suite, referred to as CloudMeter, (d) based on benchmarking results of target clouds, estimating a set of capability vectors, each of which represents the capability vector of a specific VM, (e) searching an optimal cloud configuration (i.e., combined VMs to run the target workload) until there is no more chance to minimize the price. The step (c) can be done as an offline batch process, and the batch process will be scheduled periodically since those benchmarking results can be dynamically changed over time. Identifying a best throughput is similar, so we omit explaining the process in this paper. A. Benchmarking-based Estimation For recommendation of a near optimal cloud configuration for given workload, the most important step is accurately estimating the throughput of each cloud configuration. Many quantification models are currently used in research prototypes and commercial products. We can adopt a queuing model for estimating the capability of cloud configuration, which relies on available resources, as [12] used it in the same domain. We can develop such queuing model in the experimental environment. However, the estimates computed by such analytical models may not be accurate due to the diversity of cloud configurations and workloads that have different performance characteristics in different infrastructures, as shown in [4].

Price

Price&/&Day&

Discount&

Performance

Performance&

$&12.00&&

$&2.00&

2030$/$sec$

Cloud&1&+&2&

$$12.50$

N/A$

2002$/$sec$

Cloud&1&

$$18.20$

N/A$

2240$/$sec$

Cloud&2&

$$27.00$

$$1.50$

2550$/$sec$

Compare to Other Cloud Providers Price

Discount&

Recommendation View

Greedy Search

Constraint Solver Template Manager Workload Modeling

Cloud Modeling

Workload Profiling

Cloud Profiling

Performance

Performance&

Our&Offer&

$$12.00$$

$$2.00$

2030&/&sec&

Cloud&1&+&2&

$$12.10$

N/A$

1995$/$sec$

Cloud&1&

$$12.30$

N/A$

1980$/$sec$

Cloud&2&

$$12.40$

$$1.50$

1650$/$sec$

Workload Simulator

CloudMeter

CloudMeter

Internal Cloud

Fig. 8. Fig. 7.

Comparison View

Infrastructure

Price&/&Day&

Recommendation Explorer

Back End

Our&Offer&

Workload Portfolio

Front End

Compare to Other Cloud Providers

External Cloud

CloudAdvisor architecture

An example comparison tables

In our recommendation system, we use a benchmarkingbased approximation technique. CloudAdvisor first builds an abstract performance model for given workload based on the resource usage pattern of the workload measured in our inhouse test-bed (i.e., a white-box environment). The resource usage pattern indicates the correlation of resource usages to load change. In our approach, we capture the change rates of each resource sub-system until the maximum capability is reached. It can approximately indicate the degree of contribution of each resource sub-system to the capability of a cloud configuration. Second, it computes relative performance scores of many different cloud configurations given from blackboxed clouds using CloudMeter. Third, it applies the collected performance scores into the abstract performance model to estimate capabilities. B. Comparison of Cloud Configurations Based on capability estimates for given workload, CloudAdvisor identifies a near optimal cloud configuration provided by each cloud. We encode the search problem into well-known Knapsack problem [11]. As shown in Figure 6, our system generates a base capability vector as a knapsack, each of which elements represents the required performance capability value of each resource sub-system to meet a given target throughput. Then, our algorithm packs the knapsack with items (i.e., capability vectors of candidate cloud configurations) and aims to minimize the cumulative values (i.e., the total price of cloud configuration). To solve the problem, we develop a greedy algorithm. Our algorithm is based on the best-first search algorithm, but considers the current bottleneck resource while packing combined configurations (i.e., VMs). That is, it considers which VM type has the best performance rate per unit price for the bottlenecked resource, where the bottleneck is determined by a resource having the largest remaining amount to be filled in the base capability vector. Once our algorithm computes such near optimal cloud con-

figurations of target clouds, resulting prices and throughputs are encoded into a comparison table as shown in Figure 7. When a user wants to compare prices of different clouds for the target throughput (the top table in Figure 7), CloudAdvisor shows which cloud can offer the best price among cloud providers while performance is similar to each other. Similarly, CloudAdvisor shows the best performance among clouds for the target price, when the user wants to compare performance of different clouds (the bottom table in the figure). V. I MPLEMENTATION This section focuses on the architecture of CloudAdvisor used to implement the approach and algorithms described in previous sections, and presents how its components interoperate. Then, we discuss the application and evaluation of CloudAdvisor in our data center with some external clouds. The back-end part of CloudAdvisor in Figure 8 computes the optimal configuration and the optimization comparison for the user workload and preferences given from the frontend part of CloudAdvisor. The Constraint Solver component computes the optimal configuration including price and performance and then, displays them in the Recommendation View. This computation is based on the workload and preferences provided by the Recommendation Explore. It also needs resource availabilities and dependencies provided by Workload Profiling and Cloud Profiling via Workload Modeling and Cloud Modeling. As mentioned in Section III-B, Constraint Solver uses some pre-defined templates via Template Manager when searching the optimal configuration. Greedy Search generates the comparison table in Comparison View using price and performance goals provided by Recommendation View, and using estimates provided by Cloud Modeling. To profile and model the workload and target clouds, we have developed CloudMeter containing a benchmark suite that is deployed into our own data center and external target clouds to be compared. CloudMeter periodically performs benchmarking of target cloud configurations (i.e., VMs) for CPU

Fig. 10. Fig. 9.

Shortcuts of user preferences

Web transactions workloads

(e.g., UnixBench, GeekBench), memory (e.g., CacheBench), disk (e.g., IOZone), and network latency and throughput. Currently, we deploy CloudMeter into one of our internal clouds located in Webster, NY, and VMs of external clouds including Windows Azure, Rackspace, and Amazon EC2. We are extending these measurements into other cloud infrastructures. We have also developed Workload Simulator to characterize the performance of user workload. It generates synthetic loads with various data access patterns (e.g., the ratio of database write over read transactions). If a historical load is available, Workload Simulator can sort and re-play the workload to give systematic stress to the target system. Cloud users interact with Recommendation Explore to provide their workloads and preferences. User provides a workload either by (a) uploading her workload into our testbed cluster and requesting a profiling to identify optimal cloud configuration and comparison or (b) selecting some representative workloads prepared by our measurement studies for web transactions, data-intensive analytics, and mobile applications that are common applications deployed in many cloud infrastructures. In the current demonstration, we provide some web transaction workloads as shown in Figure 9 that represent specific performance characteristics (e.g., database read/write ratio and the size of image files). As discussed in Section III-A, users can provide five abstracted preferences including Usage, Load, Budget, Performance, and Greenpoint levels (by moving sliders) for chosen workload. Additionally, users can provide preferences by selecting some shortcuts as shown in Figure 10 that represent pre-fixed combinations of preferences such as “Low Load and Economical” and “High Load and High Performance.” User can do a fine tuning with sliders after selecting a shortcut. Then, user can check her cloud configuration details by clicking “View Configuration” button. As shown in Figure 11, user can compare the offered configuration with external clouds by clicking “Compare to Other Vendors” button. Meanwhile, a cloud provider can refer to this comparison table when he designs his pricing strategy. For example, he can offer competitive SLA and cloud configuration through the comparison for specific workload. Based on our observation through CloudAdvisor, we can achieve this by not

Performance

Fig. 11.

Comparison with external public clouds

only reducing prices of virtual resources but also extending VM types. Although some cloud providers offer lower prices of their VMs than their competitors, the actual price and performance of a cloud configuration may not be competitive for some workloads due to the lack of variations of VMs. CloudAdvisor can guide cloud providers toward how they can make a competitive pricing strategy and what types of VMs they should set up for specific workloads. All components of CloudAdvisor have been developed with Java 7, and deployed into a VM in our internal server. This VM is configured with 4 vCPUs, 8 GB memory, and 120 GB local disk. The front-end part has been developed with Java Servlet, and run on Tomcat server. For databases used in CloudAdvisor, we have deployed a MySQL server. For computing near optimal cloud configurations in internal and external clouds, the accurate estimation is imperative. We have observed that the error rate of the benchmarking-based approximation is mostly around 10% for given workloads and cloud configurations. In some cases using extreme cloud configurations (e.g., set the maximum budget, Greenpoint, and/or performance levels), the error rate increases up to 18%. We keep tuning our models for such cases. We have also measured the optimality of the resulting cloud configurations against the results of an exhausted search method in average preference cases. It turns out that resulting cloud configurations have been a little over-provisioned, but reasonably accurate (i.e., the error rate is around 12%).

VI. R ELATED W ORK Due to the proliferation of clouds, many research efforts such as [4], [19], [20] and commercial services such as [1], [6], [7] have focused on evaluating and comparing different clouds in the context of functional and non-functional aspects to enable informed decision on which cloud can efficiently host user workload. However, they have principally focused on comparing rudimentary cloud capabilities. For example, CloudyMetrics [20], which has been shown in IEEE Services Cup, attempts to collect cloud meta-data from cloud providers. Then, users can access the information using programmatic query. The meta-data is mainly about VM prices and rudimentary benchmarking results. CloudAdvisor can use the metadata to enable recommendation of cloud configurations to suit complex workloads. Recently, some efforts [19], [21] have further offered testing methods to enable evaluation of various workloads on different clouds. However, scaling such offerings requires various different cloud modeling techniques or trial-and-test methods. CloudAdvisor fills the gap by integrating a generic methodology to perform detailed characterization of cloud capabilities for given workloads in various target clouds. Efficient allocation of VM instances in cloud infrastructure have been addressed previously to meet user demands [22], [23]. However, those approaches are typically non-transparent to users, and the tradeoff between performance, cost, and energy efficiency remain uncertain to users. CloudAdvisor makes such tradeoff transparent allowing users to explore various different cloud configurations and select one accordingly. VII. C ONCLUSION This paper has described a recommendation platform for cloud configuration, referred to as CloudAdvisor, that aims to allow cloud users to transparently capture the monetary and performance implication of their workload deployments. Then, we have showed how our platform has been applied to a real data center and some external clouds. With easyto-use interface, users do not need to enter specific configuration details but can explore various cloud configurations recommended based on her preferences requisition such as performance expectation, maximum budget, and Greenpoint for the delivery of a given workload. Using efficient constraints optimization algorithm, CloudAdvisor can address the tradeoff between these preferences and recommend near optimal cloud configurations. It has also integrated a benchmarkingbased approximation technique to characterize performances of target external cloud infrastructures for a given workload and then, allowed cloud users to compare offered price and performance with other clouds’ offerings for the workload. This is a cost-efficient way since we do not need to deploy the target application itself into all possible cloud configurations to measure performances. By transparently comparing with competitors, it can also support cloud provider to develop competitive pricing strategy such as price reduction driven by energy efficiency and VM type extensions.

Based on our experiences with cloud users, we are integrating a methodology to customize workloads provided by CloudAdvisor. In current implementation, it has various representative workloads with pre-fixed performance characteristics such as database read and write ratio. In our on-going work, we allow users to customize such characteristics. Additionally, the current implementation provides web transaction workloads. We also extend to various different workload types such as mobile applications and data-intensive analytics with various platforms such as Hadoop MapReduce and SAP HANA. R EFERENCES [1] (2013) Cloudharmony. [Online]. Available: http://cloudharmony.com/ [2] R. Buyya, J. Brokberg, and A. Goscinski, Coud Computing: Principles and Paradigms, Chapter 1. Wiley, 2011. [3] McKinsey, “Clearing the air on cloud computing,” Tech. Report, 2009. [4] D. Jayasinghe, S. Malkowski, Q. Wang, J. Li, P. Xiong, and C. Pu, “Variations in performance and scalability when migrating n-tier applications to different clouds,” in Proc. International Conference on Cloud Computing, 2011, pp. 73–80. [5] D. Jayasinghe, G. Swint, S. Malkowski, Q. Wang, J. Li, J. Park, and C. Pu, “Expertus: A generator approach to automate performance testing in IaaS clouds,” in Proc. International Conference on Cloud Computing, 2012, pp. 115–122. [6] (2013) Cloudvertical. Available: https://www.cloudvertical.com/ [7] (2013) Compare the cloud. Available: http://www.comparethecloud.net/ [8] S. Malkowski, M. Hedwig, and C. Pu, “Experimental evaluation of ntier systems: Observation and analysis of multi-bottlenecks,” in Proc. Int. Symposium on Workload Characterization, 2009, pp. 118–127. [9] R. Dechter, Constraint Processing. Morgan Kaufmann, 2003. [10] R. Dechter and J. Pearl, “Generalized best-first search strategies and the optimality of A*,” Journal of ACM, vol. 32, no. 3, pp. 505–536, 1985. [11] H. Kellerer, U. Pferschy, and D. Pisinger, Knapsack Problems. Springer, 2004. [12] G. Jung, K. Joshi, M. Hiltunen, R. Schlichting, and C. Pu, “A costsensitive adaptation engine for server consolidation of multi-tier applications,” in Proc. Int. Conference on Middleware, 2009, pp. 163–183. [13] B. Heller, S. Seetharaman, P. Mahadevan, Y. Yiakoumis, P. Sharma, S. Banerjee, and N. McKeown, “ElasticTree: saving energy in data center networks,” in Proc. of 7th USENIX Conference on Networked Systems Design and Implementation, 2010, p. 17. [14] G. Dasgupta, A. Sharma, A. Verma, A. Neogi, and R. Kothari, “Workload management for power efficiency in virtualized data centers,” Communications of ACM, vol. 54, no. 7, pp. 131–141, 2011. [15] Y. Lee and A. Y. Zomaya, “Energy efficient utilization of resources in cloud computing systems,” Journal of Supercomputing, vol. 60, 2012. [16] T. Mukherjee, G. Jung, K. Dasgupta, S. Gujar, and H. Lee, “An economic model for green cloud,” in Proc. of 10th International Workshop on Middleware for Grids, Clouds, and e-Science, 2012. [17] R. Koller, A. Verma, and A. Neogi, “WattApp: an application aware power meter for shared data centers,” in Proc. of International Conference on Autonomic Computing, 2010, pp. 31–40. [18] A. Kansal, F. Zhao, J. Liu, and N. Kothari, “Virtual machine power metering and provisioning,” in Proc. of ACM Symposium on Cloud Computing, 2010, pp. 39–50. [19] R. Calheiros, R. Ranjan, A. Beloglazov, C. De Rose, and R. Buyya, “CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms,” Software: Practice and Experience, vol. 41, pp. 23–50, 2011. [20] M. Smit, P. Pawluk, B. Simmons, and M. Litoiu, “A web service for cloud metadata,” in Proc. International Conference on Cloud Computing, 2011, pp. 404–411. [21] J. Gao, X. Bai, and W. T. Tsai, “Cloud testing- issues, challenges, needs and practice,” Int. Journal on Software Engineering, vol. 1, 2011. [22] G. Jung, K. Joshi, M. Hiltunen, R. Schlichting, and C. Pu, “Generating adaptation policies for multi-tier applications in consolidated server environments,” in Proc. ICAC, 2008, pp. 23–32. [23] M. Cardosa, A. Singh, H. Pucha, and A. Chandra, “Exploiting spatiotemporal tradeoffs for energy-aware MapReduce in the cloud,” in Proc. International Conference on Cloud Computing), 2011, pp. 251–258.