Provisioning for Large Scale Loss Network Systems with Applications in Cloud Computing Yue Tan
Yingdong Lu
Cathy H. Xia
The Ohio State University
IBM Thomas J. Watson Research Center
The Ohio State University
[email protected]
1.
[email protected]
MOTIVATION
Provisioning for large loss networks is a classic problem in performance, due to the fat that loss network is an important mathematical model for many applications, notably those in telephony. Lately, loss network models are utilized and extended to provide performance analysis and control for exciting new applications in statistical physics [3], workforce management [9] and cloud computing [4]. In these new studies, a loss network often serves as a crucial element in characterizing system dynamics and producing calculation for vital performance metrics. This can be seen from the application in resource provision for cloud computing. Cloud computing is rapidly gaining momentum as a new paradigm for offering computing as services via the Internet. Service provider usually offers a menu of service instances, which require the commitment of distinct resources(CPU, Memory,etc) at various amounts. Along with purchase of these instances, service level agreements (SLA) will specify the desired targets on various performance metrics that the service provider should meet. A common performance metric is service availability, defined as the percentage of time at which new service requests can be admitted into the system with their desired amount of resources fulfilled. Violation of the SLAs typically results in significant penalty. The objective of resource provisioning is to seek the balance between the resource costs and SLA penalty so that service availability can be guaranteed efficiently. We develop an integrated optimization framework to search for the optimal resource provision with SLA constraints. First, we develop a Markovian model to capture users’ flexibility on upgrade/downgrade services on demand and characterize the steady-state behavior of the offered load. Then, the multi-class multi-resource provisioning problem can be naturally mapped to a stochastic loss network model, and SLA constraints are mapped to constraints on loss probabilities. Based on Kelly’s approach for capacity planning in a loss network [6, 7, 8], we propose an optimization framework to determine resource levels that minimize the combined costs of resource and violation penalty. Since computing the exact loss probabilities is a ]P complete problem thus prohibitive for large service loads, we consider the Erlang fixed-point approximation for the blocking probabilities that has been proven to be asymptotically exact in the limiting regime as the traffic intensities and the resource capacities grow together in proportion [10, 6, 7]. We further
[email protected]
improve it by replacing the single-dimension Erlang B formula by its upper bound in [5]. We show that this improved optimization problem also converges to the original problem under proper scaling, and demonstrate by a numerical example that it yields improved provisioning solutions with better SLA guarantees. Note that although this paper is presented in the context of cloud computing, the methodologies developed are readily applicable to a broad range of other applications in which loss network models are suitable.
2. 2.1
MODEL AND ANALYSIS Base Model
Suppose that there are multiple classes of customers, labeled by r, each following an independent Poisson process with rate λr . To capture the flexibility for customers to increase/decrease resource requirements on the fly, we consider the following probabilistic model. Assume that, after holding the service template r for a random amount of time, a class r customer upgrades/downgrades to class r0 with probability prr0 and terminates the service with probP 0 ability 1 − R r 0 =1 prr . There are J types of resources, and suppose that resource j has capacity Cj units, j = 1, . . . , J. The rth class customers uses Ajr units of resource j, where Ajr ∈ Z+ . Let R be the set of all possible classes. A customer requesting contract r is blocked and lost if for any resource j, j = 1, 2, . . . , J, there are fewer than Ajr units free. Otherwise this customer is served and simultaneously holds Ajr units of resource j, for all j = 1, 2, . . . , J, for the holding period of the contract r. The contract holding period is independent of earlier arrival times and holding periods; holding period of contract r is arbitrarily distributed with mean 1/µr , r ∈ R. For class r, there is an SLA that the service will be available with probability 1 − r .
2.2
Erlang Loss Network Model
Under the above assumptions, this network can be modeled as an Erlang loss network [8]. Let nr be the number of customers in service using contract r, and let n = (nr , r ∈ R), C = (C1 , C2 , . . . , CJ ). Then, it is well known, see e.g. [8], n has a unique stationary distribution given by π(n) = G(C)−1
Y νrnr , nr !
n ∈ S (C)
r∈R
Copyright is held by author/owner(s).
where νr = λr /µr , S (C) = {n ∈ ZR + : An ≤ C} and G(C)
is the normalizing constant (or partition function) X Y νrnr . G(C) = nr ! n∈S (C) r∈R
Denote Lr as the probability that a customer requesting class r is lost, that is, the blocking probability. The exact formula is Lr = 1 − G(C)−1 G(C − Aer ), where er is the unit vector corresponding to single active customer in class r. The widely adapted Erlang fixed point approximation gives the following calculation for Lr . Y 1 − Lr ' (1 − Ej )Ajr r ∈ R.
Recall that, the following bounds are derived in [5], √ −1 Φ(α) C 2 + , (3) E(ρ, C) ≤ φ(α) 3 p where α = sgn(1 − Cρ ) −2C(1 − Cρ + log Cρ ). We will use this upper bound to replace the Erlang B formula (1). The result is the following optimization problem, (Pˆ )
J X
min
ˆj wj C
s.t.
ˆj = U (ρj , C ˆj ), E ˆj ) ρj = (1 − E
j
ˆr = 1−L
ν /C! , (1) E(ν, C) = PC n n=0 ν /n! P Q and ρj is given by ρj = r:j∈r Ajr νr i∈r−{j} (1 − Ej )Air . P It is as if each of the j Ajr units of resources requested by a customer for contract r is granted or denied independently, with each request for a unit of resource j being denied with blocking probability Ej . [8] has shown that if capacities Cj , j = 1, 2, . . . , J, and offered traffics νr , r ∈ R, increase proportionally, the above approximation will coincide with the exact loss probability eventually. And this property is called asymptotic exactness of the Erlang fixed point approximation.
2.4
min
J X
min
(P˜N ) min
J X
wj CjN
s.t.
w j Cj
r ∈ R.
˜jN wj C
j=1
s.t.
Lr ≤ r ,
˜N 1−L r =
r ∈ R.
∀j ∀j
i
Y (1 − EjN )Ajr ≥ 1 − r − N ,
∀r
j
(PˆN ) min
J X
ˆjN wj C
j=1
ˆjN − U (ˆ ˆN s.t. |E ρN j , Cj )| < N , X Y ˆiN )Air , ˆ N −1 (1 − E ρˆN Ajr νr j = (1 − Ej ) r
˜j wj C
(2)
ˆN 1−L r =
∀j ∀j
i
Y ˆjN )Ajr ≥ 1 − r − N , (1 − E
∀r
j
˜j = E(ρj , C ˜j ), E ˜j ) ρj = (1 − E
−1
∀j X r
˜r = 1−L
LN r ≤ r + N ,
˜jN − E(˜ ˜N s.t. |E ρN j , Cj )| < N , X Y ˜ N −1 ˜iN )Air , ρ˜N Ajr νr (1 − E j = (1 − Ej )
j=1
s.t.
∀r
For each N ∈ Z+ , rate of arrival for each route in N -th system satisfies, νrN = N νr , r ∈ R and CjN = N Cj for all j = 1, . . . , J. The intuition is that when the demand arrival is accelerated, the capacities have to be adjusted accordingly. For each N -th system, we have the corresponding version of problems (P ), (P˜ ) and (Pˆ ) with a square root relaxation.
r
Here the vector L = (Lr , r ∈ R) represents the loss probabilities for all the requesting contracts. Let us denote the optimal solution to problem (P ) as C∗ , and the corresponding blocking probability as vector L∗ . Replacing the blocking probabilities by their Erlang fixed point approximations, we have, (P˜ )
Y ˆj )Ajr ≥ 1 − r , (1 − E
Asymptotic Optimality Analysis
j=1
J X
∀j
i
j=1
The following optimization formulation aims to provide the most efficient capacity allocation under SLA constraints. The objective is a weighted summation of capacities for all the resources. The weight reflects the cost of maintaining such capacity. This formulation is very flexible. Extra constraints such as those on budget and dependence relations between different capacities can be easily added. min
Y ˆi )Air , Ajr νr (1 − E
ˆ ∗, E ˆ ∗, L ˆ ∗ ). The optimal solution to (Pˆ ) is denoted as (C
(PN )
Optimization
(P )
∀j X
j
C
J X
−1
r
where Ej = E(ρj , Cj ), for all j = 1, 2, . . . , J is calculated by the well-known Erlang B formula:
2.3
(4)
j=1
Y
Ajr νr
Y
˜i ) (1 − E
Air
,
∀j
i
˜j )Ajr ≥ 1 − r , (1 − E
where N = O(N −1/2 ). Let us denote these problems as ˜ ∗N , E ˜ ∗N , L ˜ ∗N ) (PN ), (P˜N ) and (PˆN ), and their solutions as C∗N , (C ∗ ˆ ∗ ˆ∗ ˆ and (CN , EN , LN ), respectively.
∀r
j
˜ ∗, E ˜ ∗, L ˜ ∗ ). This We denote the optimal solution to (P˜ ) as (C is the optimization studied in [2], where some of its structural properties are obtained. Our idea is to replace E(ρj , Cj ) ˆ j , Cj ). with the square root staffing estimation upper bound E(ρ
Proposition 1. As N → ∞, we have, X X ∗ ∗ ˆN,j wj CN,j − wj C = o(N ). j
j
To prove Proposition 1, the key idea is to show that as
N → ∞, X
∗ wj CN,j −
X
∗ ˜N,j wj C −
X
j
∗ ˜N,j wj C = o(N ),
is a hard problem, we therefore use its continuous relaxation relating to the incomplete Gamma function (see, e.g. [5]).
j
X j
4
6.6
∗ ˆN,j wj C = o(N ).
x 10
Normalized Capacity for PN 2
j=1 6.59
j
Normalized Capacity for PN 1
6.58
We also have the following asymptotic optimality result:
6.57
0
20
40
60
80
100
120
140
160
180
200
40
60
80
100
120
140
160
180
200
40
60
80 100 120 Scaling Factor N
140
160
180
200
Nomalized Capacity
5
ˆ ∗N , E ˆ ∗N , L ˆ ∗N ) solves Z+ , (C 0 0
Proposition 2. If for each N ∈ ˆ ∗N , L ˆ ∗N ) converge to a vector (E , L ) as problem (PˆN ), and (E N → ∞, and there exists a capacity vector C0 that produces loss probability L, then C0 solves the original problem (P ).
1.22
x 10
j=2
1.218 1.216 1.214
0
20 6
1.343
Proof. Proof is omitted due to space limit.
x 10
j=3
1.342 1.341
3.
NUMERICAL RESULTS
1.34
In the following example from a cloud computing application, we experiment and analyze our methodology. The system under consideration is a simplified version of Amazon EC2 [1] as shown in Table 1. In this example, |R| = 3, and r = 1, 2, 3 correspond to standard small, standard large and standard extra large classes; J = 3, and j = 1, 2, 3 correspond to EC2 compute unit (1 virtual core), memory (GB) and local instance storage (10GB). And under steady Instance Type Std. Small Std. Large Std. X Large
CPU(Unit) 1 4 8
Memory(GB) 1.7 7.5 15
Storage(10GB) 16 85 169
Table 1: Amazon EC2 Instance Description 6
8.675
x 10
Normalized total cost of PN 2
Normalized Total Cost
8.67
Normalized total cost of PN 1
8.665
8.66
8.65
8.645
0
50
100 Scaling Factor N
150
200
Figure 1: Normalized total cost of P1N and P2N . state offered loads are ν = (10000, 6000, 4000). Let wj denote unit cost of resource j. We assume when investing large amount of resources, costs grow in linear scale. Therefore, according the current market, w = ($100, $6, $1). Let Ajr denote number of units of resource j requested by class r as shown in Table 1. Originally, we assume Ajr ∈ Z+ , here, we extend the assumption and let Ajr ∈ R+ . Finally, QoS requirements are = (0.5%, 0.5%, 0.5%). We formulate the provisioning problem using two alternative optimization formulations: (P1N ), optimization formulation (2) in which the loss probabilities are replaced by the Erlang fixed point approximation; (P2N ), optimization formulation (4) in which the Erlang blocking formula in (P1N ) is replaced by the upper bound (3). To solve the above optimization problems, we use the interior point nonlinear optimization package (IPopt), which can solve large-scale problems efficiently. Note that solving (P1N ) with the Erlang blocking formula in its discrete format
20
Figure 2: Normalized capacity of P1N and P2N . Figure 1 shows the asymptotic results for normalized total cost as a function of N for both P1N and P2N . Observe that under both formulations, the normalized total costs eventually converge to the same limit, confirming our earlier analysis that the error in the objective value by (P2N ) is in the order of o(N ). Figure 2 plots the normalized optimal capacities (CjN /N ) as a function of N for each individual resource under the two formulations. Observe again that under both formulations, the normalized capacities for each resource also converge to the same limit, confirming that the error in the optimal provisioning solution resulted by the upper bound approach is also in the order of o(N ). Note that under all offered loads ν N , the provisioning solution derived under (P2N ) always dominates that under (P1N ), i.e. N N C2,j ≥ C1,j for all j. Therefore, the resulting per class blocking probability achieved by (P2N ) is also always better for all service classes.
4.
8.655
0
REFERENCES
[1] Amazon elastic compute cloud. http://amazon.com/ec2. [2] S. Bhadra, Y. Lu, and M. S. Squillante. Optimal capacity planning in stochastic loss networks with time-varying workloads. In Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, SIGMETRICS ’07, 227-238, New York, NY, USA, 2007. [3] R. Fernndez, P. A. Ferrari, and N. L. Garcia. Loss Network Representation of Peierls Contours. Ann. Probab. 29(2): 902-937, 2001. [4] A. Gera and C. H. Xia. Learning Curves and Stochastic Models for Pricing and Provisioning Cloud Computing Services. Service Science. Vol. 1, no. 3. : 99-109, 2011. [5] A. Janssen, J. V. Leeuwaarden, and B. Zwart. Gaussian expansions and bounds for the poisson distribution applied to the erlang b formula. Advances in Applied Probability, 40:122–143, 2008. [6] F. Kelly. Blocking probabilities in large circuit switched networks. Advances in Applied Probability, 18:473–505, 1986. [7] F. Kelly. Routing in circuit-switched networks: Optimization, shadow prices and decentralization. Advances in Applied Probability, Vol. 20(No. 1):112–144, Mar. 1988. [8] F.Kelly. Special invited paper: Loss networks. The Annals of Applied Probability, Vol.1 (No.3): 319-378. [9] Y. Lu, Y. A. Radovanovi´ c and M. Squillante. Workforce management via Stochastic Networks, Performance Evaluation Review, 2007. [10] W. Whitt. Blocking when service is required from several facilities simultaneously. AT&T Tech. J., 64:1807–1856, 1985.
