A Game-theoretic Model for Dynamic Pricing and Competition Among Cloud Providers Tram Truong-Huu, Chen-Khong Tham Department of Electrical & Computer Engineering National University of Singapore, Singapore Email:
[email protected],
[email protected]
Abstract—Having received significant attention in the industry, the cloud market is growing rapidly with many cloud providers who compete against each other for both existing and new cloud users. Users therefore easily change their provider from one to the other depending on the obtained benefit value. To keep existing users and attract newcomers, it is crucial for each provider to offer an optimal price policy which maximizes the final revenue and improves the competitive advantage. The competition among providers leads to the evolution of the market and dynamic cloud resource prices over time and space. In this paper, we address the competition among cloud providers and propose a dynamic pricing scheme. We employ a discrete choice model to describe the user’s behavior. The model is used to derive the probability of a user choosing to be served by a certain provider. The competition among cloud providers is formulated as a non-cooperative stochastic game where the players are cloud providers who act by proposing the price policy simultaneously. The game is modelled as a Markov Decision Process whose solution is a Markov Perfect Equilibrium. Numerical simulations are carried out to evaluate the performance and assess the validity of the proposed model. Index Terms—Cloud computing, dynamic pricing, price competition, game theory, Markov Decision Process
I. I NTRODUCTION During the past few years, cloud computing has received significant investments in the industry. Many cloud providers are participating in the market, forming a competitive environment which is referred to as multi-user and multi-provider cloud market. Hereafter, we will use the terms “providers” and “users” to reflect to the cloud actors. Providers may differentiate themselves with different usage prices and capacities for each type of resources. Since the amount of resources in a user’s request is much smaller than the total capacity of a provider, the user’s request may be satisfied by any provider. A rational user will choose the provider whose resources best satisfy his computational needs, and usage price does not exceed his budgetary constraint. The user’s satisfaction can be evaluated through a utility measure which depends not only on the resource properties (i.e., capacity and usage price) but also on the user’s preference to choose certain providers, i.e. two providers with the same resource capacities and usage price may be considered different for a user due to the user’s choice behavior and loyalty. In an attempt to keep existing users and attract newcomers to improve the revenue, providers need to price their resources optimally by taking into account a wide range of factors including preferences of
users, resource capacities and the potential competition from other providers. Thus, modeling such a dynamic optimization problem becomes a rising need of providers. However, it is not a straightforward task due to the complex characteristics of the market. In this paper, we propose a game-theoretic model to describe the competition among providers. A stochastic programming approach has been adopted to cope with the dynamic evolution of the market. The contributions of this paper are twofold. First, on the user’s side, we employ a widely used discrete choice model, the multinomial logit model [1], to describe the user’s choice behavior based on the utility value obtained by using cloud resources. We assume that the user’s utility is decomposed into two parts: the first one is calculated based on resource capacities and prices, and it is observed by providers; the second part represents the user’s preference to providers to whom it is unknown. From the utility function, we derive the probability of a user choosing to be served by a certain provider. Providers in the non-cooperative game then use the choice probabilities to determine the optimal resource prices. Second, on the provider’s side, we formulate the competition among providers as a non-cooperative stochastic game. In this game, each provider tries to price its resources optimally regarding the price policies of other competitors such that the final revenue is maximized. The game is modelled as a Markov Decision Process (MDP) whose state space is finite and computed by the distribution of users among providers. At each step of the game, providers simultaneously propose their price policies such that their revenues are maximized. Based on those price policies, users will decide which provider they will select to request resources. This also determines whether the market will move to a new state or not. The Markov Perfect Equilibrium (MPE) is considered as the solution such that none of providers can improve their revenues by unilaterally deviating from the equilibrium in the long run. The rest of the paper is organized as follows. We first introduce in Section II our assumption and the relevant models for the user’s resource request and provider’s revenue. We then describe in Section III the game formulation for competition among providers. Next, we present the algorithm for solving the stochastic game in Section IV. Section V presents the numerical simulations and analysis of results. Section VI discusses the related work, before we conclude and draw future directions of this paper in Section VII.
II. U SER ’ S DEMAND AND PROVIDER ’ S REVENUE Before presenting the models, we refer readers to Table I for all the mathematical notations used throughout this paper. TABLE I: Mathematical notations Notation N M K i j k bk λi,j pi,j Ω ω βi γ ηk,i rk,j ck,i Uk,i Pk,i Ri Vˆi
Description Number of providers in the cloud market Number of resource types supplied by each provider Number of users in the cloud market Index of cloud providers Index of resource types Index of cloud users Budgetary constraint of user k Per unit benefit of resource type j offered by provider i Per unit price of resource type j offered by provider i State space of stochastic game (state space of the market) State of the market (ω ∈ Ω) Individual state of cloud provider i Discount factor associated to the time value of money Preference of user k to provider i Number of resource instances of type j requested by user k Cost of user k needs to pay to provider i for resource usage Utility value of user k when being served by provider i Probability of user k choosing to be served by provider i Revenue of provider i Discounted sum of future revenue of provider i
A. Assumption In this paper, we focus on providers which provide Infrastructure as a Service (IaaS). Notable examples include Amazon EC2, Rackspace and Microsoft Azure. Each provider operates an IaaS cloud where users can request a number of resource instances and deploy their own computing platform. There are a total of N providers on the cloud market. These providers offer several resource types denoted by M , e.g. Amazon EC21 offers four types of resource instances classified by their capacities with four different per unit prices: small, medium, large and extra large. Depending on capacity of each resource type, providers define a fixed per unit price to charge users for resource usages. For example, for an extra large instance with 15 GB RAM, 4 vCPUs (8 ECUs) and 1690 GB of disk space, Amazon charges US$0.64 for an hour of use, while Rackspace charges US$0.90 per hour for the use of a similar virtual machine instance with 15 GB RAM, 6 vCPUs and 620 GB disk space. We denote pi = (pi,1 , pi,2 , . . . , pi,M ) as the resource prices of provider i where pi,j with j = 1, . . . , M is the per unit price of resource type j (US$/hour). We additionally define the per unit benefit λi,j of resource type j offerred by provider i to reflect the relative capacity of resource in satisfying the user’s computational needs. B. User’s demand We assume that, in total, K users distribute their resource requests among N providers. User k places a request as a bundle of resources rather than individual items which is a usual case for the cloud environment. Therefore, the request of user k is represented by a vector rk = (rk,1 , rk,2 , . . . , rk,M ) 1 Amazon
EC2: http://aws.amazon.com/ec2
where rk,j with j = 1, . . . , M is the required number of instances of resource type j. For user k, we additionally define bk as his budgetary constraint. Given the price policies of all providers, the cost that user k needs to pay for using resources offerred by provider i is defined as follows: ck,i =
M X
rk,j pi,j ,
(1)
j=1
subject to ck,i 6 bk , i = 1, . . . , N , and k = 1, . . . , K. C. Provider’s revenue At any time, the state of the market is given by ω = (ω1 , ω2 , . . . , ωK ) where ωk ∈ {1, . . . , N } is the identification or index of the provider who serves user k. Let Ω denote the state space which is the set of possible states of the market. We define a function δ : Ω×{1, . . . , N } → {0, 1}K to convert the market state to the individual state of a specific provider. Let β i denote the individual state of provider i corresponding to the market state ω, we have β i = δ(ω, i) = (βi,1 , βi,2 , . . . , βi,K ), ( 1 if ωk = i, where βi,k = 0 otherwise.
(2)
Given the individual state β i and the price policy pi , the total revenue of provider i is defined as follows: Ri (β i , pi ) = Ri (δ(ω, i), pi ) K M K X X X βi,k = rk,j pi,j = βi,k ck,i . j=1
k=1
(3)
k=1
III. S TOCHASTIC GAME FORMULATION FOR DYNAMIC PRICING COMPETITION
A. Stochastic game A stochastic game [2] is a Markov Decision Process (MDP) in a multi-player setting. Formally, it is a tuple {N, A, P, R1 , . . . , RN } where N is the number of players in the game, Ω is the state space, A = A1 × · · · × AN is the joint action space of all players, P : Ω × A × Ω → [0, 1] is the transition probability function and Ri : Ω × A × Ω → R is the reward function to player i. The game proceeds from a particular state ω ∈ Ω with all players simultaneously selecting actions a1 , . . . , aN where ai ∈ Ai . Upon the realization of the state ω 0 ∈ Ω in the next time period, the players receive the payment R1 , . . . , RN . The objective of player i in a stochastic game is to select a stationary strategy a ˆi ∈ Ai to maximize the value function (i.e., the discounted sum of future rewards). Vˆi (ω, a ˆ1 , . . . , a ˆN ) =
∞ X
γ t E Rit |ˆ a1 , . . . , a ˆN , ω0 = ω , (4)
t=0
where γ is the discount factor associated to the time value of money. It is noted that a ˆ−i = (ˆ a1 , . . . , a ˆi−1 , a ˆi+1 , . . . a ˆN ) are the strategies of players other than player i.
B. Game formulation for competition among cloud providers We formulate the competition among providers as a stochastic game where the players are cloud providers who act by proposing the price policy simultaneously. As mentioned previously, the state of the market at any time is given by ω = (ω1 , ω2 , . . . , ωK ) where ωk ∈ {1, . . . , N } is the identification or index of the provider by which user k chooses to be served. We then have the state space Ω = {φ(1), . . . , φ(N K )} where N K is the size of state space Ω and φ(l) = ω. At state ω ∈ Ω, provider i takes an action which is to propose a price policy pi = (pi,1 , pi,2 , . . . , pi,M ) to attract users sending resource requests to it in the next period. The objective of each provider is therefore to find an optimal price policy p ˆ i = (ˆ pi,1 , pˆi,2 , . . . , pˆi,M ) at all states ω that maximizes the discounted sum of future revenue. Vˆi (ω, p ˆ1 , . . . , p ˆN ) =
∞ X
PM where αk,i = j=1 rk,j λi,j is the total benefit received by PM requesting resources from provider i, ck,i = j=1 rk,j pi,j is the cost user k needs to pay for provider i, and ηk,i is the preference of user k to provider i. For simplicity of presentation, we define additional variable vk,i = αk,i − ck,i . As shown in (6), the user’s utility is decomposed into two parts: the first part, labeled vk,i , is known by provider i, and the second part, denoted by ηk,i , is the user’s preference to provider i to whom it is unknown. This unknown part is therefore treated as random variable. If we assume that random variable ηk,i is an independently, identically distributed extreme value, i.e. the distribution is also called Gumbel and type I extreme value2 , the density of ηk,i is then f (ηk,i ) = e−ηk,i e−e
−ηk,i
,
(7)
and the cumulative distribution is γ t E Rit |ˆ p1 , . . . , p ˆN , ω0 = ω ,
F (ηk,i ) = e−e
t=0
(5) where Rit is the revenue received at period t as defined in (3). The game proceeds as follows: 1) Assuming that the game is currently at the end of period t − 1. The state of the market is ω in which state of provider i is β i and state of all providers other than i is β −i , β −i is observable to provider i. 2) Given this information, all providers in the market propose a new price policy simultaneously. They can increase or decrease the price to increase the chance that users will send resource requests to them in the next period t. 3) Depending on the price policy of all providers, users then decide by which provider they will be served. The market moves to state ω 0 at period t. 4) In the period t, provider i will receive a total revenue of Rit as defined in (3). The game then repeats again from step 1 to step 4. C. User’s utility and choice probability In the current cloud market, users can easily compare prices of all providers and calculate the obtained utility before deciding to be served by a certain provider. Understanding the user’s choice behavior can help providers to strengthen their competitive advantage. Discrete choice models have been widely used to describe the user’s choice behavior and then derive the choice probability. In this paper, we employ the multinomial logit model based on its broad adoption [1]. Its formula for the choice probability takes a closed form and is readily interpretable. We define firstly the user utility function which is in the form of the logit model and then derive the choice probability from the utility-maximization behavior. Naturally, with the same amount of resources requested from a provider, the user’s utility is inversely proportional with the cost he pays for the provider, i.e. the utility decreases when the cost increases. Thus, we define the utility function Uk,i of user k to provider i according to the logit model as follows: Uk,i = αk,i − ck,i + ηk,i = vk,i + ηk,i ,
(6)
−ηk,i
.
(8)
We now derive the logit choice probabilities. The probability, denoted as Pk,i , that user k chooses to be served by provider i is Pk,i = Prob(Uk,i > Uk,i0 , ∀i0 6= i) = Prob(vk,i + ηk,i > vk,i0 + ηk,i0 , ∀i0 6= i) = Prob(η
k,i0
< ηk,i + vk,i − v
k,i0
(9)
0
, ∀i 6= i).
That means user k chooses to be served by provider i if and only if his utility is maximized when using resources offerred by provider i. If ηk,i is given, Pk,i is the cumulative distribution for each ηk,i0 , i0 6= i evaluated at ηk,i + vk,i − vk,i0 , which, according to (8), is ) −(ηk,i +vk,i −v k,i0 . (10) Pk,i = e−e Since ηk,i ’s are independent, this cumulative distribution over all i0 6= i is the product of the individual cumulative distributions: Y −(ηk,i +vk,i −v k,i0 ) . (11) Pk,i |ηk,i = e−e i0 6=i
Because ηk,i ’s are unknown to providers, so the choice probability is the integral of Pk,i |ηk,i over all values of ηk,i weighted by its density (7): Z Y −(ηk,i +ck,i −c −η 0) k,i e−ηk,i e−e k,i dηk,i . Pk,i = e−e i0 6=i
(12) Some algebraic manipulation of this integral results in a succinct, closed-form expression: evk,i Pk,i = P vk,i0 , i0 e
(13)
which is the probability of user k choosing provider i to request resources. For the detail algebra that obtains (13) from (12), we refer the reader to [1] (chapter 3). 2 Gumbel
distribution: http://en.wikipedia.org/wiki/Gumbel distribution
D. State transition probability Given the choice probability P of user k choosing to be served by provider i, Pk,i = evk,i / i0 evk,i0 , the probability of user k not choosing provider i is then 1 − Pk,i . Given the current state of the market, ω, we compute in this subsection the probability of a transition from state ω to ω 0 . According to (2), the individual state of provider i corresponding to market state ω is β i = δ(ω, i), and the target state is β 0i = δ(ω 0 , i). The weighted probability of a transition from current state β i to target state β 0i of provider i is defined as follows: qi (β 0i |β i ) = σi (β i |β 0i )
K Y
0 0 βi,k Pk,i + (1 − βi,k )(1 − Pk,i ) ,
k=1
(14) P K 0 where σi (β i |β 0i ) = exp − k=1 (βi,k − βi,k )2 is a weighting factor that places lesser weight when the sum of squared distance between state β i and β 0i is larger. The weighted probability of the market to transit from state ω to state ω 0 is then equal to q(ω 0 |ω) =
N Y
qi (β 0i |β i ),
(15)
i=1
where β i = δ(ω, i) and β 0i = δ(ω 0 , i). We now have the normalized state transition probability: q(ω 0 |ω) . P (ω 0 |ω) = X q(θ|ω)
(16)
θ∈Ω
Essentially, the formulation of this transition probability means that less weight is placed on a substantial change in the market structure. This rigidity may come from several factors such as insufficient resources to meet the new computational needs or reluctance from the users. IV. S OLVING THE STOCHASTIC GAME Once the game is formulated in Section III, we now present the algorithm for solving it. The objective is to find the optimal price policy for each provider in the presence of the users’ preferences and price policies of other providers. Solving the non-cooperative game results in a Nash equilibrium [3] from which no player can unilaterally deviate the prescribed policy to gain more revenue. In the case of a stochastic game modelled as an MDP, this equilibrium is also known as MPE. A. Dynamic algorithm for finding optimal price policies At the equilibrium of the market, the discounted sum of future revenue of provider i satisfies the Bellman equation first presented in [4]. At the end of period t − 1, when the market is at state ω, the corresponding Bellman equation is Vˆi (ω) = n h io max Ri (δ(ω, i), pi ) + γEω0 Vˆi (ω 0 )|ω, pi , p−i , pi ∈RM +
(17)
where pi is the price policy of provider i at state ω, p−i is the price policies of providers other than i, Ri (δ(ω, i), pi ) is the revenue ofhprovider i receivedi at state ω with price policy pi , and γEω0 Vˆi (ω 0 )|ω, pi , p−i is the discounted expected future revenue of provider i where the expectation is taken over the successor state ω 0 . Solving (17) gives the optimal price policy of provider i at state ω as follows p ˆ i (ω) = n h io arg max Ri (δ(ω, i), pi ) + γEω0 Vˆi (ω 0 )|ω, pi , p−i . pi ∈RM +
(18) By making explicit the expectation operator in (17) and (18), we derive a new formula for expected revenue function in (19) and the optimal price policy in (20). Each provider has its own equation versions of (19) and (20). This implies that for each provider and for each initial state, maximization is performed with respect to a non-linear equation with M variables pi,j , with i = 1, . . . , N and j = 1, . . . , M . In total, there are N × |Ω| = N × N K = N K+1 non-linear equations, where N is the number of providers, K is the number of users and N K is the size of state space. The best response of all these non-linear equations must satisfy the users’ budgetary constraints (i.e., ck,i 6 bk , i = 1, . . . , N and k = 1, . . . , K). Two well-known algorithms to solve the maximization of systems of non-linear equations are Gauss-Jacobi and GaussSeidel [5], [6] which are both iterative-based methods. The main difference between them is the way to update the value and price policy in each iteration. We refer the reader to [7] for a more detailed discussion of both methods. In this paper, we advocate the Gauss-Seidel method as it has the advantage that information is used as soon as it becomes available. The full algorithm can be seen in Algorithm 1. It is noted that, during the execution of while loop (i.e., lines 4–13 in Algorithm 1), whenever new values Vˆit (ω) and pti (ω) are available, the values are immediately used to replace the old values Vˆit−1 (ω), pt−1 (ω) and in other updates within the i same iteration. In lines 6 and 7, the cc variable is defined as the convergence criterion, it is compared to a small number for stopping the algorithm. B. Avoiding the curse of dimensionality The key difficulty of the model is the computation of the expectation over all successor states from a current state as shown in (19) and (20). As the size of state space is N K where N is the number of providers and K is the number of users, it will increase exponentially when N or K increases. The problem becomes very severe when applying this model to today’s cloud market with many thousands of users. To cope with the explosion in the number of successor states, we assume that the changes in the price policies of providers cause the state transitions that are restricted to going one level up, one level down or staying the same, i.e. providers will gain, lose one user or stay with the same set of users,
( Vˆi (ω) = max
Ri (δ(ω, i), pi ) + γ
pi ∈RM +
X
) 0 ˆ P (ω |ω, pi , p−i )Vi (ω ) 0
(19)
ω 0 ∈Ω
(
)
p ˆ i (ω) = arg max Ri (δ(ω, i), pi ) + γ pi ∈RM +
X
P (ω |ω, pi , p−i )Vˆi (ω ) 0
0
)
( Vˆit (ω) = max
pi ∈RM +
(20)
ω 0 ∈Ω
Rit (δ(ω, i), pti )
+γ
X
P (ω
0
ˆ t−1 (ω 0 ) |ω, pti , pt−1 −i )Vi
(21)
ω 0 ∈Ω
)
( p ˆ ti (ω)
= arg max pi ∈RM +
Rit (δ(ω, i), pti )
+γ
X
P (ω
0
then the expectation consists of only K + 1 successor states. If there are a total of N providers, the total number of states is N (K +1). Based on this assumption, Fig. 1 shows an example of the new successor state set generated from a current state in the market with 2 providers and 4 users. Applying this method to Algorithm 1, the while loop (i.e., lines 4–13) needs to be slightly modified such that from current state ω ∈ Ω, the algorithm first generates the set of successor states, and then uses this new set of successor states to update the value Vˆit (ω) and the price policy pti (ω) for each provider according to (21) and (22), respectively.
(22)
ω 0 ∈Ω
Algorithm 1 Dynamic algorithm for finding optimal price policies p ˆ Input: Users’ resource requests and budgetary constraints: rk and bk , k = 1, . . . , K. Per unit benefit of resources: λi,j , i = 1, . . . , N, j = 1, . . . , M . Discount factor: γ. Output: Optimal price policies: p ˆ ˆ 0 (ω) ∈ R 1: Make initial guesses for the value function V i 0 M and the price policy pi (ω) ∈ R+ for each provider i = 1, . . . , N in each state ω ∈ Ω. Pick a random state considered as the initial state of the market. 2: stop ← 0 /*stop condition*/ 3: t ← 0 /*iteration index*/ 4: while stop 6= 1 do 5: Update the value Vˆit (ω) and the price policy pti (ω) for all providers according to (21) and (22), respectively. In (21) and (22), pt−1 −i refers to the price policies of providers other than provider i at iteration t − 1. Vˆ t (ω) − Vˆ t−1 (ω) 6: cc ← max ω∈Ω 1 + Vˆ t (ω) 7: if cc < then /*satisfied by all providers*/ 8: stop ← 1 9: else 10: Compute the next state of the market based on the new price policies of all providers and users’ utility functions. 11: t←t+1 12: end if 13: end while
t−1 ˆ t−1 |ω, pti , p−i )Vi (ω 0 )
2221 2212 2222
2122 1222
Fig. 1: Successor states generated from a current state using the proposed method. The current state is 2222, which means all users choose to be served by provider 2, there will be 5 successor states in which users can stay unchanged or switch to provider 1. V. N UMERICAL SIMULATIONS AND RESULTS In this section, we present the numerical simulations and analyze the obtained results to evaluate the performance of the proposed approach. A. Simulation parameter settings Simulations were performed on a Dell Optiplex 780 machine with a 2-processor Intel Core 2 Duo running at 3.16 GHz with 4 GB RAM. Implementation of the algorithm was done in MATLAB3 version 2011b with KNITRO4 libraries for the non-linear optimization. For all simulations, we set the discount factor γ = 0.95 which corresponds to a 5% interest rate, the convergence constraint = 1e−4 . The users’ preferences which follow the Gumbel distribution are generated with the location parameter µ = 3 and the scale parameter β = 4. We set the number of resource types offered by each provider M = 4 which corresponds to the number of VM instances offered by Amazon EC2. We also use the per unit prices offered by Amazon EC2 to calculate the budgetary constraint for all users. All simulations were performed with N = 2 providers whose resources generate per unit benefit λ1 = (0.1, 0.3, 0.4, 0.8) and λ2 = (0.15, 0.25, 0.5, 0.7), respectively. The number of users is set to K = 10 for all simulations, except explicitly indicating. The users’ resource requests are generated randomly. The number of instances of each resource type in a user’s request is normally distributed in the range of [10, 25]. We performed the following simulations to evaluate the performance of the proposed model: 3 MATLAB: 4 KNITRO:
http://www.mathworks.com/products/matlab http://www.ziena.com/knitro.htm
TABLE II: Prices policies and total revenue of providers when market is monopoly and oligopoly
Monopoly Oligopoly
Type 1 0.4464 0.2184 0.4030
Price policies Type 2 Type 3 0.3132 0.3772 0.2884 0.3041 0.3030 0.3438
Type 4 1.1699 1.4663 1.2437
Total revenue 427.5530 176.1984 252.6584
1) The first simulation shows the difference of providers’ revenues and resource prices in cases of the monopoly and oligopoly market; 2) The second simulation illustrates the evolution of the market to the equilibrium state; 3) The third simulation shows the convergence of the price policies and revenues of providers; and 4) The last simulation measures the execution time of the proposed algorithm.
TABLE III: State space of the game for the scenario of 4 users and 2 providers Index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Market state (ω) 1111 1112 1121 1122 1211 1212 1221 1222 2111 2112 2121 2122 2211 2212 2221 2222
Provider 1 (β 1 ) 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000
Provider 2 (β 2 ) 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111
TABLE IV: Users’ preferences to cloud providers B. Analysis of results 1) Monopoly versus oligopoly: We performed the first simulation to evaluate the revenues and price policies of providers when the market is monopoly and oligopoly (with 2 providers). With the same number of users and the same resource requests, it is expected that the revenue of provider in the monopoly market is much higher than that in the oligopoly market since users do not have any choice in the monopoly market. When competition is present in the market, users’ requests are distributed among providers. Consequently, the total revenues of providers decrease. In any case, the price policies are optimal since providers try to maximize their revenue while still assuring that usage costs do not exceed the users’ budgetary constraints. As we can observe in Table II, the sum of revenues of all providers in the case of the oligopoly market is equal to the revenue of provider in the case of the monopoly market. 2) Evolution of the market to the equilibrium: In the second simulation, we demonstrate the evolution of the market when providers change their price policies. The results prove the existence of the MPE in a stochastic game modelled as an MDP. We performed the simulation with K = 4 users. This generates a total of 16 states in the market as presented in Table III. The second column is the market state ω = (ω1 , ω2 , ω3 , ω4 ), the third and the fourth are the corresponding states of the two providers, respectively. The users’ preferences and resource requests are presented in Tables IV and V, respectively. The results show that there always exists a unique MPE no matter what initial state is set. Indeed, we picked randomly any initial state among of 16 states in the state space, the game always converges to the equilibrium at state ω = (2, 1, 2, 1) as illustrated in Fig. 2. At the equilibrium state, the optimal price policies of two providers are p1 = (0.185, 0.010, 0.010, 2.192) and p2 = (0.010, 0.029, 1.842, 0.010) with the optimal revenues R1 = $92.427 and R2 = $78.833, respectively. This simulation also shows that the algorithm converges quickly to the equilibrium state just after one iteration though the price policies of
User index (k) 1 2 3 4
User’s preference (ηk,i ) to Provider 1 Provider 2 9.6924 14.3505 10.0355 -0.1229 6.5898 6.3442 15.8084 4.2463
TABLE V: Users’ resource requests and budgetary constraints rk,1 11 24 17 22
User index 1 2 3 4
rk,2 10 14 20 12
rk,3 20 16 22 20
p1 p2
2222
rk,4 16 14 15 24
bk 37.4038 35.4311 41.4287 62.2080
p1 =(0.056,0.435,0.283,1.665) p2 =(0.010,0.490,0.010,2.012)
p1 =(0.185,0.010,0.010,2.192) p2 =(0.010,0.029,1.842,0.010)
R 1 : $87.437 R 2 : $77.779
p1 p2 2121 R 1 : $92.427 R 2 : $78.833
p1 p2
R 1 : $87.922 R 2 : $73.490 2122
1122
p1 p2
p1 =(0.051,0.459,0.237,1.713) p2 =(0.010,0.245,0.304,1.708)
Fig. 2: State transition for the case of 4 users and 2 providers. Dotted arrows show the deviation from the optimal price policies which lead to an unequilibrium state. providers are not optimal yet. After the first iteration, the algorithm continues to repeat to achieve the convergence criterion with = 1e−4 but the market always stays at the equilibrium state. By this simulation, we confirm the natural psychology of users that prices have an important impact on the users’ choices. Normally, providers will not increase prices too much to keep their loyal users. However, if users are indifferent to providers, they will change provider easily. As shown in Table IV, the third user prefers the first provider to the
350
1
Discounted sum of future revenue
Price of each resource type
1.2 Type 1 Type 2 Type 3 Type 4
0.8 0.6 0.4 0.2 0
0
1
2
3
4
5
6
7
8
300 250 200 150 100 50 0
9
Provider 1 Provider 2 0
1
2
3
Iteration index
Fig. 3: Price convergence of the first provider.
5
6
7
8
9
Fig. 5: Convergence of discounted sum of future revenues.
1.2
1600 1400
Type 1 Type 2 Type 3 Type 4
1 0.8
Execution time (s)
Price of each resource type
4
Iteration index
0.6 0.4
1200 1000 800 600 400 200
0.2
0
0 0
1
2
3
4 5 6 Iteration index
7
8
9
Fig. 4: Price convergence of the second provider. other, i.e. the preference of the third user is 6.5898 to the first provider versus 6.3442 to the second provider. However, the price policy of the second provider makes the utility of the third user attain the maximal value. Thus, the third user changes his choice to request resources from the second provider instead of the first one. 3) Convergence of price and revenue: We present the convergence of the price policy of the first provider in Fig. 3, and for the second provider in Fig. 4, respectively. Fig. 5 shows the convergence of the discounted sum of future revenue of both providers. By comparing the price of the same resource type offered by the two providers, it is expected that the price of the resource type with higher benefit will be normally more expensive. However, if users are more familiar with the provider having lower benefit resource, this provider can increase the price of this resource type to improve his revenue. In this simulation, resource type 1 of provider 1 has lower benefit than resource type 1 of provider 2 (i.e., λ1,1 = 0.1 versus λ2,1 = 0.15), but it is more expensive (i.e, p1,1 = 0.26 versus p2,1 = 0.01). Similarly, the same remark applies for resource type 4 with λ1,4 = 0.8, λ2,4 = 0.7, p1,4 = 0.62 and p2,4 = 0.69. 4) Scalability of the model: We show in this simulation the ability of the model to scale to a realistic size of the market within an acceptable time limit. We run the simulation with 8 providers which corresponds to the number of biggest public providers in the current market. We increase the number of users up to 1024. For each value of the number of users, we execute the algorithm 10 times with different users’
1
2
3
4
5
6
7
8
9
10
Number of users (2x)
Fig. 6: Timespan of the algorithm w.r.t the number of users. preferences and resource requests. Fig. 6 depicts the increase in the execution time with respect to the number of users. It is expected that the execution time of the algorithm increases when the number of users increases. However, despite the exponential increase in the state space size of the market, the proposed method of avoiding the curse of dimensionality reduces significantly the number of successor states that the algorithm has to compute the transition probability from a current state. With 1024 users, the improved algorithm finishes the execution after 1306.534 ± 222.657 seconds, while we could not run with the original algorithm. We also tried to run the original algorithm with 20 users, it finishes the execution after 10730.3 ± 2575.349 seconds. The standard deviation of the execution time is relatively high due to the convergence speed of the Bellman equation for each provider. The scalability of our model would be even better if the implementation of the algorithm is converted into a parallel program where each execution thread will process for each provider in the game. VI. R ELATED WORK Dynamic pricing is an important issue for any commercial firm in an oligopoly market. With significant growth in the past few years, the cloud market is nowadays becoming an oligopoly with many providers. Dynamic pricing in cloud has consequently gained considerable attention from both industry and academia. Amazon EC2 has introduced a “spot pricing” feature5 for its instances where the spot price is dynamically 5 Amazon
EC2 Spot Instances: http://aws.amazon.com/ec2/spot-instances/
adjusted to reflect the equilibrium prices that arises from resource demand and supply. In [8], [9], the authors presented their studies on dynamic pricing for cloud resources. While [8] studied the pricing of cloud resources for a monopoly provider operating an IaaS cloud with a fixed capacity, [9] focussed on the pricing of an oligopoly market. However, all of these works made the assumption that the user’s demand is a concave function with respect to prices. The amount of cloud resources requested decreases when resource prices increase. This assumption is not practical when users have a processing deadline or architectural requirements for their execution platform. Users have to request the required amount of cloud resources no matter what prices are. Auction-based mechanisms have also been used to determine optimal pricing in which users can consider their budgetary and deadline constraints [10], [11]. However, these models considered the pricing of only one provider. In contrast, we consider the realistic case of the current cloud market with multiple providers. In addition, users may have their preferences in choosing to be served by a particular provider. Game theory has been widely applied in economic studies for dynamic pricing competition [12], [13], [14]. In grid and cloud computing, game theory has been applied to study different issues: scheduling and resource allocation [15], [16], [17], dynamic pricing [18], [19] and revenue optimization [20], [21]. In [17], the authors presented a scheduling algorithm based on game theory. The algorithm tries to allocate available resources to users’ tasks to guarantee QoS constraints. The obtained assignment is a Nash equilibrium. Based on game theory, another resource allocation algorithm has been proposed in [15]. The algorithm minimizes the energy consumption while guaranteeing the processing deadline and architectural requirement. In [16], a user-oriented job allocation scheme has been formulated as a non-cooperative game. The algorithm tries to minimize expected cost for executing users’ tasks. The solution is a Nash equilibrium which is obtained using a distributed algorithm. However, none of existing works considered the user’s choice behavior, although some of them assume that resources are offerred by different owners. VII. C ONCLUSION AND FUTURE WORKS From revenue management and marketing aspects, multiple cloud providers are facing the question of what prices they should set for resources to attract a common pool of potential users while maximizing their revenue in the presence of other competitors. We presented a game theoretic approach to solve the dynamic pricing problem. We integrated the discrete choice model which describes the user’s choice behavior based on the user’s utility. The discrete choice model allows providers to derive the probability of being chosen by a user. By formulating the stochastic game as an MDP, the optimal price policies of providers always converge to an MPE from which providers cannot unilaterally deviate to improve their revenue. The algorithm, which computes the optimal price policies, is shown to converge quickly. This makes it practical to be adopted by actual cloud providers. For future work, we will
consider a more complex structure of the cloud market where providers can cooperate with each other to form a competitive alliance. The problem will be then the coalition formation and revenue sharing among alliance members. ACKNOWLEDGMENT Authors would like to acknowledge the funding support of the project entitled “Towards Designing Flexible, CostEffective and Secured Service Provisioning Strategies for Heterogeneous Data Centers in a Cloud–of–Clouds Infrastructure”, Grant No. 1122804009, under the Thematic Programme on Future Data Centre Technology of A*STAR, Singapore. R EFERENCES [1] K. E. Train, “Discrete Choice Methods with Simulation,” Identity, vol. 18, no. 3, pp. 273–383, 2003. [2] J. Filar and K. Vrieze, Competitive Markov Decision Processes. New York, NY, USA: Springer-Verlag New York, Inc., 1996. [3] M. J. Osborne, An Introduction to Game Theory. Oxford University Press, 2004. [4] B. Richard, Dynamic Programming. New Jersey, USA: Princeton University Press, 1957. [5] U. Doraszelski and K. L. Judd, “Avoiding the curse of dimensionality in dynamic stochastic games,” Quantitative Economics, vol. 3, no. 1, pp. 53–93, 2012. [6] U. Doraszelski and A. Pakes, “A Framework for Applied Dynamic Analysis in IO,” in Handbook of Industrial Organization, ser. Handbooks in Economics, M. Armstrong and R. Porter, Eds. Elsevier, 2007, vol. 3, no. 06, ch. 30, pp. 1887–1966. [7] K. L. Judd, Numerical Methods in Economics. The MIT Press, 1998, vol. 1. [8] D. Niu, C. Feng, and B. Li, “Pricing Cloud Bandwidth Reservations under Demand Uncertainty,” in SIGMETRICS’12, London, UK, Jun. 2012, pp. 151–162. [9] H. Xu and B. Li, “Maximizing Revenue with Dynamic Cloud Pricing: The Infinite Horizon Case,” in ICC 2012, Ottawa, Canada, Jun. 2012. [10] F. Teng and F. Magoul`es, “Resource Pricing and Equilibrium Allocation Policy in Cloud Computing,” in CIT 2010, Bradford, UK, Jun. 2010. [11] M. Mihailescu and Y. M. Teo, “On Economic and ComputationalEfficient Resource Pricing in Large Distributed Systems,” in CCGrid 2010, Melbourne, Australia, May 2010, pp. 838–843. [12] G. Allon and I. Gurvich, “Pricing and Dimensioning Competing LargeScale Service Providers,” Manufacturing Service Operations Management, vol. 12, no. 3, pp. 449–469, 2010. [13] D. Bergemann and J. V¨alim¨aki, “Dynamic price competition,” Journal of Economic Theory, vol. 127, no. 1, pp. 232–263, 2006. [14] K. Y. Lin and S. Y. Sibdari, “Dynamic price competition with discrete customer choices,” European Journal Of Operational Research, vol. 197, no. 3, pp. 969–980, 2009. [15] S. U. Khan and I. Ahmad, “A Cooperative Game Theoretical Technique for Joint Optimization of Energy Consumption and Response Time in Computational Grids,” IEEE Transactions on Parallel and Distributed Systems, vol. 20, no. 3, pp. 346–360, 2009. [16] S. Penmatsa and A. T. Chronopoulos, “Price-based user-optimal job allocation scheme for grid systems,” in 20th IEEE International Parallel Distributed Processing Symposium, Rhodes Island, Greece, April 2006. [17] G. Wei, A. V. Vasilakos, and N. Xiong, “Scheduling Parallel Cloud Computing Services: An Evolutional Game,” in ICISE 2009, Nanjing, China, December 2009, pp. 376–379. [18] Z. Kong, B. Tuffin, Y.-K. Kwok, and J. Wang, “Analysis of Duopoly Price Competition Between WLAN Providers,” in 2009 IEEE International Conference on Communications, Dresden, Germany, June 2009. [19] D. Ma and J. Huang, “The Pricing Model of Cloud Computing Services,” in ICEC’12, Singapore, August 2012, pp. 263–269. [20] D. Niyato, A. V. Vasilakos, and Z. Kun, “Resource and Revenue Sharing with Coalition Formation of Cloud Providers: Game Theoretic Approach,” in CCGrid 2011, Newport Beach, USA, May 2011. [21] D. Niyato, P. Wang, E. Hossain, W. Saad, and Z. Han, “Game Theoretic Modeling of Cooperation among Service Providers in Mobile Cloud Computing Environments,” in WCNC 2012, Paris, France, April 2012.
