Clients. Data center hosting. K context-aware applications. Data center hosting. K context-aware applications. Data center hosting. K context-aware applications.
Dynamic request allocation and scheduling for context aware applications subject to a percentile response time SLA in a distributed cloud Keerthana Boloor∗ , Rada Chirkova? , Tiia Salo and Yannis Viniotis∗ ∗ Department
of Electrical and Computer Engineering of Computer Science North Carolina State University
? Department
IBM Software Group Research Triangle Park
Cloudcom 2010, Indianapolis, Indiana, USA
1 / 17
Agenda
Agenda
Problem description Dynamic request allocation and scheduling scheme Comparison with static allocation and FIFO/Weighted Round Robin scheduling scheme Conclusion
Cloudcom 2010, Indianapolis, Indiana, USA
2 / 17
Problem description
Problem description
More web applications are designed to be context aware. Most context aware applications are built on SOA principles. Cloud computing systems - the most preferred platform for deployment. Service Level Agreements (SLA) - terms of service and pricing model. What is this presentation about? Cloudcom 2010, Indianapolis, Indiana, USA
3 / 17
Problem description
Geographically distributed cloud computing system
Geographically distributed cloud computing system
Data center hosting K context-aware applications
Data center hosting K context-aware applications
Clients
Data center hosting K context-aware applications
Data center hosting K context-aware applications
Cloudcom 2010, Indianapolis, Indiana, USA
4 / 17
Problem description
Context aware applications
SOA based context aware application
2. Client request allocated to and scheduled at end-server
3. Load required service-endpoint
Gateway
4. Load required contextdata
Contextaware SOA applications
Contextdata stores End servers
DATA CENTER
Internet 1. Client request with context-id
Updates to contexts at contextdata stores
Cloudcom 2010, Indianapolis, Indiana, USA
5 / 17
Problem description
Model of an end-server
An end-server serving multiple user classes
Class 1
Server ‘j’ at data center ‘i’
Class 2
Class K
Each context aware application services multiple classes of users Each user class is guaranteed different quality of service based on economic considerations SLA specifies different service levels and service charges for the different user classes Cloudcom 2010, Indianapolis, Indiana, USA
6 / 17
Problem description
Percentile Service Level Agreements
Percentile Service Level Agreements
Profit
P
0
X
100
Conformance(%)
X % - the fraction of requests of a particular user class which need to have a response time less than r seconds $P - The profit charged by the cloud, if the percentile of requests that have response time less than r seconds is greater than or equal to X % Cloudcom 2010, Indianapolis, Indiana, USA
7 / 17
Problem description
Problem statement
Problem statement Allocate and schedule service requests locally at the end-servers so as to globally:
max
X
profitj
(1)
1≤j≤K
where profitj is the profit charged for conformance of the requests from users of class j.
Cloudcom 2010, Indianapolis, Indiana, USA
8 / 17
Problem description
Problem statement
Problem statement Allocate and schedule service requests locally at the end-servers so as to globally:
max
X
profitj
(1)
1≤j≤K
where profitj is the profit charged for conformance of the requests from users of class j. This problem is NP-hard!!
Cloudcom 2010, Indianapolis, Indiana, USA
8 / 17
Solution
Management scheme description
Heuristic-based data-oriented request management scheme Periodic allocation and adaptation at each datacenter.
Allocation phase
Allocation phase
Adaptation phase
Allocation phase
Adaptation phase
Allocation phase
Adaptation phase
Allocation phase
subinterval Adaptation Adaptation phase phase
Observation interval (T)
00:00
06:00
Cloudcom 2010, Indianapolis, Indiana, USA
9 / 17
Solution
Management scheme description
Heuristic-based data-oriented request management scheme Periodic allocation and adaptation at each datacenter.
Allocation phase
Allocation phase
Adaptation phase
Allocation phase
Adaptation phase
Allocation phase
Adaptation phase
Allocation phase
subinterval Adaptation Adaptation phase phase
Observation interval (T)
00:00 Adaptation phase
06:00
Datacenters exchange conformance levels. Allocation phase Rank-based request allocation and gi-FIFO scheduling. Aim at increasing global profit. Cloudcom 2010, Indianapolis, Indiana, USA
9 / 17
Solution
Rank-based allocation and gi-FIFO scheduling
Rank-based allocation and gi-FIFO scheduling
Profit-score calculation
Required global conformance: ck Current global conformance: cck If cck < ck Profit-score = pk /(ck − cck ) Else Profit-score = 0
Profit−score assigned to each arriving request of class 1 ($)
Profit: pk Class 1 SLA − Profit of 2000$ on conformance of 75% 2000
1500
1000
500
0 0
10
20
30 40 50 60 70 Current conformance of class 1 (%)
80
90
100
Cloudcom 2010, Indianapolis, Indiana, USA
10 / 17
Solution
Rank-based allocation and gi-FIFO scheduling
Rank-based request allocation 1
Query hash-based lookup table ([context-id,machine-id] or [service-id,machine-id])
2
Rank-based compatibility test
3
1
The arriving request is assigned a rank based on its profit-score and deadline.
2
Does the arriving request meet its deadline? - Machine compatible!!!
Compatible machine not found? - Choose least loaded closest to context DB
Cloudcom 2010, Indianapolis, Indiana, USA
11 / 17
Solution
Rank-based allocation and gi-FIFO scheduling
gi-FIFO scheduling Choose the request of user class with the highest current profit-score
Choose one with maximum waiting time but which results in a response time less than or equal to r If no such request exists, choose the request with higher waiting time resulting in a response time greater than r
gi-FIFO has been proven to be the most suitable for percentile SLAs for a single server serving multiple classes.
Cloudcom 2010, Indianapolis, Indiana, USA
12 / 17
Evaluation
Evaluation Dynamic scheme vs static schemes 11000 10000 9000 Dynamic rank based allocation with gi−FIFO scheduling
8000 Profit incurred ($)
Static allocation with WRR scheduling 7000
Static allocation with FIFO scheduling
6000 5000 4000 3000 2000 1000 0
5
10
15
20
25 30 Request rate
35
40
45
50
Cloudcom 2010, Indianapolis, Indiana, USA
13 / 17
Evaluation
Dynamic rank based allocation vs static allocation scheme
11000 Static allocation with gi−FIFO scheduling
10000
Dynamic rank based allocation with gi−FIFO scheduling
9000
Profit incurred ($)
8000 7000 6000 5000 4000 3000 2000 1000 0
0
50
100
150
Request rate
Cloudcom 2010, Indianapolis, Indiana, USA
14 / 17
Evaluation
Variation in subinterval length
Variation in context update interval
18000
18000 16000
16000 Uniform distribution of classes, stringent SLA Uniform distribution of classes, relaxed SLA Non−uniform distribution of classes, stringent SLA Non−uniform distribution of classes, relaxed SLA
12000
14000
Profit obtained ($)
Profit obtained($)
14000
10000 8000 6000
12000 10000
Medium Contextdata load times
4000
2000
2000
0
50
100
150
200 250 300 Subinterval period
350
400
450
0
500
High contextdata load times
6000
4000
0
Low contextdata load times
8000
0
20
40
60
80 100 120 140 Contextdata update interval
160
180
200
Cloudcom 2010, Indianapolis, Indiana, USA
15 / 17
Conclusion
Conclusion
Identified the need for dynamic request scheduling and allocation for context aware applications in a distributed cloud.
Proposed a novel rank-based request allocation and gi-FIFO scheduling scheme for managing percentile SLAs with an aim to maximize profit obtained by the cloud.
Cloudcom 2010, Indianapolis, Indiana, USA
16 / 17
Questions??
Cloudcom 2010, Indianapolis, Indiana, USA
17 / 17
