Elastic Cloud Services Compliance with Gustafson's ...

Proceedings of the Third International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2016) Sofia, Bulgaria Jesus Carretero, Javier Garcia Blas, Svetozar Margenov (Editors) October, 6-7, 2016

3JTUPW 4 1SPEBO 3 (VTFW . 1FUDV %#BSCPTB + &MBTUJD $MPVE4FSWJDFT$PNQMJBODFXJUI(VTUBGTPOTBOE"NEBIMT-BXT&O 1SPDFFEJOHTPGUIF5IJSE*OUFSOBUJPOBM8PSLTIPQPO4VTUBJOBCMF 6MUSBTDBMF$PNQVUJOH4ZTUFNT /&464 4PGJB #VMHBSJB QQ .BESJE6OJWFSTJEBE$BSMPT***EF.BESJE$PNQVUFS"SDIJUFDUVSF $PNNVOJDBUJPOT BOE4ZTUFNT(SPVQ "3$04

Sasko Ristov, Radu Prodan, Marjan Gusev, Dana Petcu, Jorge Barbosa

NESUS

1

IC1305

Book paper template • September 2014 • Vol. I, No. 1

Network for Sustainable Ultrascale Computing

Elastic Cloud Services Compliance with Gustafson’s and Amdahl’s Laws Sasko Ristov, Radu Prodan University of Innsbruck, Austria [email protected], [email protected]

Marjan Gusev Ss. Cyril and Methodius University, Skopje, Macedonia [email protected]

Dana Petcu West University of Timisoara, Romania [email protected]

Jorge Barbosa University of Porto, Portugal [email protected] Abstract The speedup that can be achieved with parallel and distributed architectures is limited at least by two laws: the Amdahl’s and Gustafson’s laws. The former limits the speedup to a constant value when a fixed size problem is executed on a multiprocessor, while the latter limits the speedup up to its linear value for the fixed time problems, which means that it is limited by the number of used processors. However, a superlinear speedup can be achieved (speedup greater than the number of used processors) due to insufficient memory, while, parallel and, especially distributed systems can even slowdown the execution due to the communication overhead, when compared to the sequential one. Since the cloud performance is uncertain and it can be influenced by available memory and networks, in this paper we investigate if it follows the same speedup pattern as the other traditional distributed systems. The focus is to determine how the elastic cloud services behave in the different scaled environments. We define several scaled systems and we model the corresponding performance indicators. The analysis shows that both laws limit the speedup for a specific range of the input parameters and type of scaling. Even more, the speedup in cloud systems follows the Gustafson’s extreme cases, i.e. insufficient memory and communication bound domains. Keywords Load, Distributed systems, Performance, Superlinear speedup.

I. Introduction Cloud computing has introduced a rapid change in the way of designing the architecture of today’s services from licensebased to as-a-service-based services [1]. The main driver was influenced by its multitenancy, on demand elastic resources and underlined virtualisation technology. Customers do not buy the license to own the software service, but instead they pay only for the period of its usage. In order to satisfy the customers’ demands, cloud providers offer various types

of resources, usually represented as virtual machine (VM) instances, each with specific computing, memory and storage capacity. The customers expectation is that the performance will follow the price. Due to its elasticity and the linear pay-as-you-go model, the cloud is preferred platform both for the granular and scalable algorithms, especially if they are low communicationintensive, such as scientific applications [2, 3]. Still, many applications are data-intensive, and provide a high throughput. This is a huge challenge in the cloud because the data 1

2

Elastic Cloud Services Compliance with Gustafson’s and Amdahl’s Laws


transfer between the cloud compute nodes and storage is a bottleneck [4]. Despite the additional virtualisation layer, the superlinear speedup is also reported, both for granular [5], and scalable application types [6]. However, despite all these benefits, the main challenge for the customers is whether they will get the performance proportionally to the cost. That is, whether the cloud elastic resources comply with the Amdahl’s Law [7] for the fixed size problems and with Gustafson’s Law [8] for the fixed time problems. In this paper, we model several performance indicators, to determine if both laws hold for the cloud elastic services, each in a specific region. Although one can argue that the web services are scalable and therefore will comply with the Gustafson’s law only, our analysis and taxonomy show in which scaled systems the Amdahl’s law limits the speedup. The rest of the paper is organised in several sections as follows. The speedup definitions and limits in parallel and distributed systems are described in Section II. Section III defines a taxonomy for scaled systems in cloud, in order to adapt the existing Amdahl’s and Gustafon’s laws for elastic services. According to the taxonomy, Section IV models the speeds and speedups for each scaled system for various load regions. Despite the virtualisation layer, the cloud environment can achieve even a superlinear speedup, as discussed in Section V. Section VI discusses further challenges. Finally, we conclude the paper in Section VII.

II.

Background

Parallel and distributed systems offer a powerful environment that can be utilised for two main purposes: to speed up some algorithm’s execution or to execute some big data problems. The former is useful in order to finish with execution in proper time; for example, we need today a weather forecast for tomorrow, and it is unusable to have it tomorrow. Distributed systems are used to solve a problem that cannot be even started on a single machine due to hardware limitation. Both parallel and distributed systems have more computing resources than a nominal single-machine or a single-processor system. In this paper, we will denote these systems as scaled systems. Two main laws exist in the computer architecture, or more broader in the parallel and distributed systems, which limit the speedup that can be achieved, according the algorithm’s type: Amdahl’s and Gustafson’s laws. Both laws target the speedup, but analyse it from different perspectives. Let’s analyze a scaled system with a scaling factor p. The metric for measuring the performance of a scaled computing system is the speed V ( p), which defines the amount of work 2

W ( p) performed for a period of time T ( p), as presented in (1). Another important metric is the normalised speed NV ( p), which measures the amount of work per processor per time period, as defined in (2). V ( p) =

NV ( p) =

W ( p) T ( p)

V ( p) W ( p) = p T ( p) · p

(1)

(2)

To compare the scaled with a non-scaled system, one should evaluate the speedup S( p), which is defined as a ratio of speeds of the scaled system and the best speed in the non-scaled system, as presented in (3). S( p) =

W ( p)/T ( p) V ( p) = V (1) W (1)/T (1)

(3)

The amount of work is constant for fixed-time algorithms, which transfers (3) to (4), where T (1) denotes the execution times of the best sequential algorithm, while T ( p) the execution time of the algorithm on scaled system with scaling factor p. S( p) =

T (1) T ( p)

(4)

Amdahl’s Law limits the size of the problem and limits the speedup to the value Smax ( p) = 1/s, where s is the serial part of the algorithm. As one can observe, the maximal theoretical value for the speedup is limited and does not depend on the number of processors. On the other side, Gustafson reevaluated the Amdahl’s Law by showing that a linear speedup Smax ( p) = p can be achieved if a problem is executed within a fixed time. He achieved a near linear speedup of impressive 1000, when running a problem on 1024 cores [9].

III.

A taxonomy of scaled systems

Usually both Gustafson’s and Amdahl’s laws are intended for granular algorithms, which can be divided into many independent sub-tasks and then scattered to a scaled system for execution. This section presents a taxonomy that we define for scaled systems in order to adapt both laws to be appropriate for cloud elastic services. We are using a similar approach for scalable algorithms, such as web services, with an exception that in this case, the parallelisation is usually not conducted by some API, but on the web server level.

S a s k oR i s t o v ,R a d uP r o d a n ,M a r j a nG u s e v ,D a n aP e t c u ,J o r g eB a r b o s a

3

Bookpap e rt emp l a t e•Sep t emb e r2 0 1 4•V o l .I , No .1

I I I .1 Defin i t ionandc l a s s ifi c a t iono fs c a l edsy s tem s L e tan om in a ls y s t emb eac loudsy s t emth a tpo s s e s s e sR c loudr e sou r c e sandi sload ed w i thLr equ e s t s ,a sp r e s en t ed inF ig .1a ) . On ewou ldexp e c tth enom ina lsy s t emcanhand l e Lamoun to fwo rkinasp e c ifi ct im ep e r iodu s ingRr e sou r c e s . Fo rex amp l e ,th elo adc anb er ep r e s en t eda sth enumb e ro f r equ e s t sfo rsom es e rv i c e wh i chi sho s t edinag roupo fVM s th a th av eato t a lo fRc loudcompu t ingr e sou r c e s . Ou rc l a s s ifi c a t ioni sb a s edons c a l ingbo thth er equ i r e m en t so fac loudcompu t ingsy s t emandc loudr e sou r c e s . Th e r e fo r e ,w ed efin etwos c a l ingtyp e sinc loudcompu t ing : s c a l ingth elo ad( r equ i r em en t s )ands c a l ingth e( c loudcom pu t ing )r e sou r c e s . Ad i f f e r en ts c a l ingf a c to rc anb eu s edfo r r equ i r em en t sandr e sou r c e s .W i thou tloo s ingg en e r a l i ty ,w e a s sum eth a tth er e sou r c e sc ans c a l ep> 1t im e s , wh i l eth e lo ad ,N t im e s . W ew i l lu s eth eno t a t ion xR yLtod efin eth et axonomy o fs c a l ingth ec loudsy s t em s wh e r ex, y ∈{n, s }a r eth e ind i ca to r sinf ron to fea chs ca l ingpa ram e t e r .Th esp r e s en c e ind i c a t e sth a tth eco r r e spond ingp a r am e t e ri ss c a l edandn i fi ti sno ts c a l ed .A c co rd ingtoth i sno t a t ion ,th enom in a l sy s t emi sd efin eda san on s c a l e dR e s ou r c e sn on s c a l e dL o a d, andd eno t eda s(nRnL)sy s t em . I fth ecu s tom e r sw an ttoimp rov eth ep e r fo rm an c eo fa s e rv i c eho s t edinac loudsy s t em ,th eyn e edtos ca l eth ec loud r e sou r c e s .Inc a s eo fs c a l ingth elo ad ,th e r ea r etw opo s s i b i l i t i e sfo rth ecu s tom e r :e i th e rtor e t a inth eco s t(k e epth e s am ec loudr e sou r c e s ) ,bu td eg r ad eth ep e r fo rm an c e ,o rto s ca l eth ec loudr e sou r c e sandtor e ta inth esam ep e r fo rman c e . Con s equ en t ly ,w ew i l ld efin eth r e ed i f f e r en ts c a l edsy s t em s wh enon lyon eo rbo thp a r am e t e r sa r es c a l ed w i th D efin i t ion s1 ,2 ,and3 .A l lth r e es c a l edsy s t em sa r ep r e s en t edin F ig .1b ) ,c )andd ) . D efin i t ion1(sRnLs c a l edsy s t em )Th esRnLs c a l edc l oudsy s t emd en o t e sac l ou ds y s t emw i t hs c a l edR e sou r c e snon s c a l ed Lo ad,t h a ti s ,as y s t emw i t hpt im e sm o r ec l ou dr e s ou r c e s . D efin i t ion2(nR sLs c a l edsy s t em )Th enR sLs c a l edc l oudsy s t emd en o t e sac l ou ds y s t emw i t hnon s c a l edR e sou r c e ss c a l ed Lo ad,t h a ti s ,ac l ou ds y s t emw i t h Nt im e sm o r el o ad . D efin i t ion3(sR sLs c a l edsy s t em )T h esR sLs c a l e dc l ou ds y s t emd en o t e sas y s t emw i t hs c a l edR e sou r c e ss c a l edLo ad ,t h a t i s ,as y s t emw i t hpt im e sm o r ec l ou dr e s ou r c e san dNt im e sm o r e l o a d . Th en ex tex amp l e sexp l a inth e s etyp e so fs c a l edsy s t em s . A s sum etha taw ebs e rv e rho s t edinac loudin s tan c ew i thon e CPUco r e(R= 1 )canhand l e100c l i en tr equ e s t s(L= 100 )in a c c ep tab l er e spon s et im e .A c co rd ingtoth eGu s ta f son ’ sLaw

on ew ou ldexp e c tth a tth ep e r fo rm an c ew ou ldb edoub l ed wh enth es am e1 0 0r equ e s t sa r eex e cu t edonas e rv e ru s ing r e sou r c e sw i thdoub l eth ecapa c i ty( sRnL ) . Ano th e rexamp l ei s wh enbo thth eloadandr e sou r c e sa r e s ca l ed ,tha ti s ,th eexp e c t edr e spon s et im eo f200r equ e s t sto b eex e cu t edonas e rv e rw i thdoub l edr e sou r c e sshou ldb e th esam ea sth enom ina lca s e-100r equ e s t sex e cu t edonon e CPUco r e(sR sL ) . And ,fo rnR sL,th er e spon s et im eshou ld b edoub l edi fth eloadi sin c r ea s edto2 00r equ e s t s .

I I I .2 Expe c t edpe r fo rman ceo fs ca ledc loudsy s t em s L e tPFb eafun c t ion( 5 )th a tr e tu rn sth ep e r fo rm an c ePo f asy s t em w i thsp e c ifi cr e sou r c e sRandlo ad ed w i thalo ad L.Th en ,( 6 )d efin e sth eexp e c t edp e r fo rm an c efo ra l lth r e e s ca l edsy s t em s . P= PF( R, L)

(5 )

sRnL: p· P= PF( p· R, L) ; 1 nR sL: · P= PF( R, p· L) ; p

(6 )

sR sL: P= PF( p· R, p· L) . Th i sc la s s ifi ca t iono fs ca l ingth esy s t emcanh e lpind e t e r m ina t iono fp e r fo rman c el im i t so fasy s t em .

IV . T h eoret icalanal y s i so fscaledsy s t em s compl iancew ithAmdahl’ sandGu sta f son’ slaw s Ino rd e rtoad ap tbo thl aw sfo re l a s t i cs e rv i c e s ,th i ss e c t ion in t rodu c e sth ewo rkp e rr e sou r c eand mod e l sth esp e edupfo r s ca l edsy s t em scompa r edtonom in a l(non s c a l ed )sy s t em s .

IV .1 Mode l ingthere sou r ceu t i l iza t ion Ino rd e rtofindth er e s ou r c eu t i l i z a t i onW w ed e t e rm in ehow mu chav e r ag e wo rk( lo ad ) Li ss en ttoap a r t i cu l a rr e sou r c e Randc a l cu l a t ei ta c co rd ingto( 7 )a sar a t ioo fth elo adand th enumb e ro fr e sou r c e s .Th i spa ram e t e rshow sth eav e rag e ” sp e ed ”o fp e r fo rm ingap a r t i cu l a rw o rkp e rr e sou r c e .T o s imp l i fyth eno t a t ion ,inth er em a in ingt ex tw ew i l lu s e abb r ev i a t ion som i t t ingth eRandLid en t ifi c a t ion s ,su cha s nnfo rth enRnLsy s t em . Wnn=

L ; R

(7 )

N ex t ,d i s t r ibu t ea l l Lr equ e s t sing roup s ,su chth a te a ch g roupha sRr equ e s t sto mapea chr equ e s ttoasp e c ifi ccom pu t ingr e sou r c e .Th en ,ine a cht im ep e r iod ,Rr equ e s t sw i l l 3

4

E l a s t i cC l o u dS e r v i c e sC om p l i a n c ew i t hG u s t a f s o n ’ sa n dAm d a h l ’ sL aw s

Bookpap e rt emp la t e•Sep t emb e r2 0 1 4•V o l .I , No .1

C loudS e r v i c e

L

L C l i en t s

R

C loudS e r v i c e

C loudS e r v i c e

NxL

C l i en t s

a )

C loudS e r v i c e

b )

pxR

NxL C l i en t s

C l i en t s c )

R

pxR d )

F i gu r e1 :N om in a l(n on s c a l e d )s y s t emnRnLa )andth r e ep o s s i b l es c a l ing sb )sRnL ,c )nR sLandd )sR sL

R e sour ce u t i l i z a t i on

i sob t a in edfo rav a lu eo fp/N = 2/4= 1/2 .W e mu s tno t e tha tWssandWnscanchang eth e i rp la c einF ig .2 ,d ep end ing wh e th e rth e p/N ra t ioi sg r ea t e ro fsma l l e rth an1 .

IV .2 Mode l ingthespeedup

Wnn

W sn L /R

Wn s

W s s

F i gu r e2 :R e s ou r c eu t i l i z a t i ono ft h es c a l e ds y s t em s b es ca t t e r edamongav a i lab l eRr e sou r c e s , wh i chy i e ld sto(7 ) . No t eth a tth el a s tg roup w i l lh av e L mod Rr equ e s t s ,and no ta l lth er e sou r c e sw i l lb elo ad ed w i thr equ e s t s . Byapp ly ingth eco r r e spond ingp a r am e t e r sfo rth eth r e e s c a l edsy s t em sin to( 7 ) ,th en( 8 )d efin e sth er e sou r c eu t i l i z a t ion s( exp r e s s eda sw o rkp e rr e sou r c e )Wsn,Wns,Wss, co r r e spond ingtoth esRnL ,nR sLandsR sLsy s t em s . N· L ; p· R (8 ) Ob s e rv ingth ed efin i t ion sfo ra l lfou rw o rkp e rr e sou r c e , on ec ancon c lud eth a ta l lo fth emd ep endonth er a t ioL/R, r ep r e s en t inganom ina lv a lu eo fa wo rkp e rr e sou r c e .Th e r e fo r e ,w ew i l lcon t inu ew i thanan a ly s i sth a td e t e rm in eth e imp a c to fth eL/Rr a t ioov e rth e wo rkp e rr e sou r c eandth e sp e edup . F ig .2p r e s en t sth er e sou r c eu t i l iza t ion(wo rkp e rr e sou r c e ) o fth enom in a landth eth r e es c a l edsy s t em s . Ob s e rv eth a t a l lr e sou r c eu t i l iza t ion sa r einashap eo fs ta i r s ,w i thal in ea r t r end l in e .Th es ta i r s ’e f f e c tapp ea r sdu etoroundupfun c t ion in(7 )and(8 ) . Obv iou s ly ,th er e sou r c e so fsRnLs ca l edsy s t em h av easm a l l e ramoun to fw o rk , wh i l eth eo th e rtw os c a l ed sy s t em shav e mo r e ,compa r edtoth enom ina lsy s t em .F ig .2

Wsn=

4

L ; p· R

Wns=

N· L ; R

Wss=

Ino rd e rto m e a su r eth eimp a c to fs c a l ing ,( 9 )d efin e sth e sp e edupS ,Sns,Sssfo ra l ls c a l edsy s t em s , wh encomp a r ed sn toth enom ina lon e .

Ssn=

L R L p· R

; Sns=

L R N· L R

; Sss=

L R N· L p· R

;

(9 )

F ig .3v i su a l lyp r e s en t sa l lth r e esp e edup sa safun c t ion o fth eL/Rr a t io ,a long w i thth e i rt r end l in e s .Th esp e edup Ssnshow sanin c r ea s ingt r ends ta r t ingf rom1 ,andsa tu ra t e s L → ∞. Wh uptoth es c a l ingf a c to rpwh en R en ev e rpi sa d iv i so ro fL/R,th esp e edupa ch i ev e si t sm ax im a lv a lu eo f p,r eg a rd l e s so fth eL/Rr a t iov a lu e ,a sd ep i c t ed w i thpo in t A( i·p, p) .A l though ,s e em ing ly ,i tlook sl ik eth a tth esRnL s c a l edsy s t emr e l i e sonth eGu s t a f son ’ sL aw ,i ti st ru eon ly wh enth e L/Rr a t ioi shug e .Fo rsm a l l e rL/Rr a t io ,s c a l ing th er e sou r c e sw i l lno tp rov id eag r e a t e rsp e edup , wh i chi s ex a c t lyth eAmdah l ’ sLaw . Th esp e edupSns s t a r t sf romSns = 1ands a tu r a t e si t s v a lu etoth epo in t1/N fo rg r e a t e rr a t ioL/R. Obv iou s ly , a l thoughSns< p,ands e em ing lyi ti sasub l in e a rsp e edup , inf a c tth i si sas lowdown . Th i si sexp e c t eds in c eth elo ad i sin c r e a s edcomp a r edtoth enom in a lsy s t em .A c co rd ingto F ig .2 ,th e wo rkp e rr e sou r c ei sin c r ea s ed , wh i ch w i l lr edu c e th ep e r fo rman c e .Th et rad eo f ffo rth i sp e r fo rman c esu f f e r ing i sth econ s tan tco s ttha tth ecu s tom e rshou ldpayfo rr en t ing th er e sou r c e s .Th i si sth er e sou r c eund e rp rov i s ion ingand ov e ru t i l i sa t ion . S im i l a rb eh av io rfo rsp e edupi sp r e s en tfo rth eth esR sL s c a l edsy s t em .Th eon lyd i f f e r en c ei sth a tth et r end l in es a t u r a t e stoth ev a lu eSss = p/N. L e tu sd i s cu s sabou tth e


5


Ssn

A(i*p, p) ( p, p)

Ssn Linear Trendline

p

p

Sss

Sns

p

Sns Linear Trendline

Sss Linear Trendline

p/N

1/N L/R

L/R

L/R

Figure 3: Speedup for various scaled systems as a function of L/R • Sublinear 1 < S( p) < p - similar to Gustafson’s scaled speedup; • Linear S( p) = p - maximum limited speedup according to the Gustafson’s scaled speedup; • Superlinear S( p) > p - greater performance than the limited speedup.

Figure 4: Expected speedup of a scaled system expected performance of this scales system (6) and the calculated speedup (9). If the resource scaling follows the scaling of the work, then the speedup will saturate to 1, which means that this scaled system scales ideally. However, if p > N, which means overprovisioning and underutilisation, we are getting closer to the sRnL scaled system, in the hands of Amdahl’s Law.

IV.3 Going beyond the speedup limits Although previous subsection presents the theoretical limits of the speedup in various scaled systems, several examples are reported where the speedup went beyond the limits, that is, a superlinear speedup is achieved. Ristov et al. [10] have modeled the performance behavior of services classifying five sub-domains of speedup: • Drawback 0 < S( p) < 1 - worse performance for the scaled system; • No Speedup S( p) = 1 - the new scaled system reports the same performance as unscaled;

The expected sub-domains, along with four regions of server load (underutilised, proportional, superior and saturated) are presented in Fig. 4 [10]. The first three regions are already expressed in our theoretical analysis. The superior region is of interest in this paper, and it appears because the web server with one core will enter in its saturation mode, while the scaled system is still in its normal mode. The superior region ends when the scaled system enters the saturation mode. This means that theoretical speedups of all scaled systems do not saturate to the constant value, but they will start to falling down when the ratio L/R will increase up to some level when even the scaled resources cannot handle the load in the appropriate time. However, the reported results show that this model works only for both computation-intensive and memorydemanding web services, while the computation-intensive only web services achieve a sublinear speedup, that is, those systems have four regions.

V.

Analysis of a superlinear speedup in cloud environment

Nowadays, cloud computing is being increasingly used for high-performance and high throughput applications. It allows the customers to rent, for example, 1000 processors and execute a certain task at peak times, instead of building their own data center. Since the cloud’s pricing strategy is linear, and expected speedup is also linear, it seems that customers will be charged fairly. However, there are several 5

P ro ce s s

6

T1

T2

P ro ce s s1

T3

a )

P2

P4

P3

T1 .1 T1 .2 T2 .1 T2 .2

P ro ce s s1

P roce s s2

P1

P2

Bookpap e rt emp la t e•Sep t emb e r2 0 1 4•V o l .I , No .1

a )

C1

P1

E l a s t i cC l o u dS e r v i c e sC om p l i a n c ew i t hG u s t a f s o n ’ sa n dAm d a h l ’ sL aw s

T4

P ro ce s s

VM1 P1

P ro ce s s2

b )

VM1 P1

VM1 P1

VM2 P1

T1 T2 T3 T4

T1 .1 T1 .2

T2 .1 T2 .2

C1 C2 C3 C4

C1 .1

C2 .1

C1 .2

b )

C2 .2

P4

P3 c )

VM1 VM2 P1 P1

C1

C2

c )

VM3 VM4 P1 P1

C3

C4

d )

F i gu r e5 :E x am p l eo fb )V e r t i c a l ,c )D i a g on a l ,an dd )H o r i z on t a ls c a l ingo fn om in a lr e s ou r c e sa )f o rg r anu l a ra lg o r i thm s ca s e s wh e r eth esup e r l in ea rsp e edupi sa ch i ev ed ,d e sp i t eth e v i r tu a l i s a t ionl ay e r .

V . 1 G r anu la ra lgo r i thm s Cu s tom e r scans ca l eth e i rr en t edr e sou r c e sho r izon ta l ly ,v e r t i ca l lyo rd iagona l lyinth ec loudfo rth esRnLo rsR sLs ca l ed sy s t em s .I fth eo r ig in a lconfigu r a t ion m ap son ep ro c e s sto aVMin s t an c eho s t edonap ro c e s so rw i thon eCPUco r e , a sp r e s en t edinF ig .5a ) ,th enF ig .5b ) ,c )andd )p r e s en t th eth r e epo s s ib l ec louds c a l ing s . Th eho r i zon t a ls c a l ing p r e s en t edinF ig .5d )in c r e a s e sth enumb e ro fs am e VM in s tan c e sand map sas epa ra t ep ro c e s s(w i thas ing l eth r ead ) toad i f f e r en tVMin s t an c e . Th ev e r t i c a ls c a l ingp r e s en t ed inF ig .5b )in c r e a s e sth enumb e ro fCPUco r e sp e rVM( r e s i z edVM )and m ap ss ep a r a t eth r e ad so fas ing l ep ro c e s sto ad i f f e r en tco r eonth esam eVMin s tan c e . Acomb ina t iono f th ebo ths c a l ingtyp e sy i e ld sad i agon a ls c a l ingp r e s en t ed inF ig .5c ) .T or e a l i s eth ev e r t i c a landd i agon a ls c a l ing ,th e cu s tom e rshou ldu s esom eAP Ifo rp a r a l l e l i z a t ion ,su cha s Op enMP , wh i ch w i l lc r e a t ep a r a l l e lth r e ad s . F ewp ap e r sa r er epo r t ingasup e r l in e a rsp e edupinbo th th eho r izon ta landv e r t i ca ls ca l ing . Asup e r l in ea rsp e edupi s r epo r t edfo rca ch e in t en s iv ea lgo r i thm sin[ 5]fo rth eca s eo f v e r t i ca ls ca l ing .A l thoughs equ en t ia lex e cu t ionu t i l i s e sca ch e inth es equ en t i a lex e cu t ion mo r e ,th esup e r l in e a rsp e edup c anb ea ch i ev eda l sofo rho r i zon t a ls c a l inginth ec loud , a c co rd ing[1 1] .Th eau tho r so f[ 1 2]hav ed e t e rm in edtha tth e c loudenv i ronm en tc anh and l eth ec a s e s wh enth ep rob l em s iz ecanb efi t t edinth ela s tl ev e lca ch em emo ryb e t t e rl ead ing toasup e r l in e a rsp e edup . 6

V .2 S ca lab lea lgo r i thm s F ig .6p r e s en t sth r e epo s s ib l ew ay showtos c a l ef romnom in a lsy s t emtoth esRnLs c a l edsy s t em .S im i l a r ly ,th e r ei sa ho r izon ta l ,v e r t i ca landd iagona ls ca l ing .Th e ma ind i f f e r en c e h e r ei sth en e c e s s i tyo fc loudloadba lan c e rtha tw i l ls ch edu l e th eloadamong manyend -po in tVMin s tan c e sfo rho r izon ta l andd iagona ls ca l ing . R i s tove ta l .[6]p ropo s edas c a l ab l ea r ch i t e c tu r efo re o rd e r ingsy s t emho s t edinth ec loud .Th e i rexp e r im en t sr e po r t edas ign ifi can tsup e r l in ea rsp e edupo f20fo rth eSRNL sy s t em w i thas c a l ingf a c to rp= 4an a ly z ingth er e spon s e t im e .Th esup e r l in e a r i tya l soapp e a r edfo rth eth roughpu t , i . e .th ep e r c en tag eo fr e spon s e sfo rg iv ennumb e ro fr equ e s t s p e rs e cond .

V .3 Sup e r l inea rspeedupo falo adba lan ce r D i s t r ibu t ingth elo adtos ev e r a lend -po in ts e rv e r si s mu ch e a s i e rinth ec a s eo falo adb a l an c e r , wh i ch w i l lfo rw a rdth e loadtoth es e rv e r sbyu s ingapa r t i cu la ra lgo r i thm .S in c ei ti s an ewl ay e r ,i tadd sasm a l lamoun to fd e l ay , wh i chu su a l ly d ep end sonth eload . How ev e r ,R i s tove ta l .[13]d ev e lop eda b a l an c e rw i thar eg ionth a ta ch i ev e sasup e r l in e a rsp e edup wh enu s ing mo r eend -po in ts e rv e r s . Ev en mo r e , wh eni ti su s ed w i thon lyon eend -po in tw eb s e rv e r ,th er e su l t sa r es t i l lb e t t e rth anth ec a s ew i thou ti t . Th i sapp e a r sb e c au s eo fth econn e c t ion sth a ta r eop en edto th eend -po in tw ebs e rv e r sa r em a in t a in ed w i thou top en ing an ewconn e c t ionfo re a chc l i en tr equ e s t .Th i sr edu c e sth e numb e ro fr e sou r c e sfo rc r ea t ingan ews e s s ioncomp en sa t ing th ed e layp rodu c eddu etoth eadd i t iona llay e r .

S a s k oR i s t o v ,R a d uP r o d a n ,M a r j a nG u s e v ,D a n aP e t c u ,J o r g eB a r b o s a

7

Bookpap e rt emp l a t e•Sep t emb e r2 0 1 4•V o l .I , No .1

1X4

S c a l eup (V e r t i c a l )

2x2 D i a gon a l s c a l i ng Howto s c a l e ?

4X1 S c a l eou t (Ho r i zon t a l )

1X1

F i gu r e6 :E x am p l eo fv e r t i c a l ,d i a g on a landh o r i z on t a ls c a l in go fn om in a ls y s t emf o re l a s t i cc l ou ds e rv i c e s

V I . Di scu s s ion Th i ss e c t iond i s cu s s e sfu r th e rch a l l eng e sandi s su e scon n e c t edtoth ep e r fo rm an c eo fc loude l a s t i cs e rv i c e s .

V I .1 G ranu la rands c a l inga lgo r i thm s ’s im i la r i ty A l thoughg ranu la rands ca l inga lgo r i thm ss e emtob eto ta l ly d i f f e r en t ,a sth eya r eex e cu t edind i f f e r en tenv i ronm en t ,th ey s t i l lhav es ev e ra ls im i la r i t i e s .Inca s eo fp e r s i s t en ta lgo r i thm s , mo r em emo ryi sn e ed edfo rbo tha lgo r i thmtyp e s . Th a t i s whyasub l in e a rsp e edupi sa ch i ev edfo rcompu t a t ion in t en s iv ew ebs e rv i c e s[ 1 0 ] . W ec anob s e rv eano th e rs im i l a r i ty . Th eg r anu l a ra lgo r i thm sex e cu t edont igh t lycoup l edp ro c e s so r sco r r e spond tov e r t i c a ls c a l ingfo rs c a l inga lgo r i thm s , wh i l eex e cu t edon loo s e lycoup l edp ro c e s so r stoho r i zon t a ls c a l ing . Th e s es im i l a r i t i e sc anb eu s edtou t i l i s eth ep ro so fon e a lgo r i thmtyp efo rth eo th e r , wh i chcou lda l sol e adtoa sup e r l in ea rsp e edup .Inth i scon t ex t ,w ecanfo l lowth eid ea o fT r ange ta l .[1 4] whoin t rodu c eda mod e lfo rp a r a l l e l ex e cu t iono fw ebs e rv i c e sp rom i s ingth a tbo thapp ro a ch e s c anb eu s edtog e th e r .

V I .2 Newch a l l eng e : Howtos c a l e ? Ca ch e in t en s iv eg ranu la ra lgo r i thm s , who s eda tar eu s ecom p l ex i tyi ss im i l a rw i thth ep rob l ems i z e ,w i l lb en efi tf roma b igg e rc a ch e .M anyIn t e l ’ s mu l t ip ro c e s so r su s ea m a rk e t ing t r i ckb a s edonahug eL 3sm a r tc a ch e . How ev e r ,on ec an e a s i lych e ckth a ti ti sno tsh a r edamonga l lco r e s ,bu ton ly

amongpa r to fth em .Fo rexamp l e ,6MBo fto ta l12MBca ch e i ssh a r edb e tw e ene a chg roupo ftwoco r e s .Inth i sc a s e ,th e v e r t i c a ls c a l ing w i l lu t i l i s e mo r eth el a s tl ev e lc a ch e . AMD mu l t ip ro c e s so r su su a l lyu s easm a l l e rL 3c a ch e ,bu ti ti s sha r edamonga l lco r e so fth e mu l t ip ro c e s so r .Th e r e fo r e ,d e p end ingonth ea lgo r i thm ,app rop r ia t ep ro c e s so rands ca l ing typ eshou ldb echo s enino rd e rtoa ch i ev eth eb e s tsp e edup , po t en t ia l lysup e r l in e a r . Onth eo th e rh and ,tod ay ’ sc loude l a s t i cr e sou r c e sc an a l sob es c a l edind i f f e r en tw ay s :ho r i zon t a l ly ,v e r t i c a l lyo r d iagona l ly ,ea cho f wh i chcano f f e rv a r iou sp e r fo rman c eand po s s ib i l i tyfo ra ch i ev ingasup e r l in ea rsp e edup .Th ev e r t i ca l s c a l ingp rov id e sab e t t e rsp e edup ,bu tth eho r i zon t a lo f f e r s mo r efl ex ib l es c a l ingo fr e sou r c e s , wh i chc an m in im i s eth e co s t . A l though ,u s ingalo adb a l an c e rinf ron to fth es ib l ing s ,asup e r l in ea rsp e edupcanb ea ch i ev eddu etor edu c ed numb e ro fop en edconn e c t ion s .

V I .3 Fu r the rcha l lenge s A ch i ev ingasup e r l in ea rsp e edupdo e sno tn e c e s sa r i ly m ean th a tcu s tom e r sw i l lob t a inth em ax imuma ch i ev em en t .In th e wo rkflowex e cu t ion sinpa ra l l e landd i s t r ibu t edsy s t em s , cu s tom e r su sua l lyu s eb i -ob j e c t iv eop t im iza t ion sto m in im i s e th em ak e sp anandco s t .Th e s etwop a r am e t e r sa r eoppo s i t e on etoano th e r .M in im i s ingth e mak e spanp rodu c e sag r ea t e r co s tandv i c ev e r sa . C loudcompu t ingcu s tom e r sc ans e tad e ad l in efo rth e ex e cu t ionr equ i r inga m in im a lco s t ,r a th e rth ana m in im a l mak e span[15] .Inth e s eca s e s ,budg e tcon s t ra in t sandr edu c ingth era c efo rth esp e edupcany i e ldth er edu c edco s tfo rth e 7

8

Elastic Cloud Services Compliance with Gustafson’s and Amdahl’s Laws


execution. For example, although a superlinear speedup can be achieved in a Windows Azure cloud for matrix multiplication when VM instances with Windows operating system are used, Linux VM instances achieved better performance cost trade-off because they are cheaper. On the other side, there is a risk of cloud resources performance variation, different setup time [16], instance failure over the time [17] and difficulty to predict the performance, which will harden the resource provisioning [18]. Additional problem in modeling the elastic cloud services’ behavior is the uncertainty in cloud provisioning and VM instability. For example, Dejun et al. [19] reported a performance uncertainty of up to 8% in Amazon EC2. Increasing the budget by duplicating the tasks on more than one instance could mitigate those risks, in order to meet the deadline [20]. Sometimes, using a bigger instance executes the task faster, rather than waiting several minutes for the deployment time to start another smaller, but an appropriate instance, which reduces the turnaround time of an activity [21]. Not all offered pricing models are linear. For example, some providers charge the customers on hourly based policy, while others charge some amount at the beginning plus charge then per smaller time unit. For example, Google charges the usage for the first 10 minutes, and then per minute. Also, Google have recently introduced the non-linear model by including the VM usage sustainability. All these issues impact on choosing the appropriate scaled system for a specific cloud elastic service.

VII. Conclusion Cloud services are scalable and can be executed in both the parallel and distributed systems by load balancing among the scaled resources. This balancing reduces the amount of work per resource, which speedups the average execution time. Predicting and measuring the performance of such services is very difficult because the real cloud elastic service receives client requests with an unknown distribution probability function. Also, they are hosted on an unpredictable resource provisioning, which makes their modeling almost impossible. Still, by using the upper and lower limits of the speedup, one can compare the fairness of the pricing model. The Amdahl’s and Gustafson’s laws set limits on the speedup that a scaled system achieves, but usually for granular algorithms. However, even in the traditional parallel and distributed systems, there are many cases when these laws are disproved due to the nature of the algorithms, hardware and software architecture. The uncertainty of the VM provisioning and performance, along with many differences 8

between the scaling and granular algorithms, questions their compliance with both laws. However, our modeling and theoretical analysis showed that cloud elastic services are compliant with both laws. Such general laws are push drivers to enable the technologies and pull drivers that lead toward technical innovations. This chain of push and pull drivers makes the positive feedback that enables the overall technology continual development.

Acknowledgment This work is partially supported by the European Union’s Horizon 2020 research and innovation programme under the grant agreements 644179 ENTICE: dEcentralized repositories for traNsparent and efficienT vIrtual maChine opErations (first two authors) and 643946, CloudLightning: Selforganizing, self-managing Heterogeneous Clouds (fourth author). The authors would like to acknowledge networking support by the COST programme Action IC1305, Network for Sustainable Ultrascale Computing (NESUS).

References [1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, et al., A view of cloud computing, Communications of the ACM 53 (4) (2010) 50–58. [2] A. Gupta, D. Milojicic, Evaluation of hpc applications on cloud, in: Open Cirrus Summit (OCS), 2011 Sixth, 2011, pp. 22–26. doi:10.1109/OCS.2011.10. [3] S. A. Tsaftaris, A scientist’s guide to cloud computing, Computing in Science Engineering 16 (1) (2014) 70–76. doi:10.1109/MCSE.2014.12. [4] L. Liu, M. Zhang, Y. Lin, L. Qin, A survey on workflow management and scheduling in cloud computing, in: Cluster, Cloud and Grid Computing (CCGrid), 2014 14th IEEE/ACM International Symposium on, 2014, pp. 837–846. doi:10.1109/CCGrid.2014.83. [5] M. Gusev, S. Ristov, Superlinear speedup in Windows Azure cloud, in: Cloud Networking (IEEE CLOUDNET), 2012 IEEE 1st International Conference on, Paris, France, 2012, pp. 173–175. [6] S. Ristov, F. Dimitrievski, M. Gusev, G. Armenski, Scalable system for e-orders as a service in cloud, in: International Conference on Computer as a Tool (IEEE EUROCON 2015), Salamanca, Spain, 2015, pp. 1–6.


9


[7] G. M. Amdahl, Validity of the single-processor approach to achieving large scale computing capabilities, in: AFIPS Conference Proceedings, Vol. 30, AFIPS Press, Reston. Va., Atlantic City, N.J., 1967, pp. 483–485. [8] J. L. Gustafson, Reevaluating Amdahl’s law, Communication of ACM 31 (5) (1988) 532–533. [9] J. Gustafson, G. Montry, R. Benner, Development of parallel methods for a 1024-processor hypercube, SIAM Journal on Scientific and Statistical Computing 9 (4) (1988) 532–533. [10] S. Ristov, M. Gusev, G. Velkoski, Modeling the speedup for scalable web services, in: A. M. Bogdanova, D. Gjorgjevikj (Eds.), ICT Innovations 2014, Vol. 311 of Advances in Intelligent Systems and Computing, Springer International Publishing, 2015, pp. 177–186. [11] M. Gusev, S. Ristov, Resource scaling performance for cache intensive algorithms in Windows Azure, in: F. Zavoral, J. J. Jung, C. Badica (Eds.), Intelligent Distributed Computing VII, Vol. 511 of SCI, Springer International Publishing, 2014, pp. 77–86. [12] M. Gusev, S. Ristov, The optimal resource allocation among virtual machines in cloud computing, in: Proceedings of The 3rd International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2012), Nice, France, 2012, pp. 36–42. [13] S. Ristov, K. Cvetkov, M. Gusev, Implementation of a scalable L3B balancer, Scalable Computing: Practice and Experience 17 (2) (2016) 79–90. doi:10.1109/TE.2014. 2327007. [14] M. X. Trang, Y. Murakami, T. Ishida, Cloud Computing: 6th International Conference, CloudComp 2015, South Korea, Springer International Publishing, 2016, Ch. Modeling Parallel Execution Policies of Web Services, pp. 244–254.

[15] M. A. Rodriguez, R. Buyya, Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds, IEEE Transactions on Cloud Computing 2 (2) (2014) 222–235. doi:10.1109/TCC.2014. 2314655. [16] M. Mao, M. Humphrey, A performance study on the vm startup time in the cloud, in: Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on, 2012, pp. 423–430. [17] F. Wu, Q. Wu, Y. Tan, Workflow scheduling in cloud: a survey, The Journal of Supercomputing 71 (9) (2015) 3373–3418. doi:10.1007/s11227-015-1438-4. [18] A. Tchernykh, U. Schwiegelsohn, V. Alexandrov, E.G. Talbi, Towards understanding uncertainty in cloud computing resource provisioning, Procedia Computer Science 51 (2015) 1772 – 1781. [19] J. Dejun, G. Pierre, C.-H. Chi, Service-Oriented Computing. ICSOC/ServiceWave 2009 Workshops: International Workshops, ICSOC/ServiceWave 2009, Stockholm, Sweden, November 23-27, 2009, Revised Selected Papers, Springer Berlin Heidelberg, Berlin, Heidelberg, 2010, Ch. EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications, pp. 197–207. [20] R. N. Calheiros, R. Buyya, Meeting deadlines of scientific workflows in public clouds with tasks replication, IEEE Transactions on Parallel and Distributed Systems 25 (7) (2014) 1787–1796. doi:10.1109/TPDS.2013.238. [21] M. Mao, M. Humphrey, Scaling and scheduling to maximize application performance within budget constraints in cloud workflows, in: Parallel Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on, 2013, pp. 67–78. doi:10.1109/IPDPS.2013.61.

9