2012 IEEE Fifth International Conference on Cloud Computing
Scalability Patterns for Platform-as-a-Service Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Davide Rebeccani Universit`a degli Studi di Milano – DTI 26013 Crema - Italy Email: fi
[email protected]
Marco Ughetti Telecom Italia, TILab 10148 Torino - Italy Email:
[email protected]
even though virtualization makes it seem like that is the case. PaaS, where operating systems and middleware are virtualized and provided as a service to customers, promises an unprecedented ability of increasing the capabilities of allocated virtual resources (scale up), seamlessly adding new resources (scale out), and releasing them when they are no longer needed (scale down). Indeed, PaaS may seem to provide a quick fix to a time-honored problem of capacity planning: the lack of predictability of service usage. In this paper, we present an analysis of some scalability patterns often applied to PaaS, which implements a ServiceOriented Architectures (SOA) middleware stack. Our claim is that these patterns may give to the customers, and even to the providers, the impression of managing infinite physical resources, raising expectations that may prove unfounded in the end; still it is difficult to evaluate their behavior a priori. Therefore, we carried out an experimental evaluation of cloud scalability patterns in a WSO2 platform [3], a clouddeployable, Java-based service-oriented platform [4]. For our experimentation, we set up a monitoring infrastructure supporting automatic scalability management. Our evaluation shows that performance improvements achieved by adding virtual resources are highly constrained by the underlying physical system, and their speedup shows a small pseudolinear ramp up zone. The remainder of this paper is organized as follows. Section II illustrates our scenario. Section III presents architectural patterns for scalability. Section IV describes our solution for monitoring PaaS performance. Section V contains our experiments comparing different scalability patterns in different security environments. Finally, Section VI describes the related work and Section VII draws our conclusions.
Abstract—Platform-as-a-Service is a cloud-based approach that provides enterprises with all the functionalities for developing, deploying, and administering services, without the burden of installing, configuring, and managing the underlying middleware, operating system, and hardware. In this context, scalability becomes a fundamental requirement, and appropriate solutions need to be studied and evaluated. In this paper, we present different scalability patterns for a Platform-as-a-Service infrastructure and a two-level approach to performance monitoring allowing automatic scalability management. We also provide a performance evaluation of the scalability patterns on a Service-Oriented Architecture (SOA) PaaS, which considers the impact on performance of SOA security standards. Keywords-Cloud; Platform-as-a-Service; Scalability.
I. I NTRODUCTION The cloud computing paradigm supports a vision of distributed IT where software services and applications are outsourced and used on a pay-as-you-go basis [1]. While the cloud vision has many advantages, it makes the end user unaware of the infrastructure performance and capacity constraints. Often, cloud providers present the cloud as a scalable infrastructure whose infinite resources are available on demand. Our work started with the observation that cloud users’ expectations of scalability can differ depending on the services they use. Cloud services are released at three levels [2]: i) Infrastructure-as-a-Service (IaaS), ii) Platform-as-a-Service (PaaS), and iii) Software-as-a-Service (SaaS). As far as IaaS is concerned, it looks reasonable to assume that users will carry over to the cloud the performance expectations they have when dealing with on-premise systems. For instance, when users assign two independent tasks to two virtual machines, they are led to think that there will be no overhead, even if the two machines share the same physical hardware. On the PaaS level, users expectations are harder to guess. PaaS users do know that resources are virtualized, so at some level they may expect interference between virtual platforms relying on the same physical environment. For this reason, they usually require such interference to be bounded by specific Service Level Agreements (SLAs). On the SaaS level, users may not even be aware of virtualization; however, SaaS applications are not deployed stand-alone, 978-0-7695-4755-8/12 $26.00 © 2012 IEEE DOI 10.1109/CLOUD.2012.41
II. R EFERENCE SCENARIO Our reference scenario is a PaaS infrastructure that includes the following parties: i) a SOA platform, providing the PaaS functionalities to customers/tenants; ii) customers/tenants, designing, developing, deploying, managing their services in isolation on a shared SOA platform; iii) clients, requesting access to services provisioned by customers/ tenants. Our proposal is validated on the PaaS offering by 718
installed on it is released to each customer in the cloud.1 SPP is the simplest pattern and has high manageability (i.e., low management and set-up complexity). For each customer joining the PaaS, a new (set of) virtual machine is started. This pattern is characterized by scarce resource utilization, since the resource distribution is done in advance independently by the real load, and low scalability, because the number of virtual machines and platforms is linear in the number of customers.
Telecom Italia, a leading European telecommunication company, that consists of a multi-tenant WSO2 installation [3] based on the following suite of products. • Carbon middleware. An OSGi framework based on Equinox implementing common infrastructural capabilities. Each WSO2 product is a set of OSGi features deployed on the common Carbon layer. • WSO2 Application Server. A runtime execution service aimed at hosting web services and web applications. It is an Application Server (AS) based on Apache Tomcat and Apache AXIS2 built in on the Carbon middleware. • WSO2 Enterprise Service Bus. The sole, secure, and controlled access point for all services. It is based on Apache Synapse and it provides typical Enterprise Service Bus (ESB) capabilities like security, access throttling, monitoring, load balancing, and failover. • WSO2 Governance Registry. A typical SOA registry able to store, manage, and customize all WSO2 metadata (e.g., WSDL, XML schema, ESB configurations). In the following, we will describe how architectural patterns for scalability can be applied to the above PaaS scenario, and we will carry out a performance analysis of such patterns.
Shared Platform Pattern (ShPP). It considers a multitenant scenario where a single platform is installed on a (set of) virtual machine. In this case, the platform is shared by multiple tenants, all having the right to manage a portion of it and deploy their services independently. ShPP preserves the performance level of the platform among load variations by dynamically scaling up/scaling down the resources assigned to the virtual machine. Whenever the load increases degrading the performance metrics of the platform, RAM, CPU, and/or bandwidth can be increased. By contrast when the load decreases, the resources can be freed and made available to other processes in the architecture. In ShPP, resources are shared and need to be managed in a way that preserves security and isolation among tenants. As a consequence, ShPP provides high utilization, while it does not provide linear scalability increase due to the additional overhead for resource management.
III. S CALABILITY PATTERNS We identified five scalability patterns that provide loadbased dynamic scaling of resources to a PaaS. Our patterns can be classified depending on the type of action used for resource scaling. • Vertical scaling (scale up): additional resources (e.g., CPU, RAM, bandwidth) are assigned to a single machine when the load increases. The resources can be both physical resources added to a server hosting several applications, and virtual resources dynamically assigned to a virtual machine or its applications. • Horizontal scaling (scale out): new machines are added to the system providing more software and hardware resources. This paradigm permits the injection of new resources by clustering or by sharing the workload using additional machines (i.e., servers). For example, horizontal scaling adds a machine to a clustered platform or a new instance of a service/platform. We note that a scale down approach can be applied to both vertical and horizontal scaling to release those additional resources and/or machines which are not needed anymore. For the sake of conciseness, here we provide an informal description of our scalability patterns. A formal description of our patterns is proposed in [5], where we used the POSA architectural templates [6] to specify the scalability patterns in the context of our big telecommunication environment.
Clustered Platform Pattern (CPP). It is the solution adopted by almost all major PaaS providers (e.g., WSO2 , Google App Engine) [7]. A single platform is deployed supporting clustering and is shared by all tenants. Multiple instances of the platform components (e.g., AS, ESB) can be deployed in different machines of the cluster. CPP therefore supports a scalability solution that guarantees the performance levels of the platform by dynamically scaling out/scaling down the resources assigned to the cluster. This is achieved by adding/removing machines to the clustered system when needed. CPP has lower manageability than ShPP. Similarly to ShPP, this pattern manages shared resources preserving security and isolation among tenants. In addition, it implements a set of functionalities including load balancing, PaaS monitoring, and elastic autoscaling. CPP provides high resource utilization, since the machines in the cluster are shared among different tenants, and some scalability, as system resources can be incrementally extended. Finally, CPP promises high availability and reliability due to the introduced redundancy. Multiple ShPP (MShPP). It is an extension of ShPP and mix both vertical and horizontal scaling. At initialization time, a single, shared, multi-tenant platform is deployed. Upon an increase in the load, additional resources (e.g.,
Single Platform Pattern (SPP). It is the basic scalability pattern and considers a single-tenant scenario. A single, isolated, and complete virtual machine with a platform
1 The platform installation can include a set of virtual machines, as for instance, one machine for each of the components in Section II.
719
Table I PATTERN EVALUATION
CPU, RAM, bandwidth) are assigned to maintain the performance metrics. In case additional resources are not sufficient, a new platform is deployed and a part of the existing tenants are migrated to the new platform together with the services they own. When the load decreases, the additional platforms can be removed causing the tenants to migrate back to the available platforms. MShPP has lower manageability than ShPP. If on one side the manageability of MShPP is similar to the one of ShPP for load manageable using a simple scale up of resources, on the other side heavy loads may require the deployment of new platforms, introducing the need of migrating tenants and their services, configurations, and data. The latter process is expensive especially when we consider stateful services. MShPP provides high resource utilization, while its scalability depends on the specific scenario and number of deployed platforms. In the worst case, when all tenants experienced a traffic peak, a platform is deployed for each tenant and therefore scalability of MShPP is equivalent to the one of SPP; in the average case, MShPP provides high scalability.
Pattern
Manageability
SPP ShPP CPP MShPP MCPP
High Medium/High Medium Low Low
Resource Utilization Low High High High High
Scalability Low Medium Medium High High
Availability Reliability Medium Low High Medium High
CPP, respectively, with an additional step of horizontal scaling, provide high resource utilization, high scalability, and medium (MShPP) and high (MCPP) availability/reliability, at the price of a substantial increase in complexity due to the need of adding/removing platforms on demand. There is a subtlety, however, to consider when evaluating our scalability patterns. SPP, MShPP, and MCPP rely on the instantiation and deployment of additional platforms, and are mainly adopted in scenarios where virtual resources assigned to the system are directly mapped on physical resources.2 By contrast, ShPP and CPP mainly consider a scenario in which virtual resources assigned to the system are only indirectly mapped on physical resources. This case may trigger interference phenomena between the virtual resources assigned to the PaaS, thus potentially reducing (nullifying in the worst case) the linear speedup in performance that is usually guaranteed when physical resources are distributed. In the following, we will experimentally evaluate ShPP and CPP, to measure the impact that the addition of virtual resources, indirectly mapped on physical ones, may have on the quasi linear speedup in performance usually achieved by solutions assuming a direct mapping.
Multiple CPP (MCPP). It is an extension of CPP and provides horizontal scaling at two levels of abstraction. At initialization time, a single, shared, multi-tenant platform supporting clustering is deployed. Upon an increase in the load, additional resources (i.e., machines in the cluster) are added to maintain the performance level. In case the extended cluster is not sufficient to manage the new load, a new platform supporting clustering is deployed, and a part of the existing tenants are migrated to the new clustered platform together with the services they own. When the load decreases, the additional platforms can be removed causing the tenants to migrate back to the available platforms. MCPP has the lowest manageability among patterns. In addition to the complexity of CPP, MCPP may require the deployment of additional platforms, introducing the need of migrating tenants and their services, configurations, and data. MCPP usually provides high utilization of resources, although utilization may decrease in case of multiple platform deployments. MCCP promises high scalability due to the shared clusters, and high availability and reliability due to the introduced redundancy.
IV. P ERFORMANCE MONITORING The patterns presented in Section III aim to manage unexpected peaks in traffic loads. To prove or disprove that this is actually the case, we designed an architecture able to monitor the platform performance and select the pattern(s) that better fits the scenario under control. Our monitoring architecture will raise alarms when events, that may affect platform performance and availability, are observed, that is, when a set of performance metrics break predefined thresholds. We apply a two-level monitoring of the infrastructure: platform-level and host-level. Platformlevel monitoring checks the state of the platform and its services through a set of performance metrics. For instance, WSO2 exposes specific interfaces that can be implemented to monitor and collect performance metrics of the ESB via common Java applications (e.g., JConsole). We consider the following metrics for each endpoint: • Total Count (TC): number of messages forwarded to a given endpoint. If the metric exceeds a known thresh-
A. Discussion We compared the above patterns, evaluating their complexity, resource utilization, scalability, and availability/reliability (see Table I). SPP provides the highest manageability, with no interference between tenants, at the cost of a poor resource utilization and scalability. ShPP improves resource utilization and scalability, since tenants share a single platform, still keeping medium/high manageability. CPP provides high resource utilization, high availability/reliability, and medium scalability, at the price of an increase in complexity. MShPP and MCPP, which extend ShPP and
2 These scalability patterns have been extensively studied in the literature on high performance and availability systems (e.g., [8], [9]).
720
•
•
old, the performance could be affected and an alarm is raised. Fault Count (FC): number of messages that resulted in a fault while being forwarded to the endpoint. Strictly related to TC, it gives an evaluation on how the service is working, and if it is still able to fulfill all requests. Minimum Time (MinT), Maximum Time (MaxT), Average Time (AveT): the minimum, maximum, and average time taken to send a request to the endpoint and receive a response. Great variance in MinT and MaxT indicates that the service may not be responsive for high load.
Table II A LARM RULES ID 1 2 3 4 5 6 7 8
Host-level monitoring checks the state of the underlying virtualization system in terms of CPU cycles and memory occupation. In our case, where virtual machines (VMs) are implemented using VMWare technologies, we use VMWare ESXi PowerCLI APIs to identify which VMs are in a critical state and, among them, which need more resources. PowerCLI can be used to manage and monitor the system, collecting data on the status of the physical server and the execution of each VM. We consider the following metrics collected through PowerCLI: •
•
•
•
Alarm Rule if F C/T C IS HIGH if M axT − M inT is HIGH if AveTi − AveTj is HIGH if T C is LOW and M O is HIGH if T C is HIGH and CL is LOW and HA is LOW if CL is HIGH and T C is LOW if N U is HIGH if T C is LOW and CL is HIGH and HA is LOW
Figure 1.
Raised Alarm
Message Alarm
Processing Alarm
Alarm-driven selection of scalability patterns
A basic assumption behind automatic scalability on the cloud is that alarms like the ones above can be used to trigger the application of scalability patterns, thus maintaining the same level of performance upon load variations. Figure 1 presents a simple algorithm for automatic scaling as a finite state automaton. The initial node ∗ represents the basic installation scenario in which different tenants share the same platform with default configurations. Upon an increase in the load that raises a message alarm, the algorithm moves to node ShPP and applies a ShPP pattern. By contrast, if a processing alarm is raised, the algorithm moves to node CPP and applies a CPP pattern. When the ShPP pattern is not sufficient to solve further alarms, it moves to node MShPP in case of message alarms or to node CPP in case of processing alarms. To conclude, the algorithm moves from CPP to MCPP for both types of alarms, while it moves from MShPP to MCCP in case of processing alarms. Following the finite state automaton in the reverse way, patterns get deactivated and the corresponding resources released as the load decreases.3 The assumption behind automatic scaling is that scalability patterns will work on the virtualized architecture as one would expect, that is, more or less in the same way as on a physical architecture. We will deal with this assumption in Section V.
CPU Load (CL): CPU utilization on host and guest systems. High values of CL in a VM identify a problem in the fulfillment of request messages backlog. Memory Occupancy (MO): memory utilization on host and guest systems. Services that manage great amount of data (e.g., huge XML files) may require substantial portions of memory at the detriment of other services. Network Utilization (NU): utilization of network bandwidth. High values of NU may suggest the re-allocation of external resources to manage a peak of requests. Host Availability (HA): number of virtual machines available and accessible through the network. The falling of the HA under a pre-defined threshold indicates the need for new machines.
Our twofold monitoring approach led us to the definition of two categories of alarms, message and processing alarms, which can drive the dynamic selection of the patterns in Section III. The measurements returned by the monitoring architecture are used within the set of rules in Table II to raise message and processing alarms. Specifically, a Message Alarm is raised when i) the system is not able to manage the message queue efficiently, ii) the average message delivery time, or iii) the difference between the maximum and minimum message times is above a preset threshold. A Processing Alarm is used for scenarios where service executions may involve high execution time or a lot of resources. For instance, Row 6 in Table II describes the case where the CPU Load is high, while the total number of requests is low. This case can indicate that specific services are monopolizing processor time. HIGH and LOW thresholds in Table II can be defined on the basis of previous experimental tests and/or expert knowledge.
V. E XPERIMENTAL EVALUATION We experimentally evaluated the performance of our scalability patterns, under different environments, security contexts, and loads. As discussed in Section III-A, we focused on the performance achieved by applying the two scalability patterns with an indirect mapping on physical 3 As discussed in [10], many services show yearly and daily performance profiles. These data can be used to do preventive scaling, hopefully reducing the window of time where a performance decrease is observed. We plan to consider this issue in our future work.
721
resources: i) ShPP that increases RAM assigned to AS and ESB components of the WSO2 platform; ii) CPP that adds new virtual machines, each one providing an AS, to the WSO2 platform. Our experiments on CPP used a pure round-robin scheduling algorithm with no session affinity, that is, requests have been routed by the load balancer without taking into account their source address.
1000
160
RT TPS
140 120 100
A. Experimental setting
80 60
10
We set up an experimental environment where a WSO2 platform, with default configurations, runs on a physical machine Dell PowerEdge 6850 equipped with 4 Intel Xeon Quad Core 2.6 GHz, 16GB ECC RAM, 6x 146 GB Serial Attached SCSI 10K RPM, and 2x 1Gb/s Ethernet NIC. The basic installation of the WSO2 platform consists of four virtual machines installed on VMware ESXi v.4.0.0, providing an Enterprise Service Bus v.4.0.0, an Application Server 4.1.0, a Governance Registry 4.0.0, and an Identity Server 3.2.0, each one with Ubuntu Server 11.04 kernel 2.6.38 operating system. Each virtual machine is equipped with 2 vCPU, 20GB of disk, 2x 1GB vNIC, and 2GB, 4GB, 1GB, 4GB of RAM, respectively. The WSO2 platform supports clustering and has been tuned to provide 2GB of heap memory to each of its components, to improve the performance of the underlying Java Virtual Machines. We simulate a real scenario with concurrent requests coming from different clients (i.e., threads). Each test case starts with 20 active clients sending SOAP requests, which ramp up to a maximum of 100 clients. All test cases have a duration of 60 seconds and simulate different loads varying the request per seconds (rps), from 10rps to 500rps. The load has been generated and sent to an echo service, deployed on the WSO2 platform, using soapUI (http://www.soapui.org/), an open source functional testing solution for web services. The echo service has been developed in two versions: a basic one with no support for security, and a secured one where each SOAP message is signed and encrypted according to the WS-Security specifications [11]. Each message of our secured echo service is then extended with security headers, which guarantee security of the communications over the Internet and isolation in the shared enterprise service bus. In a multi-tenant PaaS paradigm, in fact, ensuring SOAP message isolation is critical, since different tenants share the same enterprise service bus, and could conceivably exploit their privileges to eavesdrop on the bus and interfere with other tenants’ activities. For each test scenario, we repeated the test case five times and we measured the mean transactions per second (TPS) and the average response time (RT) over the five executions.
TPS
RT (ms)
100
40 20 1
0 0 50
0 20
s rp
s rp
s rp
ps
ps
ps
0 10
r 50
r 20
r 10
(a)
10
RT TPS
10000
8
1000
6
100
4
10
2
1
0
TPS
RT (ms)
100000
50
20
10
s rp
s rp
s rp (b)
Figure 2.
Baseline measurements with no security (a) and security (b)
show RT (in a logarithmic scale) and TPS, when no security and WS-Security are used, respectively. When no security is considered (Figure 2(a)), the system supports a load of at most 152 TPS (breaking point). For loads below such a threshold, our results show similar RT in the range [18ms,24ms]. For loads over the breaking point, RT growth substantially to 472ms. We note that RT for 200rps and 500rps are similar since the overall load adapts to the server performance to avoid its congestion. As far as WS-Security is concerned (Figure 2(b)), we observe a relevant decrease in performance. The maximum breaking point is reached at 3.82 TPS, with a considerable increase in RT (more than two orders of magnitude). No difference in RT is observed among different loads (i.e., 10rps, 20rps, 50rps), because they are over the breaking point. C. ShPP vs CPP
B. Baseline and security overhead
Figure 3 shows our results for ShPP varying the message load, that is, 100rps, 200rps, and 500rps in a scenario with no security, 10rps, 20rps, 50rps in a scenario with security, and the RAM memory given to AS (i.e., 2GB
We analyze the performance of our basic installation varying the load, that is, 10rps, 20rps, 50rps, 100rps, 200rps, and 500rps, and the security context. Figures 2(a) and 2(b)
722
1000
Figure 4 shows our results for CPP varying the message load as for ShPP and the number of virtual machines (VMs) in the cluster (i.e., 2, 3, and 5). Each additional machine is a clone of the machine in the basic configuration providing AS. When no security is considered (Figure 4(a)), the addition of virtual machines provides a substantial increase in TPS and decrease in RT. More in detail, for two and three virtual machines in the cluster, we observe that the breaking point moves to 274 TPS and 344 TPS, respectively, with an increase of 80.3% and 126.3% with respect to the baseline (i.e., 1VM scenario). RT is in the range [20.9ms,25.5ms], when the load is under the breaking point, showing a decrease around 95% for 200rps with reference to the baseline. For a load of 500rps, we observe a decrease, with respect to the baseline, of 44.2% for two machines and 55.9% for three machines. By contrast, adding five machines to the cluster reduces the performance gain, that becomes less than the one achieved with three machines. As far as WS-Security is concerned (Figure 4(b)), no relevant performance gain is achieved. This is due to the fact that ESB becomes the bottleneck because of the additional overhead in security management.
180
RT TPS
160 140 120
RT (ms)
100
TPS
100 80 10
60 40 20
1
0 B
4G
200rps
B
B
3G
2G
B
4G
B
B
3G
2G
B
4G
B
B
3G
2G
100rps
500rps
(a)
16000
10
RT TPS
14000 8
10000
6
TPS
RT (ms)
12000
8000 4
6000
D. Discussion
4000
The first result of our evaluation is that security causes a substantial decrease in the performance of a SOA deployed on the cloud. This result confirms the claims made by previous works on SOA performance evaluation [12]. WSSecurity, in fact, introduces the need of parsing, validating, encrypting/decrypting, and querying complex XML files. A less expected result regards ShPP and CPP patterns when no security mechanisms are in place. When we apply ShPP, we observe a performance gain both on TPS and RT with respect to the baseline. We also note that the best results are achieved with 3GB, while no further gain is obtained with 4GB. This result shows that no more than 3GB of memory are necessary to accomplish all service requests. CPP provides a further improvement with respect to ShPP showing the suitability of the horizontal scaling solution in the cloud. However the quasi-linear speedup range is rather limited, as the best performance is achieved using three machines in the cluster. When five ASs are deployed, the performance decreases due to the additional overhead introduced by the clustered architecture and to the fact that virtual scaling is bounded by physical resources. The reduced range of linear scaling becomes more clear when security is considered. In our experimental setting, scalability patterns have a low impact on performance improvements when security mechanisms are in place. This result shows that the impact of performance enhancers in a virtualized architecture is highly constrained by the amount of physical resources made available to the hypervisor that manages the virtual machines of the PaaS. The injection of virtualized resources, in fact, can provide benefits and a pseudo-linear ramp up in
2 2000 0
0 B
B
4G
B
3G
2G
B
B 20rps
4G
B
3G
2G
B
B
4G
B
3G
2G
10rps
50rps
(b)
Figure 3.
Performance using ShPP: no security (a) and security (b)
as in the baseline, 3GB, and 4GB).4 When no security is considered (Figure 3(a)), the addition of RAM gives a moderately increasing trend in TPS and a decreasing one in RT. Focusing on the 3GB scenario, TPS increases of 11.4% for 200rps and 12.5% for 500rps with respect to the baseline (i.e., 2GB scenario), achieving a breaking point around 171 TPS. RT (in a logarithmic scale in Figure 3(a)) decreases of 46.1% for 100rps, 11.2% for 200rps, and 11.1% for 500rps. Also, we note that 3GB of memory already gives the highest performance values, meaning that further additions of RAM do not influence the performance. As an example, with 4GB of RAM, we have performance improvements of less that 1% for both TPS and RT, independently by the load. As far as WS-Security is concerned (Figure 3(b)), the increase in memory does not provide substantial improvements. This is due to the fact that our loads (i.e., 10rps, 20rps, 50rps) largely exceed the maximum breaking point of 3.94 TPS achieved with 4GB and 10rps. However, we observe a decreasing trend in RT when the memory increases. 4 The RAM memory assigned to ESB is increased accordingly (i.e., 4GB as in the baseline, 5GB, and 6GB).
723
RT (ms)
450
Another line of research has considered the problem of implementing multi-tenant SaaS and PaaS solutions, which guarantee security, availability, and reliability [17]–[20]. Mietzner et al. [20] introduce a set of multi-tenancy patterns for building SOA applications. Azeez et. al. [18] present a multitenant SOA middleware for cloud environments, allowing customers to design, build, and manage their services on the cloud. The authors also discuss the problem of protecting security among different tenants by guaranteeing isolation of data and execution. Pathirage et al. [19] provide a multitenant workflow engine allowing several tenants to execute their workflows on the same architecture and in a secure way. In principle, multi-tenancy may cause interference among services shared by different users, although this phenomenon has been seldom observed in practice. We did not model multi-tenancy in our experimentation, as the literature unanymously says that its effect - if any - is always in the direction of a performance reduction. Other works have addressed a variety of issues in the context of performance optimization on the cloud. As an example, Yi et al. [21] present a different view of the scalability problem in SaaS, namely the On-line Tenant Placement Problem (OTPP), and propose an heuristic that, given a fixed number of nodes, optimally distributes tenants to maximize their total number. Finally, a line of research related to ours is the one on models for performance testing. Perera and Lindton [22] present a study on the performance of major ESB in different scenarios, including security, together with an ESB performance test suite. Stantchev [23] describes a methodology to performance evaluation of cloud computing solutions varying the platform instances from one to three. Iosup et al. [24] analyze the performance of cloud computing services for scientific computing workloads. Differently from all above works, our solution analyzes a set of scalability patterns for PaaS. Also, our evaluation has considered a SOA PaaS deployment and assumed a scenario without a one-to-one static link between physical and virtual resources.
400
RT TPS
360
400
320
350
280
300
240
250
200
200
160
150
120
100
80
50
40
0
0 5V M
M
M
M
M
M
M
M
M
M
M
M
200rps
3V
2V
1V
5V
3V
2V
1V
5V
3V
2V
1V
100rps
TPS
500
500rps
(a)
16000
10
RT TPS
14000 8
10000
6
TPS
RT (ms)
12000
8000 4
6000 4000
2 2000 0
0 M
M
5V
M
3V
M
20rps
2V
1V
M
M
5V
M
3V
M
2V
1V
M
M
5V
M
3V
M
2V
1V
10rps
50rps
(b)
Figure 4.
Performance using CPP: no security (a) and security (b)
performances if and only if the underlying physical resources are able to sustain the scale of traffic and load. This means that the assumption “the more the virtualized resources, the higher the performances” must be taken with a grain of salt in our experimental setting, although it may still apply to hardware pay-as-you-go scenario, where physical computational resources can be increased on demand.
VII. C ONCLUSIONS VI. R ELATED WORK
Scalability is often mentioned as a major opportunity offered by cloud and SOA paradigms. In this paper, we presented some scalability patterns that can be applied in a PaaS scenario, discussing basic techniques to achieve automatic scalability with changing environments and loads. The experimental evaluation of scalability patterns has shown that adding virtual resources provides quasi-linear speedup only on a very limited range, which gets further reduced when security mechanisms are in place. Solutions relying on scalability-by-addition (of virtual resources) should be thoroughly tested before adoption of PaaS. The analysis presented in this paper leaves space for further work. First, we will provide a more detailed per-
Scalability is an important requirement in distributed environments and solutions to provide scalable systems can be developed at different levels [13], [14]. C´aceres et al. [15] provide an overview of how cloud computing can support the development of scalable applications. Much in line with the work in this paper, Yu et al. [16] analyze the performance of the WebSphere Application Server, when a scale out approach is used with a typical industrial workload. The scale out approach is also compared with the scale up one in terms of performance and cost effects. Differently from our work, they consider neither the cloud scenario nor the impact of security on performances.
724
formance evaluation which focuses on multi-tenancy, and considers more complex services and requests. Then, we will evaluate all five scalability patterns in this paper. Finally, we will study a solution for reducing the overhead introduced by WS-Security on PaaS performance, as for instance, by providing a WS-Security implementation which works on XML binary encoding [25], [26].
[13] R. Jimenez-Peris, M. Pati˜no-Martinez, B. Kemme, F. PerezSorrosal, and D. Serrano, “A system of architectural patterns for scalable, consistent and highly available multi-tier serviceoriented infrastructures,” in Architecting Dependable Systems VI, R. de Lemos, J.-C. Fabre, C. Gacek, F. Gadducci, and M. ter Beek, Eds. Springer Verlag, 2009, vol. LNCS 5835. [14] J. S¨oderlund, Scalability patterns and an interesting story..., August 2010, http://thebigsoftwareblog.blogspot.com/2010/ 08/scalability-fundamentals-and.html, Accessed in date May 2012.
ACKNOWLEDGEMENTS This work was partly funded by the European Commission under the project ASSERT4SOA (contract n. FP7257351) and by Telecom Italia S.p.A. (contract n. 9/2011).
[15] J. Caceres, L. Vaquero, A. P. L. Rodero-Merino, and J. Hierro, “Service scalability over the cloud,” in Handbook of Cloud Computing, B. Furht and A. Escalante, Eds. Springer, 2010.
R EFERENCES
[16] H. Yu, J. Moreira, P. Dube, I. Chung, and L. Zhang, “Performance studies of a websphere application, trade, in scale-out and scale-up environments,” in Proc. of IEEE IPDPS 2007, Long Beach, CA, USA, March 2007.
[1] M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “Above the clouds: A berkeley view of cloud computing,” in Tech. Rep. UCB/EECS-2009-28, EECS Department, U.C. Berkeley, February 2009.
[17] S. Subashini and V. Kavitha, “A survey on security issues in service delivery models of cloud computing,” Journal of Network and Computer Applications, vol. 34, no. 1, pp. 1–11, 2011.
[2] P. Mell and T. Grance, The NIST Definition of Cloud Computing, National Institute of Standards and Technology (NIST), July 2009, http://thecloudtutorial.com/ nistcloudcomputingdefinition.html, Accessed in date May 2012.
[18] A. Azeez, S. Perera, D. Gamage, R. Linton, P. Siriwardana, D. Leelaratne, S. Weerawarana, and P. Fremantle, “Multitenant SOA middleware for cloud computing,” in Proc. of IEEE CLOUD 2011, Miami, FL, USA, July 2010.
[3] WSO2 Carbon, http://wso2.com/products/carbon/. [4] A. Azeez, S. Perera, S. Weerawarana, P. Fremantle, S. Uthaiyashankar, and S. Abesinghe, “WSO2 stratos: An application stack to support cloud computing,” it - Information Technology, vol. 53, no. 4, pp. 180–187, 2011.
[19] M. Pathirage, S. Perera, I. Kumara, and S. Weerawarana, “A multi-tenant architecture for business process executions,” in Proc. of IEEE ICWS 2011, Washington, DC, USA, July 2011.
[5] PaaS Scalability Patterns: A POSA-Based Description, SESAR Lab, http://sesar.dti.unimi.it/ScalabilityPatterns/.
[20] R. Mietzner, T. Unger, R. Titze, and F. Leymann, “Combining different multi-tenancy patterns in service-oriented applications,” in Proc. of IEEE EDOC 2009, Auckland, New Zealand, September 2009.
[6] F. Buschmann, K. Henney, and D. Schmidt, Pattern-Oriented Software Architecture: A Pattern Language for Distributed Computing. Wiley & Sons, 2007. [7] M. Yuan, Java PaaS shootout, IBM, April 2011, https: //www.ibm.com/developerworks/java/library/j-paasshootout/, Accessed in date May 2012.
[21] Y. Zhang, Z. Wang, B. Gao, C. Guo, W. Sun, and X. Li, “An effective heuristic for on-line tenant placement problem in SaaS,” in Proc. of IEEE ICWS 2010, Miami, FL, USA, July 2010.
[8] R. Buyya, High Performance Cluster Computing: Architectures and Systems, Volume 1. Prentice Hall, USA, 1999.
[22] A. Perera and R. Linton, ESB Performance Testing, http:// esbperformance.org/, Accessed in date May 2012.
[9] V. Cardellini, E. Casalicchio, M. Colajanni, and P. Yu, “The state of the art in locally distributed web-server systems,” ACM CSUR, vol. 34, pp. 263–311, June 2002.
[23] V. Stantchev, “Performance evaluation of cloud computing offerings,” in Proc. of ADVCOMP 2009, Sliema, Malta, October 2009.
[10] A. Iosup, N. Yigitbasi, and D. Epema, “On the performance variability of production cloud services,” in Proc. of IEEE/ACM CCGrid 2011, Newport Beach, CA, USA, May 2011.
[24] A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema, “Performance analysis of cloud computing services for many-tasks scientific computing,” IEEE TDPS, vol. 22, pp. 931–945, June 2011.
[11] A. Nadalin, C. Kaler, R. Monzillo, and P. Hallam-Baker, Web Services Security: SOAP Message Security 1.1, OASIS, February 2006.
[25] C. Werner, C. Buschmann, and S. Fischer, “Compressing soap messages by using differential encoding,” in Proc. of IEEE ICWS 2004, San Diego, CA, USA, July 2004.
[12] M. Juric, I. Rozman, B. Brumen, M. Colnaric, and M. Hericko, “Comparison of performance of Web services, WSSecurity, RMI, and RMI-SSL,” Journal of Systems and Software, vol. 79, pp. 689–700, May 2006.
[26] W3C, XML Binary Characterization Working Group, http: //www.w3.org/XML/Binary/, Accessed in date May 2012.
725
