Interconnection of Federated Clouds Massimo Ficco, Luca Tasquier, and Beniamino Di Martino
Abstract Cloud Federation is an emerging computing model where multiple resources from independent Cloud providers are leveraged to create large-scale distributed virtual computing clusters, operating as within a single Cloud organization. This model enables the implementation of environmental diversity for Cloud applications, and overcomes the provisioning and scalability limits of a single Cloud, by introducing minimal additional cost for the Cloud consumer. In such a scenario, it is necessary to leverage specific middleware technologies that enable the effective support of interCloud communication services between Cloud providers. This paper proposes an interconnection solution for Cloud Federations based on publish-subscribe services. The final publication is available at: http://link.springer.com/chapter/10.1007%2F978-3-319-01571-2 29
1 Introduction In the last years, the Cloud services market experienced an extremely rapid growth, as reported in several market research reports, which may lead to severe scalability problems [12]. Therefore, in order to cope with the resource capacity limits of a single Cloud provider, as well as to address the vendor lock-in problem associated to the choice of a single proprietary Cloud solution, the concept of federating multiple heterogeneous organizations is receiving an increasing attention by the key players in the Cloud services market [2]. Cloud Federation extends the scalability of Cloud systems by introducing extreme elasticity in resource management, and hence, enM. Ficco, L. Tasquier, and B. Di Martino Department of Industrial and Information Engineering, Second University of Naples, via Roma 29, 81031 Aversa, Italy. e-mail:
[email protected] e-mail:
[email protected] e-mail:
[email protected]
1
2
Massimo Ficco, Luca Tasquier, and Beniamino Di Martino
abling service requests to be satisfied also in presence of rapid increasing consumer demand and heavy usage of the infrastructure. On the other hand, the main problem in Cloud Federation is to seamlessly and transparently join, from the operational point of view, two or more Clouds, characterized by substantial differences in their organizations. Therefore, a large effort is being spent within the Cloud Computing community to identify open solutions and standards for Clouds interoperability, such as the Open Cloud Manifesto [17], Open Cloud Computing Interface (OCCI) [6], Open Cloud Standards Incubator [16], and the Cloud Data Management Interface (CDMI) [19]. However, although significant benefits may result from the interconnection of multiple Clouds into an uniform way, the open and dynamic nature of these systems coupled with their heterogeneity, makes the communication in inter-Cloud environments an extremely challenging task. Thus, a suitable network virtualization framework is needed to deal with the heterogeneity of the different available Cloud solutions in lack of an uniform data communication model. Specifically, in order to enable what is called in the literature “Cloud Federation”, i.e., empowering the run of distributed applications on resources located across independent Cloud infrastructures, it is essential for all the resources belonging to the cooperating Clouds to be able to communicate with each other, by acting as virtual nodes operating within a single distributed organization. Therefore, in this paper we envision a federation solution based on the well-known publish-subscribe service [8]. We propose an inter-Cloud middleware, supporting the seamless interconnection of distributed resources in different administrative domains.
2 Related Work Cloud Federation is a research topic of great interest. At the state of art, many research works and projects have been devoted to Cloud Computing issues related with the concept of federation. Several vendor-agnostic open-source solutions have been proposed, like LibCloud [21], DeltaCloud [22], jClouds [23], SimpleCloud [15], mOSAIC [5, 11], and MODAClouds [1]. However, they mainly present approaches, abstractions and toolkits for the design and execution of applications on multiple Clouds that aim at supporting system developers and operators in exploiting multiple Clouds and in migrating their applications from Cloud to Cloud as needed. For example, mOSAIC provides a Software Platform and a Cloud Agency for supporting the Cloud developers during the development and deployment of a Cloud application in a Sky computing infrastructure. In particular, it introduces supplementary layers of abstractions that offer uniform access to heterogeneous Cloud resources, independently from the Cloud provider and the technologies it supports, allowing applications to work with abstract Cloud resources. However, only simple pipes and shared memory are provided to support peer to peer communication at application level between distributed components. The LibCloud project is a client library for accessing Clouds, supporting a wide range of providers, implemented in
Interconnection of Federated Clouds
3
Python and Java. Analogously, jClouds, is another stable library implementation targeting portable abstractions for Cloud access. DeltaCloud is an open source project developing an API to deal with several Cloud service providers. It targets REST access and backward compatibility across versions, providing long-term stability for scripts, tools and applications. Being a service-based REST API, it works like an intermediary layer, receiving commands and sending them through specially created Cloud drivers, which provide direct operation mapping with the provider’s API. To the best of our knowledge, ViNe [24] is the only work that addresses connectivity among virtual components distributed across several Cloud administrative domains. ViNe incorporates the necessary mechanisms for recovering full connectivity among ViNe-connected nodes, which could potentially be distributed across several Cloud administrative domains. However, it has not been conceived as a standard solution, but only as an embedded approach for deployment of user-level virtual routers for implementing overlay network communication.
3 Publish-Subscribe Service For Clouds Interconnection The proposed publish-subscribe service-based solution, enables the interconnection of multiple Cloud organizations scattered throughout the network. It provides an integration layer that manages the heterogeneity among the Clouds without the necessity of implementing several interfaces and adapters requiring heavy coding efforts and complex software artifacts. The proposed middleware represents an efficient solution for Cloud interoperability thanks to the decoupling and flexibility offered by the publish-subscribe interaction model, that can be effectively exploited in order to deal with Cloud orchestration and the definition of advanced aggregated services on top of the ones offered by the single federated Cloud organization. The adopted publish-subscribe service offers data-centric communications based on an event-based messaging model. The clients can play two distinct roles: publishers that produce information (e.g., available data sets, runtime resource, storage space) and distribute the associated notifications, and subscribers that consume notifications in which they are interested. A notification is the act of delivering an event. The interest of subscribers in the published notifications are specified in terms of subscriptions (consisting of several predicates). Typically however, a client can take on both publisher and subscriber roles. The brokers are the entities, located on the different sites belonging to the federation, that perform matching and forwarding of messages with respect to a certain event schema (e.g., the advertisement tree or message type) and according to specific error recovery policies. They realize a virtualized network infrastructure overlaid on top of the physical connections, which supports the communication between the publisher and subscriber entities operating within the federation. The main strength of publish-subscribe systems is their ability of ensuring asynchronous communication among the involved nodes that interact in a loosely decoupled way, resulting in an extremely scalable infrastructure for information exchange and distributed workflows.
4
Massimo Ficco, Luca Tasquier, and Beniamino Di Martino
The proposed integration layer articulates according to a well known schema based on two fundamental building blocks that can be implemented through general reusable software solutions or design patterns: • the Adapter resolves the technological heterogeneity of each Cloud with respect to the integration layer according to a “lingua-franca” approach, i.e., a common language whose format and syntax is used for the communications between the Clouds. Each Cloud has an adapter from the common language to its own one. • the Broker is a mediation component between the Clouds, containing the whole integration layer logic. In this manner, it is not necessary to explicitly connect each Cloud with the other ones by an ad-hoc connector, but only plug its Adapter to the Broker, without bringing any change to the integration solutions and/or the Broker itself. Therefore, the integration is transparent from the perspectives of both the individual integrated Clouds and the resulting federated solution. Figure 1 shows the proposed solution, where the front-end of each Cloud is interconnected to an Adapter, and the Adapters are interconnected with the Brokers through a publish/subscribe service (in order to improve the offered scalability and reliability, the federation can encompass more than one Broker). The integration logic within the Broker and the Adapter is structured by means of well known Enterprise Integration Patterns (EIP) [13], resulting into an efficient implementation. In particular, the EIPs are implemented by using a versatile modular integration framework, such the Apache Camel [14]. Such framework allows us to define routing and mediation rules in a variety of domain-specific languages. Moreover, in our realization of the integration layer, we extended this solution by considering a componentoriented approach for the development of the Broker. Therefore, we do not limited ourselves in using only an integration framework, but relied an Enterprise Service Bus (ESB) [18] for designing and implementing the interaction and communication between mutually interacting software components in service-oriented architecture (SOA). Such a solution is typically already equipped with an integration framework, but in addition it provides a complete component container for realizing the integration through component-oriented programming. This facilitates the realization of the Broker, since we can use the available implementation of the components we may need, as well as it allows to include the Adapters already available in the IT market for most of the adopted technologies, such as Web Services, CORBA or RMI. The result is a considerable limitation of the amount of code to make, and the adoption of standard and open-source frameworks. Moreover, it increases the flexibility in the number and kind of information sources to be integrated. In addition, we have considered an asynchronous communication pattern between the Brokers and all the integrated components (realized as an ESB). This allows us to have a scalable and easy solution to dynamically plug new information sources. An event-driven middleware [8] based on a well-known standard, such as Java Service Message (JMS) [20] has been used for this purpose. It provides content-based filtering by means of selector strings expressed with a subset of the SQL92 conditional expression syntax. Therefore, with a publish-subscribe service the Clouds can articulate requests toward the federated components by using notifi-
Interconnection of Federated Clouds
5
Fig. 1 Cloud federation architecture based on a publish/subscribe service.
cations. Moreover, XML is the adopted lingua-franca within the considered Cloud federation and the format used for the notifications exchanged among the Adapters by the adopted publish-subscribe service [4]. Therefore, the adapter implementation can be assumed in terms of EIP: • An incoming Message Endpoint is realized in order to accept messages incoming from the front-end of a given Cloud. There is also a similar, but outgoing, Message Endpoint for sending messages toward the interior front-end of the federated Cloud. Such endpoints are coded based on how the front-end has been realized, e.g., through a socket or a CORBA object. • A Message Translator performs the conversion of the received data in the internal format used by the federated Cloud, to a notification in XML and of performing the dual operation for passing from XML to the internal format. • Two components are present so as to publish the obtained notifications and/or to receive notifications published by the other Clouds within the federation. In such a federation, there are several general key requirements that the adopted publish/subscribe service-based solution should satisfy, including scalability, reliability, performability, and security [7].
4 Conclusions This paper presents a flexible federated Cloud architecture, in which the cooperation between the involved organizations is based on a scalable publish-subscribe middleware for dynamic and transparent interconnection of multiple types of resources and entities. In the future work we aim at designing specific techniques for monitoring performance and security aspects [10, 9, 3].
6
Massimo Ficco, Luca Tasquier, and Beniamino Di Martino
Acknowledgements This work has been supported by mOSAIC (grant FP7-ICT-2009-5-256910), Collaborating Smart Solar-powered Micro-grids (CoSSMic - grant FP7-608806) projects, and PRIST 2009, Fruizione assistita e context aware di siti archelogici complessi mediante terminali mobile, founded by Second University of Naples.
References 1. Ardagna, D., et al.: Modaclouds: A model-driven approach for the design and execution of applications on multiple clouds. In: Modeling in Software Engineering (MISE), 2012 ICSE Workshop on, pp. 50–56 (2012) 2. Buyya, R., Ranjan, R., Calheiros, R.: InterCloud: Scaling of Applications across multiple Cloud Computing Environments. Proceedings of the 10th Int. Conf. on Algorithms and Architectures for Parallel Processing (2010) 3. Casola, V., Cuomo, A., Rak, M., Villano, U.: Security and performance trade-off in perfcloud. Artificial Intelligence and Lecture Notes in Bioinformatics LNCS, 6586, 633–640 (2011) 4. Cilardo, A., Coppolino, L., Campanile, F., Romano, L.: Adaptable parsing of real-time data streams. Conf. on Parallel, Distributed and Network-based Processing pp. 412–418 (2007) 5. Di Martino, B., Petcu, D., Cossu, R., Goncalves, P., M´ahr, T., Loichate, M.: Building a mosaic of clouds. Lecture Notes in Computer Science pp. 529–536 (2011) 6. Edmonds, A., Johnston, S., Metsch, T., Mazzaferro, G.: Open Cloud Computing Interface Core & Models. Available at http://occi-wg.org/about/specification/ (2010) 7. Esposito, C., Ficco, M., Palmieri, F., Castiglione, A.: Interconnecting Federated Clouds by Using Publish-Subscribe Service. Cluster Computing pp. 1–17 (2013) 8. Eugster, P., Felber, P., Guerraoui, R., Kermarrec, A.M.: The many Faces of Publish/subscribe. ACM Computing Surveys 35(2) (2003) 9. Ficco, M.: Security event correlation approach for cloud computing. Journal of High Performance Computing and Networking 7(3) (2013) 10. Ficco, M., Romano, L.: A generic intrusion detection and diagnoser system based on complex event processing. Proceedings - 1st Int. Conf. on Data Compression, Communication, and Processing pp. 275–284 (2011) 11. Ficco, M., Venticinque, S., Di Martino, B.: mOSAIC-based intrusion detection framework for cloud computing. LNCS 7566, 628–644 (2012) 12. Gartner: Forecast: Platform as a Service, Worldwide, 2010-2015, 3Q11 Update. available on line at http://www.gartner.com/id=1792219 (2011) 13. Hohpe, G., Woolf, B.: Enterprise Integration Patterns. Addison-Wesley Professional (2003) 14. Ibsen, C., Anstey, J.: Camel in Action. Manning Publications (2011) 15. Inc, Z.T.: Simple Cloud API. Available at: http://simplecloud.org/ (2012) 16. Incubator, O.C.S.: Cloud Management Initiative. Available at http://www.dmtf.org/standards/Cloud (2011) 17. Open Cloud Manifesto Community: Open Cloud Manifesto. Available at http://www.opencloudmanifesto.org (2009) 18. Rademakers, T., Dirksen, J.: Open-Source ESBs in Action: Example Implementations in Mule and ServiceMix. Manning Publications (2008) 19. SNIA: Cloud Data Management Interface. Available at http://www.snia.org/cdmi (2012) 20. Snyder, B., Bosanac, D., Davies, R.: ActiveMQ in Action. Manning Publications (2011) 21. The Apache Software Foundation: Apache Libcloud Python library. Available on line at http://incubator.apache.org/libcloud (2011) 22. The Apache Software Foundation: Deltacloud API. Available on line at http://deltacloud.apache.org/ (2011) 23. The Apache Software Foundation: jClouds. Available on line at http://code.google.com/p/jclouds (2011) 24. Tsugawa, M., Matsunaga, A., Fortes, J.: User-level virtual network support for sky computing. In: Fifth IEEE Int. Conf. on e-Science, 2009. e-Science ’09., pp. 72–79 (2009)
