Architecture and protocol for intercloud ... - Semantic Scholar

Information Sciences xxx (2013) xxx–xxx

Contents lists available at SciVerse ScienceDirect

Information Sciences journal homepage: www.elsevier.com/locate/ins

Architecture and protocol for intercloud communication Jaime Lloret a,⇑, Miguel Garcia a, Jesus Tomas a, Joel J.P.C. Rodrigues b a b

Instituto de Investigación para la Gestión Integrada de Zonas Costeras, Universidad Politècnica de Valencia, Spain Instituto de Telecomunicações, University of Beira Interior, Portugal

a r t i c l e

i n f o

Article history: Available online xxxx Keywords: Protocol Architecture Interclouds Internetworking between clouds

a b s t r a c t A cloud does not have infinite computational and storage resources in its infrastructure. If it saturates, it will not be able to satisfy new requests for service allocations sent by its customers. Clouds should interrelate through networking protocols in order to provide scalability, efficiency and flexibility by using the services and the computational and storage resources of the infrastructures of other clouds. In this paper we describe an architecture and protocol that allows exchanging information, data, services, computing and storage resources between all interconnected clouds. It is highly scalable and permits to add new clouds easily, while tries to balance the load of the nodes involved in the intercloud communication. Our protocol design includes node discovery, authentication and fault tolerance. We show the protocol operation and provide the performance results in a controlled test bench. A comparison of our architecture and protocol with other published intercloud architecture proposals shows the benefits of implementing this new architecture in the real world. 2013 Elsevier Inc. All rights reserved.

1. Introduction Cloud computing is an infrastructure that is able to deliver different levels of services, by using many type of platforms and sharing the resources inside of it, to customers outside the cloud (over the Internet). The offered services can be dynamically configured (via virtualization or other approaches) and delivered on demand to the external customers, and it is massively and dynamically scalable. It allows a location/device/host independent computing in which it does not matter where is placed the information and where the computation and processing is taking place, so information and computing is available anywhere and anytime in the Internet. Cloud computing must be distinguished from grid computing. Although clouds and grids share many common features, such as architecture and technology, they differ in several aspects such as security, programming model, business model, compute model, data model, and type of user applications [16]. One of the main objectives of cloud computing is to provide several standalone services, that currently exist on a personal computer, through a simple web browser [2]. In the very beginning of the appearance of cloud computing, there were few services available. But, along the years this number is increasing and a wide range of services is appearing [28]. There are several works that explain in detail the most important concepts associated with cloud computing and analyze the layered model of cloud computing, which is based on five layers: Cloud Application Layer, Cloud Software Environment Layer, Cloud Software Infrastructure Layer, Software Kernel Layer and Hardware/Firmware Layer [9].

⇑ Corresponding author. Tel.: +34 609549043. E-mail addresses: [email protected] (J. Lloret), [email protected] (J.J.P.C. Rodrigues).

(M. Garcia),

[email protected] (J. Tomas), [email protected]

0020-0255/$ - see front matter 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ins.2013.05.003

Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003

2

J. Lloret et al. / Information Sciences xxx (2013) xxx–xxx

Cloud computing has several advantages and drawbacks, which provokes proponents and detractors. On one hand, providing services to end-users using Internet helps to improve small and medium enterprises [24]. These companies may provide more cloud-based services and may take larger potential market. In addition, final users will only pay for the used services without having to worry about purchasing the hardware or software required to perform these tasks. Therefore, their potential only depends on their intellectual capabilities and not on their infrastructure. On the other hand, the main observed problem is the high grade of centralization from the organizational and architecture perspective. The centralization of resources was abandoned due to the improvements made by other architectures. Moreover, there are also other problems, for instance: the dependence of the applications proposed by the companies, the services can solve the needs of many end users, etc. As we can see both sectors are partly right, for that reason in Table 1 we see the main advantages and drawbacks of the services offered by cloud computing [26]. One issue to take into account in cloud computing is its architecture [1]. In most cases it is a centralized architecture with all the benefits and drawbacks of these architectures. From the viewpoint of network topology is not debatable that the centralization of many resources over Internet is a problem, but it lacks of fault tolerance and causes a bottleneck. One of the most important problems in the cloud computing is the security. It is being studied widely by the scientific community. Subashini and Kavitha explain that the security issues reduce the growth of cloud computing. In [31], they characterized the security threats by the cloud computing service delivery. In order to avoid the security weakness in cloud computing, Zissis and Lekkas propose to introduce a trusted third party solution based on cryptography, specifically Public Key Infrastructure operating in concert with Single-Sign-On and LDAP (Lightweight Directory Access Protocol), to ensure the authentication, integrity and confidentiality of the data and the communication [37]. There are other techniques to increase the security of cloud computing such as using virtualization because it can protect both the integrity of guest virtual machines and the cloud infrastructure components [23]. Many clouds coexist in Internet (Google, IBM, Microsoft, etc.), each one performing its own protocol. There are also appearing new service platforms to take advantage of these clouds (Microsoft Azure, Joyent Accelerators, Google App Engine, Sun Project Caroline, etc.). Some governments are taking advantage of cloud computing benefits [34], and some institutions, such as the University of Westminster, make use of its flexibility and let them to save costs [32]. Each cloud could have different logical topology, although the devices could be in the same physical place. From the business point of view, cloud computing provide many strengths and opportunities [24], so economic benefits are also obtained. An internetworking between clouds will also increase the benefits to the customer and decrease the economic costs of each cloud because more services and resources will be available to the customers. Therefore, in this paper we present a new architecture and protocol for intercloud communication which will allow sharing resources, services or data between clouds. It is highly scalable and permits to add new clouds easily, while tries to balance the load of the nodes involved in the intercloud communication. Our protocol design includes node discovery, authentication and fault tolerance. The proposed intercloud architecture satisfies the following objectives: – – – – –

Each cloud should be able to work and offer its services without any dependence with other clouds. Resources, services and data are shared through the intercloud architecture. The intercloud architecture is scalable and able to add new clouds. The availability of the resources, services and data should not depend on the customer´s applications. The architecture should be able to support the load given by the traffic exchanged between clouds.

The rest of the paper is structured as follows. Section 2 describes some related works on internetworking between clouds. Section 3 discusses the main design issues to take into account for intercloud communication. Our architecture is described in Section 4. Section 5 shows the analytical models of the proposed architecture. The protocol operation and fault tolerance is detailed in Section 6. Section 7 provides some simulations and a test bench with the performance test of our proposal. A

Table 1 Advantages and drawbacks of cloud computing. Advantages

Disadvantages

Access to the information and services anywhere Service availability and/or web application 24 h/7 days/365 days Accessibility supported by different technologies, such as PDAs, smart phones, laptops, and notebooks Free and paid services according to the user needs Unsaturation hard disk usage on the computer or application used, because they only need a web browser, and Internet Large and dynamic scalability for companies Higher processing and storage capacity without installing local machines Third parties could access all the information Dependence on the online service features (availability, speed, etc.) The difficulty for end-users to find and select trustworthy or reputable cloud providers/services amongst the numerous available ones In most cases it is a centralized architecture Lose control of the handling, storage and use of this information



3

comparison between our proposal and other published intercloud architecture proposals is shown in Section 8. Finally, Section 9 gives our conclusions and future work.

2. Related work There are several works in the literature where connections are established between nodes from different group of nodes or networks, but all of them have been developed to solve specific issues. In 2002, Wierzbicki et al. proposed Rhubarb, which organizes nodes in virtual networks, allowing connections across firewalls/NAT, and efficient broadcasting [33]. Rhubarb system has only one coordinator per group and coordinators could be grouped in groups in a hierarchy. The system uses a proxy coordinator and all nodes inside the network make a permanent TCP connection with the proxy coordinator. Xiang et al. presented a Peer-to-Peer Based Multimedia Distribution Service in 2004 [35]. They propose a topology-aware overlay in which nearby hosts self-organize into application groups. End hosts within the same group collaborate with each other to achieve Quality of Service (QoS) awareness. When a node in this architecture wants to communicate with a node from other group, the information is routed through several groups until it arrives to the destination. Moreover, the strategy of joining networks, or groups of network nodes, have been widely implemented by the authors of this paper in several research areas such as for scaling grids [22], for joining Content Delivery Networks [20] and for connecting Wireless Sensor Networks [21]. Bearing in mind the benefits of cloud computing and the appearance of architectures for market-oriented allocation of resources within Clouds [10], there is an emerging potential for joining clouds to enable the successful adoption of cloud computing. According to Vinton Cerf, a co-designer of the Internet’s TCP/IP protocols and considered a father of the Internet, an important issue of cloud computing is the interoperability between current clouds [18]. In this interview, Cerf indicates that nowadays it is impossible or very difficult to move his data from cloud A to cloud B. So, the next step on cloud computing studies will be aimed to develop protocols and standards to interact multiple clouds in order to take advantage of the offered computing power. We can find very few papers in the literature, where the authors analyze the communication between clouds (intercloud communications). David Bernstein et al. study protocols and formats for cloud computing interoperability on [4]. A set of protocols, called intercloud protocols, and a set of mechanisms are numbered on this paper. Besides, the authors analyse that the best option for addressing is IPv6, these addresses will also aid for other types of services in the intercloud’s process. Finally, the authors indicate that XMPP (Extensible Messaging and Presence Protocol) could be a good option as an interchange message protocol and they pay attention to some use cases. In [5,6], Bernstein and Vij explain how to use the XMPP for intercloud communication. They identify three entities for describing the internetworking between clouds. On one hand, the intercloud gateways would provide mechanism for supporting the entire profile of intercloud protocols and standards. On the other hand, the intercloud root and intercloud exchanges would facilitate and mediate the initial intercloud negotiating process among clouds. They test XMPP in several cases and they conclude that XMPP is flexible and usable, and it could be a core intercloud transport protocol. In [7], the same authors focus their research on a mechanism for such mediation utilizing a resources catalogue approach, defined using the Semantic Web Resource Definition Framework and a common Ontology of Cloud Computing Resources across heterogeneous cloud providers. They propose this catalogue instead of establishing connectivity between each cloud provider in a Point-to-Point manner. None of the proposed protocols provide fault tolerance to the system. Moreover, none of them provides load distribution to the system or Quality of Service. Another work where the authors talk about intercloud communications is published in [11]. In this paper Buyya et al. present CloudSim. It is an extensible simulation toolkit that enables modelling and simulation of cloud computing environments. The main important issue for us is that it allows internetworking between clouds. It is interesting to know how to carry out these communications between clouds because there is not a standard yet. Analyzing the paper, we show that their proposed architecture is based on an idea, where each cloud has a cloud coordinator. Who communicates with other cloud coordinators using a cloud exchange. It acts as a market maker for bringing together service providers and consumers. The authors do not specify the messages involved, but their idea could help us to see another proposal. The same authors of the previous reference detail their intercloud idea in [12]. After presenting the vision, challenges, and architectural elements of interclouds for utility-oriented federation of cloud computing environments, they explain the foremost features of an intercloud framework. Their proposal offer powerful capabilities to address both services and resources management, but their end-to-end combination aims at dramatically improve the effective usage, management, and administration of cloud systems. Finally, their proposal is simulated in order to validate its performance. In this case, the proposed system has a centralized scheme, which makes their proposal lack of scalability. Moreover, the protocol does not have fault tolerance implemented and there is no security (neither node authentication nor secure communication) developed in their proposal. In [14], Celesti et al. explain how to make up an interoperable heterogeneous cloud environment for intercloud communication, where clouds cooperate to accomplish trust contexts and provide new business opportunities such as cost-effective assets optimization, power saving, and on-demand resources provisioning. The authors propose a three-phase cross-cloud federation model. The federation establishment, between a cloud needing external resources and a cloud offering resources, passes through the following three main phases: (a) the cloud looks for other available clouds, (b) the cloud selects, between Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003

4


the discovered clouds, the ones which fit as much as possible its requirements and (c) the cloud establishes a trust context with the selected clouds. The same authors also introduce a security example, which can be implemented on interclouds [13]. They analyze identify management (IdM) and authentication issues in communications between clouds. Then, they analyse IdP/SP model, which is based on the service provider (SP), a system, or administrative domain, that relies on information supplied to it by the identity provider. The identity provider (IdP) is the system, or administrative domain, that asserts information about a subject. For instance, the Identity Provider asserts that an end-user has been authenticated and has been given associated attributes. Using this model, they propose a security intercloud infrastructure. This protocol lacks of several issues. On one hand, the proposed protocol does not have fault tolerance and it does not provide load distribution to the system. On the other hand, although it provides node authentication, there is no secure communication developed in their proposal. There is a Global Intercloud Technology Forum whose aim is to promote standardization of network protocols and the interfaces needed for internetworking between clouds. A white paper with use cases and functional requirements for intercloud computing is published in [17]. As we have seen, there are few works developing a communication system for interclouds. Published works are only descriptions based on very specific ideas or solutions (there is not any design with performance tests or simulations). Moreover, they do not take into account all objectives introduced in the previous section for an intercloud communication. Our intercloud communication proposal is based on an open solution, which does not depend on what kind of cloud is being used or the cloud’s infrastructure, and satisfies all aforementioned objectives. We have gone to one step beyond existing works by presenting the design of an architecture and protocol for intercloud communication and showing its performance. 3. Intercloud design requirements In an intercloud system, nodes from a cloud would use resources, services or data from other clouds (see an example on Fig. 1). The main goal of an intercloud system is to create open interfaces that would govern the exchange and portability of data from a cloud to the others. In order to design these features, there is a necessity of making an efficient protocol to exchange messages between clouds. Connections should be established between the devices that will be the boundary of the cloud. All cloud protocols should be able to be translated to other protocols that must be understandable in other clouds. Moreover, a common protocol would be needed to exchange information between clouds. An internetworking between clouds will provide many benefits to the involved clouds. Some of the main ones are: – There will be more services available for a user than in a single cloud. Normally each cloud provides a service to each user. If the clouds are able to exchange services, a user connected to a cloud can have all the required services. – Service and storage availability will be increased. If several clouds are connected, a user connected to a cloud will access many services. – There could be data or content replication in different clouds. A final user could have his data saved in several clouds in order to keep his/her data safety. – A cloud customer will not need to change its desktop application when it is going to join another cloud. Communication between clouds should be transparent to end user. – Customer desktop applications could use the services and store information and data of any cloud using only one open service. – The probability to find a desired resource will be increased because it could be offered in the customer’s cloud from other clouds.

Fig. 1. An example of information exchange between two clouds to perform a task.


5


Many design requirements should be taken into account to establish connections between devices from different clouds: – Nodes may use different protocols to connect to different clouds. – Nodes may be offering services that are not understandable by nodes from other clouds. – The number of virtual connections to nodes from other clouds should be limited, so there should be a maximum number of outside connections. – Nodes with higher computing or processing capacity should be preferred to establish connections with them. – Services could be used by different types of devices such as hosts, mobiles, and PDAs. There are some current client platforms that are able to join different clouds simultaneously, so they are able to use more than one specific protocol. They can be used as gateways between clouds. But this option is not good to join several clouds due to several reasons. On one hand, customers that should act as gateways between clouds should have many resources and computing capacity in order to join many clouds. Moreover, they should be able to translate protocols between clouds and when a new cloud is added, the software of all gateways should be changed. On the other hand, regular customers should have an updated list of all gateways in its cloud in order to reach the services offered by other clouds. In order to solve this problem, we propose a layered architecture that allows connections between clouds. 4. Architecture description The proposed architecture is implemented over existing clouds. Each cloud offers its own services and protocols. It uses the nodes with high computing capacity and higher bandwidth, now called supernodes, to form the intercloud architecture. Furthermore, these supernodes must be safe for not getting compromised by an attacker. The proposed architecture is formed by 3 layers. The lowest one is the Access Layer, which is formed by the regular nodes of the cloud. The medium layer is called the Distribution Layer, which is formed by Dnodes. The highest layer is called Organization Layer. It is formed by Onodes. Onodes and Dnodes are supernodes. Fig. 2 shows the aforementioned layers. Each cloud has several Onodes, Dnodes, and regular nodes, belonging to their corresponding layer. A cloud could have more than one Onode. Dnodes have logical connections with Dnodes from other clouds. They are in charge of exchanging information between clouds, so they act as an interface between its cloud and the Dnode of other cloud. Onodes are used to organize logical connections between Dnodes from different clouds. An Onode could also be Dnode, but, as we will see later, the more tasks it has as Onode, the fewer tasks it will have as Dnode. Every new Onode should authenticate Onodes from its cloud and with Onodes from other clouds. Every new Dnode must authenticate with an Onode from its cloud in order to become Dnode. Dnodes use Onodes to know which Dnode from other cloud is more suitable to establish connection with. Although there are some security models that can be used for intercloud systems, the most prevalent ones are based on Public Key Infrastructure (PKI) trust models [8]. In our case, we use the Onodes to secure the whole system. Their certifications are signed by the Certificate Authorities (CAs) placed in their cloud. Thus, the authentication process is performed using PKI model. Every supernode must have the information shown in Table 2. CloudID is unique for each cloud and it is received when the node joins the cloud. It allows exchanging information between clouds. CloudID numbers are created automatically using a Onode

Onode

Onode

Organization Layer Dnode

Distribution Layer

Dnode

Dnode

Dnode Dnode

Dnode Dnode

Dnode

Dnode

Dnode

Dnode

Dnode

Reg

Cloud 1 Access Layer

Reg

Reg

Reg

Cloud 2

Cloud 3

Reg

Reg

Connection to the Cloud it belongs to

Logical Connection between Dnode and Onode

Logical connection between Onodes

Logical connection between Dnodes from different clouds

Fig. 2. Architecture proposal.


6

J. Lloret et al. / Information Sciences xxx (2013) xxx–xxx Table 2 Information of each supernode. Parameter

Description

CloudID NodeID ShareID MaxCon Load UpBW DownBW

Cloud identifier Node identifier Type of resources, services and data shared Maximum number of allowed connections with other supernodes Available load for the architecture tasks. It is measured in % Maximum Uplink Bandwidth. It is measured in Kbps Maximum Downlink Bandwidth. It is measured in Kbps

hash operation between the physical address of the first node and a random number. CloudID allows knowing the cloud which an Onode belongs to. NodeID is unique for each node in the same cloud and it is assigned when a node joins the cloud. In order to avoid NodeID duplications we will use a system based on a chain of a random number of four bits and twelve bits obtained from the twelve last bits of the obtained hash of the supernode data. More information about the procedure to avoid duplicity is explained in [19]. Based on the computation capacity of the supernode node, it is able to estimate the maximum number of connections that it could have with other supernodes. The maximum uplink and downlink bandwidth can be obtained from the internet connection or using an independent Internet connection test. The Load is the percentage of available load assigned for the architecture tasks. If a percentage of CPU is not assigned, 100% of CPU usage will be taken. In order to provide compatibility between clouds there should be an agreement between all clouds connected to the architecture where the identifier ShareID should identify the same type of resource, service, or data shared in all clouds. Every ShareID and its descriptions will be saved in a shared database. The available ShareIDs in a cloud are received by the cloud once it becomes a Dnode. When a new Onode appears in the architecture, it must authenticate with Onodes from the same cloud and from other clouds, so the first step is to discover which Onodes are reachable. They can be previously known, because a structured model is used [27], or by Bootstrapping [15]. The Onode sends a Discovery message with its CloudID in broadcast mode to its reachable Onodes. First, it authenticates with selected Onodes of its cloud and agrees its NodeID. This authentication is performed using a public key infrastructure, where the CA is placed inside their cloud. After having received the replies for a limited period of time, it authenticates with selected Onodes of other clouds and sends them, though a Connection message the SharedIDs of its cloud with their descriptions, MaxCon, Load, UpBW, and DownBW. Only Onodes that have not arrived to their MaxCon will reply with their SharedIDs, MaxCon, Load, UpBW, and DownBW. The neighbors Onode selection is taken based on the parameters received by the Onodes (it will be discussed later). Neighbor Onodes will add the new Onode in their Onode neighbor table and will forward this new entry to its neighbors. The new Onode will build its neighbor table with the information obtained from the replies. Then, the neighbors will send their Onode database and the new node will build its Onodes database. Next, Onodes will exchange Keepalive messages in order to know which Onodes are alive. The algorithm procedure when a new Onode joins the architecture is described in Fig. 3. Onode

New Onode Send Discovery message No

Yes

After t, any Discovery reply received?

Any Connection message received?

No

Send Discovery message reply Yes

Yes Send Connection message to selected Onodes

Any discovery message received?

No

Build Onode neighbor table Receive Onode database from its neighbor

Build its Onode database

Exchange Keepalive messages with its neighbors

Fig. 3. Steps followed by a new Onode.


7


When a Dnode joins the architecture, first it must authenticate with an Onode in its cloud. Its first task is to send a Discovery message with its CloudID to the Onodes in its cloud. After the authentication, Onodes in its cloud reply with the SharedIDs of its cloud with their descriptions, and the Onode’s MaxCon, Load, UpBW, and DownBW. This information will allow Dnodes to select the best Onode to connect with and will send its MaxCon, Load, UpBW, and DownBW to it. As a reply of this message the new Dnode will receive its NodeID from the Onode. The Onode builds a Dnode’s table with all Dnodes under its coverage. Then, the Dnode will exchange Keepalive messages with the Onode in order to let them know that it is alive. Dnodes must have connections with Dnodes from other clouds in order to build the intercloud connections. First, it sends a Dnodes request to its Onode that includes its NodeID and its network IP address (it could be v4 or v6). The Onode will add the CloudID and will forward it to other Onodes through the Onode’s network using Shoretst Path First (SPF) algorithm [30]. When an Onode of a different cloud receives a Dnodes request message, it will search the best candidates in its other_clouds_Dnode’s table. To choose the best Dnode will assure as much as possible an intercloud Quality of Service. This selection is based on the parameters MaxCon, Load, UpBW, and DownBW. The Onode will send the IP address of the source Dnode to the best and the second best candidates. Then, they send a Connection message to the source Dnode providing their SharedIDs of its cloud with their descriptions, and their MaxCon, Load, UpBW, and DownBW. The authentication will be performed using a public key infrastructure, where the CA is placed inside their cloud. The second candidate will act as a backup in case of a failure or a lost message. After being neighbors, they will send Keepalive messages in order to let their neighbor know that they are alive. Each Dnode has an other_clouds_Dnode’s table with the information of the Dnodes of other clouds. The algorithm procedure when a new Dnode joins the architecture is described in Fig. 4. Once the node is a Dnode, it is able to receive resource and service discovery requests and it can provide the ones it is offering and the ones offered by the other cloud (because it is connected with Dnodes from other clouds. How the resources, services and data can be discovered is out of the scope of this paper. It can be performed by several ways, such as using a gateway discovery protocol [36] or using multi agent systems [29]. So, any of them can be used in our architecture. 5. Analytical model Let n be the number of clouds sharing resources, services or data, where n e [1, 2K] . In order to fix a maximum number of clouds, we will assume that K = 32 gives a value that is high enough (but it could be higher). This let us know that the CloudID will be a 32-bits identifier. Each cloud i will have one or more Onodes (Oi), which are also Dnodes, several Dnodes (Di) and the rest of servers, devices and clients that belong to the access layer (Ai). The number of nodes involved in the intercloud can be estimated by Eq. (1).

M¼

n X

ðjOi j þ jDi j þ jAi jÞ

ð1Þ

i¼1

New Dnode Send Discovery message No

After t, any Discovery reply received? Dnode Yes Send Connection message to selected Onode

Exchange Keepalive messages to the Onode

No

Send a request to connect with Dnodes of other clouds to its Onode

Any connection message from a Dnodes of other clouds?

Yes

Send it a connection message Exchange Keepalive messages to the Dnode

Fig. 4. Steps followed by a new Dnode.


8


In order to provide fast and stable support to the customers when they are requesting resources, services or data from other clouds, each cloud will have at least one Dnode every b nodes in the access layer and one Onode every a Dnodes in the distribution layer. Let m be the number of nodes in the cloud, Eq. (2) gives the number of nodes that will be involved in the proposed architecture (K).

K ¼ integer

m m þ 1 þ integer þ1 b ab

ð2Þ

Fig. 5 shows the number of nodes involved in the architecture as a function of the number of nodes in the cloud when

a = b = 32, when a = 16 and b = 64, and a = 32 and b = 64. There is not a big difference between the second case and the third case, that is, a variation does not significant affect the quantity of nodes that will be involved in the architecture. We define the suitability parameter (S) as the parameter that let us know which the best node to connect with is. The higher value, the more suitable is that node. It depends on UpBW and DownBW, Load (in %), the number of Connections taken (Taken_Con), and MaxCon (see Table 2). We will not consider nodes which UpBW + DownBW is lower than 256 Kbps. We have considered the root of the sum of bandwidths in order to decrease its weight in the suitability estimation. Then, we define S by expression 3.

S ¼ k1

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðUpBW þ DownBWÞ Load ðMax Con Taken ConÞ

ð3Þ

where 0 6 Taken_Con 6 Max_Con, 0 6 Load 6 100. Load = 0 when the node is overloaded. When Load = 0 or Max_Con = Taken_Con, S = 0, so the node will not be chosen to connect with it. Fig. 6 shows the suitability for each uplink and downlink bandwidth as a function of the number of taken connections when the node has 75% of load and the Max_Con = 64. We have used K1 = 106 in order to obtain S values between 0 and 10. K1 is a parameter that allows us to scale the values obtained to a desired range. When a node with GigabitEthernet network connection has a number of connections greater than or equal to 42, other nodes with lower bandwidth connection could be preferred (if they have few connections). Let G = (O, E) be the Onodes network, where O is the set of Onodes and E is the set of connections between Onodes. o is the number of Onodes in the organization layer (including all clouds), and Oi the number of nodes in the ith cloud. Then, for n P clouds,o ¼ ni¼1 Oi . Let p be the number of connections in the Onodes network (p = |E|). Let Q = [Mij] be the metric matrix to compute the cost to reach each Onode (there are many ways to add costs between nodes: manually, using the round trip time, bandwidth of the interface to reach the other node, maximum throughput of the path between them, etc., so some inputs could be added manually or estimated by the nodes. Our main purpose is to design an automatic system, based on some known parameters, in order to avoid random connections that may not be effective). For each ith Onode, the set of neighbors is defined by CN(i) = {j|{i, j} e E}. Every ith Onode estimates Mij "jth Onode and will build its clouds table by using Dijkstra’s algorithm. Every ith Onode will broadcast its CN(i) and they will forward to their neighbors until the network has converged.

Fig. 5. Number of nodes involved in the architecture.

Fig. 6. Suitability as a function of the number of taken connections.



9

Fig. 7. Time to become neighbors two Dnodes.

Each Onode will estimate the shortest path to the rest of Onodes. Bearing in mind the conclusions obtained in [3], the number of messages transmitted to the network when a new Onode joins the Onode layer will depend on the number of Onodes existing in the organization layer, and the time slots needed to converge will depend on the Onode’s network diameter. The Onode’s network diameter is the minimum number of hops between the farthest nodes. In order to estimate the time needed to establish a connection between Dnodes from different clouds, we will suppose an Onode’s network. The steps followed by the message to become neighbors have been described in the architecture description section. Thus, the time between cloud 1 and cloud 2 can be measured by the following equation.

T DD ¼ ½T Dcloud1Ocloud1 þ ½T Ocloud1Ocloud2 þ ½T Ocloud2Dcloud2 þ ½T Dcloud2Dcloud1 þ ½T Dcloud1Dcloud2

ð4Þ

From Dnodes to Onodes (and vice versa) there is only one hop. Once the connection is established between Dnodes, there is also one hope between them. TOcloud1-Ocloud2 depends on the number of Onodes in the path between Onode of cloud 1 and the Onode of cloud 2. Its maximum value depends on the Onode’s network diameter. Let it be d hops between the Onode of cloud 1 and Onode of cloud 2 and let tp be the average propagation time between all supernodes (Onodes and Dnodes). We will consider the processing time of each supernode approximately 0 because it is very small compared to the propagation time between two supernodes. The time between Onodes is given by the following equation.

T Ocloud1Ocloud2 ¼ d t p

ð5Þ

Adding tp and Eq. (5) in Eq. (4), TD-D can be estimated by the following equation.

T DD ¼ ð4 þ dÞ tp

ð6Þ

Fig. 7 shows the time to become neighbors two Dnodes as a function of the number of hops between the clouds. In a regular mesh network with m nodes, the message complexity reaches up to the order of X (m2). In order to estimate the message complexity of our system, we will take into account that we are using Dijkstra’s algorithm at the Onodes network. Bearing in mind the study provided in [25], the message complexity in Onodes network is of order X(olog (o)), where o is the number of Onodes. If we assume an intercloud system with n clouds and d Dnodes in each cloud, that are connected with all other clouds, then we obtain a message complexity of order X(dn2). Thus, the message complexity will be given by X(olog (o) + dn2). 6. Protocol operation and fault tolerance Our protocol works in the application layer of the TCP/IP protocol suite. So, it works over IP protocol and it does not depend on the network infrastructure of lower layers. The fault tolerance is based on finding the most suitable supernode for each case, so the S parameter will be taken into account to select the backup node. Onodes send Keepalive messages to the Dnodes in their cloud periodically. Moreover, they also send their neighbor Onode’s table and the Dnode’s table to the Dnode with highest S value. When an Onode leaves the architecture voluntarily, it selects the Dnode with highest S parameter in its Dnode’s table and sends it a Change message. It will also send a Disconnection message to the Dnodes of other clouds in its other_clouds_Dnode’s table (because it is also a Dnode). When a Dnode receives a Change message, automatically sends an acknowledgement message in order to let the Onode know that it has been informed. After it, the Dnode will become Onode and will send to the neighbor Onodes a Replace message indicating that it is replacing an Onode. The Onodes will replace the leaving Onode by the new one in their Onode’s table. It will also send a Replace message to the Dnodes in its cloud. The described protocol is shown in Fig. 8. When the Dnode with highest S value does not receive a Keepalive message in a hold time interval, it becomes Onode and sends the Replace message to the neighbor Onodes and to the Dnodes in its Dnode’s table. Dnodes send Keepalive messages periodically to the Dnodes in its other_clouds_Dnode’s table. When a Keepalive message is not received in a hold time interval from a Dnode, the Dnode will be considered failed. In this case, this entry is erased from Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003

10


Dnodes from other clouds

Promoting Dnode

Leaving Onode

Neigbor Onodes

Dnodes of its cloud

Change Disconnection Acknowledgement Replace Replace

Fig. 8. Message flow when an Onode leaves the organization layer.

the other_clouds_Dnode’s table, and the backup Dnode for this cloud will be the main one. If the backup Dnode fails, the Dnode will request a new Dnode for that cloud. The protocol procedure is shown in Fig. 9. 7. Performance tests In this section we first simulate the performance of the Onodes’ network. It will allow us to know how many queries are sent through the Onodes’ network. Then we will test the performance of the architecture in terms of number of Bytes, number of messages and number of broadcasts sent to the network. This test is performed for three scenarios. 7.1. Onode’s network simulation In order to simulate our system, we have chosen a random topology with 56 Onodes and 27 interconnected clouds. The topology has 80 connections and a diameter of 12 hops. It is shown in Fig. 10. The number of interclouds connections in this intercloud varies from 1 (e.g. cloud 5), to 7 (e.g. cloud 19). The main reason to select this topology has been because there are a reasonable number of interconnected clouds, each cloud has different number of connections and the number of hops in the Onodes network is quite high taking into account the number of Onodes. We have selected four clouds (1, 8, 12 and 13) to measure the number of queries generated through the intercloud every slot time when each one of them appears as a new cloud in the intercloud. We have chosen these clouds because they have been placed in different location in the topology, they have different connections with other clouds and their size is different so the impact should be quite different. Fig. 11 shows how the Onodes’ network evolves every slot time and provides the number of queries generated in the intercloud. We can see that the more centric the Onode is in the topology (cloud 13), the faster it converges. Cloud 1 is in the border so it has worst convergence time. We have not observed a big difference in terms of convergence time when we compare a cloud that has 4 connections with other clouds (cloud 8) with another with less connections (cloud 12 has 2 connections). 7.2. Test bench In this test bench, we have used a cloud topology composed by 24 computers (Intel Celeron 2 GHz with 256 MB of RAM). The operating system used in all PCs is Windows XP. All computers have used 100BaseT links to their network. We have used Dnode cloud 1

Onode cloud 1

Onode cloud 2

Dnode with highest S

Dnode with 2nd highest S

Hold Time for Dnode of cloud 2 Hold Time for backup Dnode of cloud 2

Request Dnode cloud 2

Request Dnode cloud 2

1st Selected Dnode 2nd Selected Dnode

Connection Connection Keepalive Keepalive

Fig. 9. Message flow when the both Dnodes from other cloud fail.


11


2 14

3

1

18 27

10 4

15

19

26

13

5

8 16

11

25

6 22

9

20

23

7 17 21

12

Cloud

24

Onode

Fig. 10. Random topology with 56 Onodes and 27 clouds.

Queries per reached cloud

12

Cloud 1 Cloud 8

10

Cloud 12 Cloud 13

8 6 4 2 0

0

1

2

3

4

5

6

7

8

9

10

11

12

Propagation time Fig. 11. Queries per reached cloud.

a Pentium 4 2.4 GHz, and 256 MB of RAM, laptop to gather the information transferred between clouds. Its operating system is Microsoft Windows XP and the application used to capture data has been Sniffer Pro 4.70.04. We have tested three scenarios: 1. The first scenario consists of a cloud where there is only one Onode. There are 23 Dnodes in the cloud. The logical topology is shown in Fig. 12. The application to take measurements starts before the Onode is initialised in the architecture. 10 s later, Dnodes start sequentially every 10 s. 2. The second scenario has 2 clouds with one Onode in each one. There are 11 Dnodes in each cloud. The logical topology is shown in Fig. 13. We start taking measurements before the first Onode is initialised. Firstly, the Onode of the first cloud starts and, 10 s later, the Onode of the second cloud starts too. Then, the 11 Dnodes of the first cloud start sequentially every 10 s, and, 10 s later, every 10 s the 11 Dnodes of the second cloud start sequentially. 3. The third scenario has 3 clouds with one Onode in each one of them. There are 7 Dnodes in each cloud. The logical topology is shown in Fig. 14. We start to take measurements before the Onodes are initialised. Firstly, we started the Onodes of each cloud every 10 s sequentially. After 10 s, we started the 7 Dnodes of the first cloud sequentially, 10 s later we started the 7 Dnodes of the second cloud sequentially and, finally, 10 s later, we started the 7 Dnodes of the third cloud sequentially. In all scenarios we have used the same values of the supernode parameters. The values can be viewed in the Table 3. In next subsections we are going to analyse the consumed bandwidth when our protocol is running, the amount of control messages sent to the network by our system and the amount of broadcast messages in the network when architecture is running. Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003

12


Dnode

Dnode

Dnode

Dnode

Fig. 12. First test bench scenario.

Cloud 1 Dnode

Dnode

Onode Dnode

Dnode

Dnode Dnode

Dnode

Dnode

Cloud 1 Fig. 13. Second test bench scenario.

7.2.1. Bandwidth consumed Fig. 15 shows the bandwidth consumed by the protocol when the first scenario is set up. Sometimes there are several peaks because discovery messages and keepalive messages are joined at the same time. The number of Bytes per second when the network has converged is about 2000 Bytes, but there are peaks between 10,000 and 12,000 Bytes every 60 s approximately. Fig. 16 shows the bandwidth consumed in the second scenario. After 40 s the average number of Bytes in the network is increased due to the request of Dnode connections to Dnodes from other clouds. Moreover, the number of average of Bytes sent to the network is higher because there are more supernodes sending keepalive messages. The average number of Bytes per second is around 3000 Bytes and the peaks are between 10,000 and 12,000 Bytes. Fig. 17 shows the consumed bandwidth in the third scenario. Once the third group has been started, the consumed bandwidth is higher. Despite of one peak (which is around 16,000 Bytes), all peaks are between 12,000 and 14,000 Bytes. It has increased compared to the previous scenarios, because there are more Dnode requests to find Dnodes from other clouds. 7.2.2. Messages Fig. 18 shows the number of messages that there are in the architecture when the first scenario is started. The highest peaks occur every 60 s, after 1 min and 40 s, approximately. It is because of the keepalive messages and the discovery messages. Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003

13


Cloud 1 Dnode

Dnode

Dnode

Dnode

Dnode

Dnode Dnode

Dnode

Onode Dnode

Dnode

Dnode

Dnode

Cloud 2

Cloud 3 Fig. 14. Third test bench scenario.

Table 3 Values of the parameters used on the nodes. Parameter

Value

Downlink bandwidth Uplink bandwidth Keepalive Holdtime Maximum percentage of CPU utilization K1

1024 Kbps 256 Kbps 30 s 60 s 50% 106

16000 14000

Bytes

12000 10000 8000 6000 4000 2000 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 15. Number of Bytes per second in the first scenario.

Fig. 19 shows the number of messages sent in the second scenario. After 1 min and 20 s there are more peaks in the network because of the Dnode requests sent between clouds. The mean value is decreased as the time goes on because the connections between Dnode from different clouds are established. Fig. 20 shows the number of messages sent in the third scenario. After one minute and 20 s, the average number of messages is decreased because the intercloud connections are converging. We can see that the average number of messages in the intercloud is not so different from the second scenario, but in this case there are more peaks and are higher than in the previous scenario.


14

J. Lloret et al. / Information Sciences xxx (2013) xxx–xxx 16000 14000

Bytes

12000 10000 8000 6000 4000 2000 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 16. Number of Bytes per second in the second scenario.

18000 16000 14000

Bytes

12000 10000 8000 6000 4000 2000 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 17. Number of Bytes per second in the third scenario.

140 120

Messages

100 80 60 40 20 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 18. Number of messages per second in the first scenario.

7.2.3. Broadcasts Fig. 21 shows the number of broadcasts per second in the architecture when the first scenario is started. In this case, the number of broadcast messages is higher between 20 and 40 s, but when the intercloud has converged, we found very few broadcasts. Fig. 22 shows the number of broadcasts per second in the second scenario. We can see that the broadcasts are mainly distributed between the 20th second and one minute. It starts after 20 s because Dnodes of every cloud start exchanging messages to find Dnodes from other clouds. This phenomenon always happens when a new cloud joins the system and when Onodes network has converged. After it, broadcasts are sent sporadically to perform a query on a cloud or between clouds. Fig. 23 shows the number of broadcasts per second that are sent to the intercloud in the third scenario. We obtain the similar results than in the second scenario happens: broadcasts are distributed between the 20th second and one minute and 20 s. If we change the values of the parameters in Table 3, we will observe that Downlink bandwidth, Uplink bandwidth, Maximum percentage of CPU utilization and K1 only will affect to the Onodes and Dnodes selection decision (so if we change


15


Messages

80 60 40 20 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 19. Number of messages per second in the second scenario.

140 120

Messages

100 80 60 40 20 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 20. Number of messages per second on the third scenario.

6

Broadcasts

5 4 3 2 1 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 21. Number of broadcasts per second in the first scenario.

these values , but all nodes have the same values, the same selections will be performed). Keepalive and holdtime times will only affect to the spread of the messages along the time, but the shortest they are, the faster convergence the architecture will have. After having observed the performance of our architecture, we will compare it with other published architectures. 8. Architecture comparison In this section we compare the intercloud architecture and protocol described in this paper with intercloud architectures and protocols proposed by other authors. Due to the poor information obtained from existing publications, and because many of the intercloud architectures and protocols are in their first stage, we can only compare them in terms of their features and qualities. Published works are only descriptions based on very specific ideas or solutions. None of them provide an implementation with performance tests or simulations, so the information gathered from the published papers does not let us build a performance comparison. Table 5 shows the features of our proposal compared with the three intercloud systems Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003

16


Broadcasts

5 4 3 2 1 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 22. Number of broadcasts per second in the second scenario.

14

Broadcasts

12 10 8 6 4 2 0 0:00

0:30

1:00

1:30

2:00

2:30

3:00

Time (seconds) Fig. 23. Number of broadcasts per second in the third scenario.

Table 5 Comparison of the interclouds features. Protocol

Bernstein et al. [4–7]

Buyya et al. [11,12] Celesti et al. [13,14]

Our proposal

XMPP + SASL + SAML

Architecture

Uses intercloud Root and intercloud Exchanges Proprietary Protocol Uses Cloud Exchange XMPP + XACML + SAML Uses Cross-cloud Federation Manager Proprietary protocol Two layer architecture, organization layer and distribution layer

Organization Fault tolerant Protocol

Load Node Secure QoS Distribution authentication communication

Distributed

No

No

Yes

Yes

No

Centralized

No

Yes

No

No

Yes

Distributed

No

No

Yes

No

Yes

Distributed

Yes

Yes

Yes

No

Yes

found in the literature. We can see that our proposal has all features included in this table (we are adding now secure communication in our protocol). Looking the aforementioned references, we can observe that the only work that has measured the performance in a real test bench is the one presented in this paper. The other ones do not provide any real measurement or simulation.

9. Conclusions and future work In this paper we have proposed an intercloud architecture and protocol. It is based on three layers: the organization layer, the distribution layer and the access layer. Organization layer is formed by Onodes, which are used to organize connections between Dnodes of different clouds. Dnodes allow transferring resources, services and data between clouds. Access layer is formed by the regular nodes of the cloud. We have defined several identifiers in order to build this structure and we have Please cite this article in press as: J. Lloret et al., Architecture and protocol for intercloud communication, Inform. Sci. (2013), http:// dx.doi.org/10.1016/j.ins.2013.05.003


17

presented the analytical model of the proposed architecture. Neighbor selection method is based on a suitability parameter. This parameter allows balancing the available capacity and load of the nodes and assures the best QoS between clouds. Moreover, when the same service is offered by several clouds, nodes can choose the one that offers high QoS. We have also designed the algorithms and the messages for the proper operation of the architecture. We have also included a fault tolerance procedure to recover node leavings and failures, allowing self-configuration. When the architecture has converged, internetworking between clouds can be done without the need of the organization layer. It is only used to connect Dnodes, so the organization layer will only be needed for fault tolerance purposes. Analytical model shows that it is a feasible architecture and the measurements obtained from the test bench show how the architecture performs in different cases. The bandwidth consumed by our proposal is not too high and proves it viability. We have also seen that increasing the number of clouds, the number of broadcast messages remains approximately constant, so adding more clouds do not overload the system. Our future work will add security to the communication protocol in order to encrypt messages and do not let anybody know the information transmitted between clouds. Moreover, we will improve the protocol in order to add QoS and Service Level Agreements between clouds. Moreover, several features could be added in order to improve the system. Some of them are the following ones: – Let the nodes change neighbor connections based on the suitability. – An appropriate design will let Onodes control the topology of the Dnodes. – Allow dynamic neighbors in order to balance the load when a lot of information must be transferred between clouds.

Acknowledgments This work has been partially supported by Instituto de Telecomunicações, Next Generation Networks and Applications Group (NetGNA), Portugal, by National Funding from the FCT – Fundação para a Ciência e a Tecnologia through the PEst-OE/EEI/LA0008/2011 Project. References [1] M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, M. Zaharia, Above the Clouds: A Berkeley View of Cloud computing. Technical Report No. UCB/EECS-2009-28, University of California at Berkley, USA, February 10, 2009. [2] M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, M. Zaharia, A view of cloud computing, Communications ACM 53 (4) (2010) 50–58, http://dx.doi.org/10.1145/1721654.172167. [3] B. Awerbuch, Distributed Shortest Paths Algorithms, ACM Symposium on Theory of Computing (STOC 1989). 21st Annual ACM Symposium on Theory of Computing, May 14–17, 1989, Seattle, Washington, USA. [4] D. Bernstein, E. Ludvigson, K. Sankar, S. Diamond, M. Morrow, Blueprint for the intercloud – protocols and formats for cloud computing interoperability, in: 4th International Conference on Internet and Web Applications and Services (ICIW ‘09), May 24–28, 2009. Venice, Italy, pp. 328– 336. http://dx.doi.org/10.1109/ICIW.2009.55. [5] D. Bernstein, D. Vij, Using XMPP as a transport in intercloud protocols, in: 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud ‘10), June 22, 2010, Boston, MA. [6] D. Bernstein, D. Vij, Intercloud directory and exchange protocol detail using XMPP and RDF, in: Proceedings of the 6th World Congress on Services (SERVICES’10), Miami, FL, USA, 5–10 July 2010, pp. 431–438. http://dx.doi.org/10.1109/SERVICES.2010.131. [7] D. Bernstein, D. Vij, Using semantic web ontology for intercloud directories and exchanges, in: 11th International Conference on Internet Computing (ICOMP’10), Las Vegas, USA, July 12–15, 2010. [8] D. Bernstein, D. Vij, Intercloud security considerations, in: IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom 2010), Indianapolis, USA, November 30 2010–December 3 2010, pp. 537–544. [9] M. Böhm, S. Leimeister, C. Riedl, H. Krcmar, Cloud computing and computing evolution, in: Cloud Computing Technologies, Business Models, Opportunities and Challenges, CRC Press, 2011, pp. 1–28. [10] R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, I. Brandic, Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility, Future Generation Computer Systems 25 (6) (2009) 599–616, http://dx.doi.org/10.1016/j.future.2008.12.001. [11] R. Buyya, R. Ranjan, R.N. Calheiros, Modeling and simulation of scalable cloud computing environments and the CloudSim toolkit: challenges and opportunities, in: International Conference on High Performance Computing & Simulation (HPCS ‘09), Leipzig, Germany, 21–24 June 2009, pp.1–11. http://dx.doi.org/10.1109/HPCSIM.2009.5192685. [12] R. Buyya, R. Ranjan, R.N. Calheiros, InterCloud: utility-oriented federation of cloud computing environments for scaling of application services, Algorithms and Architectures for Parallel Processing. Lecture Notes in Computer Science 6081/2010 (2010) 13–31, http://dx.doi.org/10.1007/978-3642-13119-6_. [13] A. Celesti, F. Tusa, M. Villari, A. Puliafito, Security and cloud computing: intercloud identity management infrastructure, in: 19th IEEE International Workshops on Enabling Technologies: Infrastructures for Collaborative Enterprises, Larissa, Greece, June 28–30, 2010, pp.263–265. [14] A. Celesti, F. Tusa, M. Villari, A. Puliafito, How to enhance cloud architectures to enable cross-federation, in: Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing (CLOUD ‘10), Indianapolis, USA, November 30–December 3, 2010, pp. 337–345. http://dx.doi.org/ 10.1109/CLOUD.2010.46. [15] C. Cramer, K. Kutzner, T. Fuhrmann, Bootstrapping locality-aware P2P networks, in: IEEE International Conference on Networks 2004, vol. 1, pp. 357– 361. [16] I. Foster, Y. Zhao, I. Raicu, S. Lu, Cloud Computing and Grid Computing 360-Degree Compared. Grid Computing Environments Workshop, (GCE ‘08), Austin, TX, USA, 12–16 November 2008, pp. 1–10. [17] Global Inter-Cloud Technology Forum, Use Cases and Functional Requirements for Inter-Cloud Computing. GICTF White Paper. August 9, 2010. (accessed 10.12). [18] P. Krill, Cerf urges Standards for Cloud Computing. InfoWorld, January 08, 2010. . [19] R. Lacuesta_Gilaberte, L. Peñalver_Herrero, IP addresses configuration in spontaneous networks, in: 9th WSEAS International Conference on Computers. Athens, Greece, July 14–16, 2005.


18


[20] J. Lloret, M. Garcia, D. Bri, J.R. Diaz, Study and performance of a group-based content delivery network, Journal of Network and Computer Applications 32 (5) (2009) 991–999. [21] J. Lloret, M. Garcia, D. Bri, J.R. Diaz, A cluster-based architecture to structure the topology of parallel wireless sensor networks, Sensors 9 (12) (2009) 10513–10544. [22] J. Lloret, M. Garcia, J. Tomas, S. Sendra, A group-based architecture for grids, Telecommunication Systems 46 (2) (2011) 117–133. [23] F. Lombardi, R. Di Pietro, Secure virtualization for cloud computing, Journal of Network and Computer Applications (in press). http://dx.doi.org/ 10.1016/j.jnca.2010.06.008. [24] S. Marston, Z. Li, S. Bandyopadhyay, J. Zhang, A. Ghalsasi, Cloud computing – the business perspective, Decision Support Systems 51 (1) (2011) 176–189. ISSN: 0167-9236, doi:10.1016/j.dss.2010.12.006. [25] J. Moy, OSPF Protocol Analysis, RFC 1245. . [26] L. Qian, Z. Luo, Y. Du, L. Guo, Cloud computing: an overview, Cloud Computing. Lecture Notes in Computer Science 5931/2009 (2009) 626–631, http:// dx.doi.org/10.1007/978-3-642-10665-1_6. [27] R. Ranjan, L. Zhao, X. Wu, A. Liu, A. Quiroz, M. Parashar, Peer-to-peer cloud provisioning: service discovery and load-balancing, Computer Communications and Networks 0 (2010) 195–217, http://dx.doi.org/10.1007/978-1-84996-241-4_1. Part 2. [28] J.J.P.C. Rodrigues, L. Zhou, L.D.P. Mendes, K. Lin, J. Lloret, Distributed media-aware flow scheduling in cloud computing environment, Computer Communications 35 (15) 1819–1827. http://dx.doi.org/10.1016/j.comcom.2012.03.004. [29] V.V. Rybakov, Logic of knowledge and discovery via interacting agents – decision algorithm for true and satisfiable statements, Information Sciences 179 (11) (2009) 1608–1614. [30] S. Sendra, P.A. Fernández, M.A. Quilez, J. Lloret, Study and performance of interior gateway IP routing protocols, Network Protocols and Algorithms 2 (4) (2010) 88–117. [31] S. Subashini, V. Kavitha, A survey on security issues in service delivery models of cloud computing, Journal of Network and Computer Applications 34 (1) (2011) 1–11, http://dx.doi.org/10.1016/j.jnca.2010.07.006. [32] N. Sultan, Cloud computing for education: a new dawn?, International Journal of Information Management 30 (2) (2010) 109–116, http://dxdoi.org/ 10.1016/j.ijinfomgt.2009.09.004. [33] A. Wierzbicki, R. Strzelecki, D. Swierczewski, M. Znojek, Rhubarb: a tool for developing scalable and secure peer-to-peer applications, in: Second IEEE International Conference on Peer-to-Peer Computing (P2P2002), Linöping, Sweden, 2002. [34] D.C. Wyld, Moving to the Cloud: An Introduction to Cloud Computing in Government, IBM Center for the Business of Government, Washington, DC, 2009. [35] Z. Xiang, Q. Zhang, W. Zhu, Z. Zhang, Y. Zhang, Peer-to-peer based multimedia distribution service, IEEE Transactions on Multimedia 6 (2) (2004). [36] D. Xu, K. Nahrstedt, D. Wichadakul, MeGaDiP: a wide-area media gateway discovery protocol, Information Sciences 141 (1–2) (2002) 37–59. [37] D. Zissis, D. Lekkas, Addressing cloud computing security issues, Future Generation Computer Systems 28 (3) (2011) 583–592, http://dx.doi.org/ 10.1016/j.future.2010.12.00.
