2011 IEEE Ninth Ninth IEEE International Conference on Dependable, Autonomic and Secure Computing
A Distributed Application Component Placement Approach for Cloud Computing Environment Zhicheng Jin, Jian Cao*, Minglu Li *Corresponding Author Shanghai Key Laboratory of Scalable Computing and Systems, Department of Computer Science and Engineering, Shanghai Jiao Tong University Shanghai, China
[email protected],
[email protected],
[email protected] collaboration of every participating server in the network. We develop this distributed solution for ACP because we find that a centralized method is hard to be used in cloud because the following reasons: • First is the privacy concern. For some distributed application, the components may well be deployed across several collaborate clouds. A centralized approach generally needs a database to keep the private information of all the servers in the clouds locally. It is very likely that the cloud providers are not willing to share their hardware information to others and make the database impossible to construct. • Second, a centralized approach cannot respond in time to the changes in the system especially with the introduction of virtualization technology. It is essential for a centralized algorithm to keep the information of all hardware in the infrastructure up to date. However, virtualization technology makes infrastructure changes happen more often since a virtual machine’s resource or configuration can be changed at run-time and several operating systems may run on a single server. To keep all these information updated in the central database is hard and consume much network resources and computing power in the system. For a distributed approach that does not need to keep this kind of information, these concerns do not exist and it can get the optimal solution based on most updated information. The ACP problem can be modeled as a Distributed Constraint Optimization Problem (DCOP) [1]. A DCOP problem is a tuple of the following form:ۃǡ ǡ ۄsuch that (for the definition of COP, refer to [1]): • ൌ ሼଵ ǡ ଶ ǥ୬ ሽ is a set of agents(e.g. servers in the network); • ൌ ሼଵ ǡ ଶ ǥ୬ ሽ is a set of disjoint, centralized COPs; each ୧ is called the local subproblem of agent ୧ , and is owned and controlled by ୧ . For the definition of COP (Constraint Optimization Problem) which is actually a centralized version of DCOP, refer to [1]; • ୧ୟ ൌ ሼଵ ǡ ଶ ǥ୩ ሽ is a set of interagent utility functions defined over variables from several different local subproblems ୧ . Each
Abstract—In cloud computing environment, it is common for a multi-component application to be deployed into the IT infrastructure on demand. Although the Application Component Placement Problem or ACP has been studied both in the academy and industry for some years, so far these approaches all rely on a centralized node to collect the information about the network topology and hosts of the targeting environment so that it can make the optimized decision. In cloud environment, computing resources are shared by many applications and the IT infrastructure is dynamically changing, which makes collecting the dynamic information of the environment a more difficult task. In addition, when two or more clouds are involved in the deployment, there may exist privacy concern between different clouds so that a centralized approach cannot be applied. In this paper, we propose a distributed approach to solve the ACP problem. In our approach, we transform the ACP into a Distributed Constraint Optimization Problem or DCOP and solving the ACP becomes solving the DCOP. The detail of the algorithm and experiments are given. Keywords- ACP, Intelligent Agent, DCOP, DPOP
I.
INTRODUCTION
The fast development of new networking technologies and the emergence of virtualization and Cloud has been accompanied by a trend favoring the use of componentbased models for construction of wide-area network applications. Finding the best deployment plan for a distributed application in a wide-area infrastructure had drawn attention from the industry and this problem has been classified as the Application Component Placement problem or ACP. Present studies of ACP mainly focus on the deployment across a cluster of servers or some other hardware such as routers. With the fast deployment of cloud in IT industry, it is necessary to find a solution for ACP in the cloud environment. Present studies of ACP problem are mostly centralized, that is, a central brain knowing everything about the topology of the network and the capabilities of hardware in the network will decide which component will be placed on which server by itself. Finding an optimal solution has been proven to be NP-Hard, and several algorithms have been developed to solve it. By introducing the intelligent agent technology and the DPOP algorithm, we developed a distributed solution to the problem. The decision making process is no longer made by a single brain but by the information exchange and 978-0-7695-4612-4/11 $26.00 © 2011 IEEE DOI 10.1109/DASC.2011.94
489 488
୧ ǣ
ሺ୧ ሻ ՜ express the rewards obtained by the agents involve in ୧ for some joint decision. The agents involved in ୧ have full knowledge of ୧ and are called “responsible” for ୧ . As in a COP, hard constraints are simulated by utility functions which assign 0 to feasible tuples, and -Ğto infeasible ones. Since we can model the ACP problem as a DCOP problem (see the problem formulation part), the algorithms which solve DCOP can be used to solve ACP problem. We used the DPOP algorithm which is introduced in [1] by Adrain Petcu and made some modifications to it to solve ACP. II.
it as a mixed integer program. In his work [9], he also presents a solver using GAMS and CPLEX to automate the decision-making process and is tested in a hundred scale server cluster. His ACP solver can work in many deployment scenarios including fail-over and migration. Their solution’s drawback is that they cannot deploy several components on the same server and they try to solve it using virtualization technology. Colin Low [10] from HP laboratories examines the ACP problem that is somewhat similar with our approach that his approach is also decentralized. His idea inspired by market based approaches to resource allocation where autonomous agents interact through a market to achieve an equitable allocation of resources. Even though his approach is decentralized, there are some differences between our study and his. First, his work is based on the GRID architecture and relies on the UD, where some information such as network topology and components placement information are kept. His algorithm for the deployment of application is decentralized but it still needs some global knowledge of the system while in our approach no such global information is needed. Second, his approach is an incremental method to provide an on-demand computing service and the result may not be optimal. Our solution tries to find an optimal solution based on the current conditions of the system and cannot provide further changes. There is a similar kind of problems which can be referred to as Application Placement Problem or APP. It is defined as: given a set of machines and a set of applications with dynamically changing demands, an application placement controller decides how many instances to run for each application and where to put them, while observing all kinds of resource constraints. This problem has been studied in [11]. This kind of problem differs from ACP mainly because that it is more concerned with a single application and its many instances while ACP is concerned with various different components of the application. Constraint satisfaction and optimization problems had already been widely studied. M. YoKoo et al in [12] lay foundation for Distributed Satisfaction Problem or DisCSP. Later works by Adrain Petcu mainly focus on the Distributed Constraint Optimization Problem (DCOP). In [13], DPOP was first introduced as a good algorithm to solve DCOP. Later publications by Adrain such as [14-16] are all improvements of the DPOP algorithm in various special situations, including H-DPOP, O-DPOP and PC-DPOP. Compared with all these related works, this paper mainly contributes in the following aspects: • Solved the ACP problem in distributed cloud environment where privacy is a concern. • Proposed a non-centralized algorithm to deal with the fast infrastructure changes in the cloud.
RELATED WORK
Multiprocessor scheduling or task assignment in distributed systems share some common properties with ACP problems. They both focus on the problem of how to optimally allocate resources to achieve highest performance. The classic work by Stone studied the optimal assignment of functional modules of a program to an n-processor system using a max-flow min-cut algorithm [2]. Later work on similar or harder problems explored other approaches, including integer programming [3], dynamic programming [4], and greedy type heuristics [5]. The ACP problem can be viewed as a special kind of these problems with its unique requirements such as that network capacity may be a main concern while not the case in a multiprocessor environment. The ACP has been widely studied in the industry. Several existing researches (by Kichkaylo et al. [6], Karve et al. [7]) have shown that the ACP can be defined and resolved via resource consumption optimization on the servers which enables the maximal system throughput. To find an optimal solution of this problem has already been proven to be NPhard. These studies generally provide a model for the problem and the most important work is to provide an algorithm for the ACP planner to actually generate an optimal placement plan. Classical planners are usually centralized, that is, the actual component placement plan is decided by a central brain. KichKaylo in his work [6] proposed a general model for the component placement problem and presented an algorithm for it. Its algorithm is based on planning algorithms developed by the artificial intelligence community which are widely studied and with good efficiency. Its limitation lies in the fact that the algorithm may fail if resources are tight and only happen at the final phase of the four phased algorithm. Zimmerova focuses his work on the component-relation aspect of the problem. In his paper [8], he identifies a lack in existing solutions to the component placement problem that most of them lack sufficient attention of the interaction aspect of the problem and he proposed a solution to the interaction aspect while allowing integration with existing algorithms for solving the non-interaction aspect. The problem has also received a lot of attention from company laboratories. Xiaoyun Zhu et al from HP laboratories, formalize the ACP problem using a mathematical optimization framework and further formulate
III.
PROBLEM FORMULATION
The Application Component Placement problem can be defined as: (i) a topology of a utility fabric consisting of switches, servers, and storage devices with varying capabilities; (ii) a component-based distributed application with requirements for processing, communication and
489 490
storage, to decide which server from the utility fabric should be assigned to each application component. We assume that every server in the problem space can communicate directly or indirectly. However, though the servers have the capability to communicate, we only allow neighbors to communicate directly and prevent other communication during problem solving. Two agents are called neighbor if they share a bid (see later description) for the same component.
following components: (i) a mail server that manages e-mail accounts and can be accessed using (ii) mail client components of differing capabilities, (iii) view mail server components that replicate the mail server as desired, (iv) encryption/decryption components that ensure confidentiality of interactions between the other components, and a (v) key repository that holds cryptographic keys. These components allow the mail application to be deployed in different environments. If the environment is secure and has high available bandwidth, the mail client can be directly linked to the mail server. Various inter-component constraints exist in the application. For example, the Mail Client component and Mail Server component may require a minimum communication bandwidth of 50Mbps. The Key Repository component and the Mail Server component may require a secure link between them. For a detailed description of the application, refer to [6].
A. Application Formulation The software is described by a Software Description Document (SDD), which describes the various aspects of the software and every component of the software. This document is composed of two parts: the Component Description (CD) and the Inter-Component Constraint (ICC). CD mainly describes the various requirements of a component to be placed on a server, including: • The component identifier, which distinguishes every component from the others; • The component’s requirement for hardware capabilities, which includes CPU speed, main memory capacity and speed, minimum storage and etc; • The valid operating systems that the component supports. ICC specifies various constraints between components. Typically, there are two type of constraints between two components: the Communication Bandwidth Requirement and the Secure Communication Channel Requirement. Two components must not be placed at two servers that cannot satisfy the various constraints specified in ICC.
D. Cloud Infrastructure For the ACP problem in cloud environment, there may be not a single cloud involved and many types of hardware. In our research, we assume that every hardware that can be a host for the Agent is a candidate for component deployment target. The network topology in a single cloud may have varying forms. However, no matter what physical form the network may possess, it is a fact that a server in the Cloud can know the address of another server and can communicate directly or indirectly through a router or something else. For servers that exist in different Clouds, they cannot communicate like they are in the same Cloud environment. In order to solve this communication problem, we can actually use a delegate agent in each cloud that forward messages from agents from other clouds to inner cloud and vice versa. The delegate agent‘s address is open to every involved cloud. A typical network is shown in Fig. 1.
B. Component Bid Inspired by the bidding system, the utility for a component to be placed on a server is expressed by the server as a “bid” for the component. A bid for a component is a number which ranges from 0 to 100. The higher the bid’s value, the more “suitable” for the component to be placed on the server. Generally speaking, the bid value is measured by the server’s hardware capacity. The servers with more capable hardware will bid for a component with a higher value and will be more likely to get it. A server gets its bid for a component based on the SDD document. Every agent that resides on its host server will receive a copy of the document from a software distribution center which is actually an agent itself, and by analyzing the content it can make a “bid” for every component. By analyzing the CD part, the agent calculates its utility for the component to be placed on the server. If the utility value is above a threshold, then the server will bid for the component with the utility value as the bidding value.
IV.
PROBLEM MODELING
We model the ACP problem as a DCOP problem. The definition of a distributed ACP is given as follows: Definition 1 (distributed ACP): a distributed ACP is defined as a tuple ܣۃǡ ܤǡ ۄܥܪsuch that:
C. Mail Application A typical multi-component application we use in this paper for example is the mail application introduced in [6]. The mail service is constructed by flexibly assembling the
Figure 1. A typical network topology containing two clouds. Servers can communicate directly or indirectly in the same Cloud while they communicate through delegate if they belong to different clouds.
490 491
ܣൌ ሼଵ ǡ ଶ ǥ୬ ሽ is a set of agents where each agent ୧ resides on its corresponding server ୧ . • ܥൌ ሼଵ ǡ ǥ ǡ ୩ ሽ is the Component Set of the software system. The detail information of a component and the relation between components is described in the Software Description Document. • ܤൌ ሼଵ ǡ ଶ ǥ୬ ሽ. Every ୧ is agent ୧ ’s bid for components and it is the single variable for the agent in our model. The domain of ୧ is the set of components that may be deployed on ୧ , which is a subset of component set ܥ. Value ( ୧ ൌ
୩ ሻ is agent ୧ ’s valuation for component
୩ . Note ୧ always contains the value NULL, which means that ୧ bids for none of the components. ୨ • ܥܪൌ ሼ ଵଶ ǥ ୧ ሽ is the between-agent Hard Constraints. Every Hard Constraint between two agents actually consists of two constraints. The first is the Component-Exclusive Constraint and the other is the Bandwidth Constraint. Componentexclusive constraint ensures that a component is exactly deployed on a single server, that is, no two distinct servers can get the same component at the same time. Bandwidth Constraint forbids the two components to be placed on two servers between which the communication bandwidth does not meet the requirement of the two components. Component-exclusive constraint is modeled naturally, by just assigning - Ğ to the combination of the same components. Bandwidth Constraint is modeled by the related agents themselves. Two agents test their connection bandwidth to see if they meet the requirement. An example of a typical Hard Constraint between two servers ୧ and ୨ is shown in Fig. 2: In this example, we assume that the bandwidth between ୧ and ୨ is 10Mbps, and the minimum bandwidth requirement of components C1 and C3 is 20Mbps. We can see that for ۃ୧ ൌ ͳǡ ୨ ൌ ͳ ۄand ۃ୧ ൌ ͵ǡ ୨ ൌ ͵ۄ, the value is minus infinite, which means that ୧ and ୨ cannot get the same component (C1 or C3) at the same time. We can see also that the tuple ۃ୧ ൌ ͳǡ ୨ ൌ ͵ ۄand ۃ୧ ൌ ͵ǡ ୨ ൌ ͳ ۄis forbidden, this is because the bandwidth between ୧ and ୨ does not meet the requirement of C1 and C3. So this combination will eventually be eliminated during the decision process. The goal is to find a complete instantiation כ ܤfor the variables ୧ that maximize the sum of bids while satisfying all the hard constraints. Fig. 3 shows an example of a DACP model with four agents (servers) and the task distribution system mentioned above. Every agent’s bids for the component are shown beside the agent. For example, agent ଵ bids for two components ଵ ଷ , and its bid values are 15 and 15 respectively. For the agents that share bid for the same component, they are connected by an edge as shown in the figure while others do not (e.g. ଶ and ସ ). •
Aj C1 C2 C3 NULL C1 0 Ğ 0 Ğ Ai C3 0 Ğ 0 Ğ NULL 0 0 0 0 Figure 2. a hard constraint example
Figure 3. a DACP example
V.
DACP-DPOP PROTOCOL
The DACP-DPOP protocol allows automated agents, each representing the interest of a particular server, to act in some degree, concurrently and asynchronously without much global control, in deciding which component to be placed on which server with regard to users’ special needs. The protocol guarantees the privacy of the servers in the cloud, as it need not to upload its private information to a central brain as in a centralized solution. The servers calculate their bid value total by themselves and need only to exchange some calculated results when needed. So a server need not know anything about another server, and privacy is guaranteed. In our protocol, though there is no central brain, there exists an agent called “Dispatcher”, whose main task is to facilitate the whole process. Its main responsibility includes: • Distribute the Software Description Document (SDD) to targeted agents; • Collect the decision result and validate it; • Dispatch the deployment plan. A. Bid Formation This is the first step in the process. It is initiated by user request for the deployment of particular software. The Dispatcher Agent knows about this request and sends the SDD to every server. After receiving the SDD documents, every agent begins executing a local procedure to make a decision to bid for which components and its bid value. We define the component’s performance utility (PU) for a particular
491 492
hardware such as CPU or memory by the following formulation:
distributed environment. For every agent, when finished building bids, it uploads its component set in the bid to Dispatcher Agent. The Dispatcher then executes a procedure to decide each agent’s neighbor and send the result to the corresponding agent. The agent will return a message to the Dispatcher after receiving its neighbor model, and the dispatcher will know the end of the process when received all replies from the agents. Fig. 4(a) shows an example network with four agents and one software with three components. Each agent’s bid for components is shown besides each agent. Under this model, we construct the communication graph as Fig. 4(b) shows. We can see from Fig. 4(b) that only agents with a bid for the same component are connected by an edge (e.g. A1 and A3 have an edge between them while A2 and A4 do not have).
PU = Physical Hardware Performance / minimum (1)
performance requirement
For example, if a server’s CPU speed is 3.0GHZ and the minimum requirement specified in CD is 2.0GHZ, then the CPU’s PU is calculated as: PU(CPU) = 3.0/2.0 = 1.5. For memory and hard disk we use the same calculation. The component’s bid value is obtained by calculating the weighted sum of various PUs: ൌ ୡ୮୳ כୡ୮୳ ୫ୣ୫୭୰୷ כୱ୲୭୰ୟୣ ୱ୲୭୰ୟୣ כୱ୲୭୰ୟୣ (2)
C. DFS Tree Construction and Hard Constraint Formation DFS tree is a special case of pseudo trees which were first introduced to be used in constraint satisfaction problem by Freuder in [2]. The idea is that nodes lying in different branches of the DFS tree become conditionally independent when all their ancestors are removed. For detailed description of DFS trees and related concepts, refer to [1]. Generating DFS trees in a distributed manner is a task that has received a lot of attention, and there are many algorithms available. In our protocol, we adopt a simple algorithm introduced by Adrain in [1] and extend this algorithm to satisfy our needs. When constructing the DFS tree, we need not only the tree structure but also we need to construct all the hard constraints during this phase. A hard constraint between two agents is kept by the descendant, as soon as ancestor-descendant relation is formed, the hard constraint will be built between them. A hard constraint is constructed by examining the components they bid for and the relations between the components. Specifically, the descendant will check: • If they both bid for the same component Ci, set the value to 0 in hard constraint matrix.
Note that PU is always larger than 1, if the calculate PU
