2015 2nd International Conference on Information Science and Control Engineering
A private cloud instances placement algorithm based on maximal flow algorithm Kun Qian School of Electronic and Optical Engineering Nanjing University of Science and Technology, Nanjing, China (People‘s Republic of) e-mail:
[email protected]
Jian Guo, Dongxu Han and Gongxuan Zhang School of Computer Science and Engineering Nanjing University of Science and Technology, Nanjing, China(People’s Republic of) e-mail:
[email protected] Abstract—With the continuous development of cloud computing, bigger and bigger data processing run in the cloud computing environment. In generally, large companies or organizations have the demand of big data processing. However, these companies or organizations do not want to expose their business processes and data to third parties (Amazon, Google, etc.). The private cloud could meet their needs. In private cloud, if the instances of disk resource consuming are placed in the same physical node, clearly, the disk I/O bandwidth would be used up quickly that would affect the performance of the entire node seriously. In this paper, we propose an instances placement algorithm FFDL that based on the maximal flow method and would adopt the disk I/O load balancing strategy and reduce competition for the disk I/O bandwidth between instances. We have validated our approach by conducting a performance evaluation study on the open source privates cloud platform-OpenStack. The results demonstrate that our algorithm has immense potential as it provides significant savings in computation time than the Greedy algorithm and demonstrates high potential for the improvement of disk I/O load balancing in the entire private cloud system for the big data processing.
paced in different physical nodes. Obviously, the instances which be used to process big data need higher CPU and disk performance than other kinds of instances. If the instances of disk resource consuming are placed in the same physical node, clearly, the disk I/O bandwidth would be used up quickly that would affect the performance of the entire node seriously. In the private cloud environment, how to place the instances suitably on the physical nodes and to make the entire cloud system to achieve their works steadily and efficiently are the research hot points. Figure 1 shows how the private cloud scheduler is work. In order to meet needs of scientific research and project, I deployed a private cloud system with OpenStack [2] which is a global collaboration of developers and cloud computing technologists producing the ubiquitous open source cloud computing platform for public and private clouds. We use it to create and manage the instances that running big data processing programs. Apart from big data processing, these are also lots of programs need both CPU and disk I/O performance. So, we assume that the instances running in our private cloud system can be divided into CPU-consuming instances and disk I/O-consuming.
Keywords—Private cloud, Instances placement, Disk I/O load balancing, OpenStack, Maximal flow
I.
I found that the competition of disk I/O between instances would lead to a significant decline in the performance of the system, and I consider that the disk I/O is the bottleneck of the system while running a lot of instances to process the big data. In our private cloud, these are some instances (CPU-consuming or disk I/O-consuming) running on different physical nodes. How to place these instances into different physical nodes reasonably is a hard job scheduling problem.
INTRODUCTION
Cloud computing can be classified as a new paradigm for the dynamic provisioning of computing services that usually employ virtual machine technologies for consolidation and environment isolation purposes [1]. Cloud computing delivers an infrastructure, platform, and software (applications) as services that are made available to consumers in a pay-as-yougo model. In industry these services are referred to as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) respectively.
II.
Due to the infrastructure as a Service (IaaS), larger infrastructures were made available to boost the efficiency of scientist computing such as data mining, signal processing, etc. which have big data needed to be analyzed and handled. Both the private cloud and the public use the virtualization to build a cloud computing resource pool that provide virtual machines to users elastically which called infrastructure as a Service (IaaS). There are not much different between private cloud and public cloud in core technology. In private cloud, tasks run at multiple instances (also known as virtual machines), which could be 978-1-4673-6850-6/15 $31.00 © 2015 IEEE DOI 10.1109/ICISCE.2015.22
RELATED WORK AND EVALUATE
The MIT’s researchers [3] compared with the cloud computing and the cluster computing using the Amazon EC2 in the scientific computation and concluded that the EC2 consumes less energy and money than the traditional cluster computing. The cloud system would place virtual machines on whole physical nodes compatibly on the basic of scheduling algorithms which are designed for different application scenarios as well as goal. In conventional cluster, if the System-level process migration mechanism [4] be used, the migration of instances [5] and the Checkpoint/Restart function [6], which needs to spend a huge price and less effective. Arun 59
B N et al. [7] proposed a fault-tolerant method which based on instances migration and system monitoring. However, this method do not take system hardware into account such as CPU load and disk I/O load etc. In the paper [8] Fabien H et al. proposed an integrated scheduling algorithm which utilize migration technology to integrate instances on the physical nodes as little as possible. This algorithm could allocate CPU and Memory resource fair but not disk I/O resource. In the paper [9] Chase et al. have considered to determine the resource demand of each application at its current request load level and to allocate resources in the most efficient way. The network switches are dynamically reconfigured to change the active set of servers when necessary. However, in contrast to [10], the system deals only with the management of the CPU, but does not consider other system resources. Rodero-Merino et al. [11] have proposed an additional layer of infrastructure management in Clouds with the ability to automatically deploy services with multi-instances configurations. However, the system does not optimize the disk I/O load and network communication between instances. WU Song et al. [12] presented an algorithm that uses the network I / O load balancing strategy to reduce the competition of the virtual machine on the network I / O resources. However, this work also does not take the disk performance into account. Cardosa et al. [13] have proposed a method for the problem of powerefficient allocation of instances in virtualized heterogeneous computing environments. However, the approach suits only enterprise environments as it does not support strict SLAs and requires the knowledge of application priorities to define the shares parameter. Other limitations are that the allocation of instances is not adapted at run-time (the allocation is static). III.
node y. Lave
The average disk I/O load of physical node per unit time in the private cloud.
Lt
The disk I/O consumed by tasks, which could be predicted with history data.
A. The objective of the algorithm According to the previous two principles of the algorithm, the aims of the algorithm are: 1) To balance the disk I/O load in the private cloud, the disk I/O load balancing metric is the sum of absolute values which each physical node of disk I/O load deviate from the whole private cloud's average disk I/O load, and that can be represented by the formula (1).The Lave among the formula (1) can be determined by formula (2),
Min N i , j Lave Lave
i
One instance in the set I.
J
The set of physical nodes in the private cloud, |J| is the number of physical nodes.
j
One physical node in the set j.
M
The placement matrix of instances. Mx;y = 1 means that instance x is placed in physical node y.
N
(2)
J
(3)
3) The formula (4) and formula (5) define constraint conditions for formula (1~3) relatively. The formula (4) means that one instance can only be placed in a single physical node. And the formula (5) means that one physical node can contain A instances limitedly, i I , j J , M i , j 0orM i , j 1 (4)
j J , M i , j A
(5)
iI
B. Algorithm Design and Implementation This algorithm has two key points, 1) to calculate the Lt with the historical information of tasks. 2) Utilizing the directed-graph model which set up the link between instances and physical nodes to solve the instances placement matrix M, which based on the strategy of disk I/O load balancing. So, we consider that the entire average disk I/O load is Lt which can be concluded with formula (6), in which the TotalTimes shows how many times the data of disk I/O of instances been recorded and the IOt shows the average values of disk I/O load of each record.
Meanings
The set of instances, |I| is the number of instances.
Ni, j
jJ iI
Table 1 The name rules of FFDL
I
jJ
Max N i , j
In the section, we propose our instances placement algorithm FFDL based on the maximal flow method which focus on the placement algorithm of instances and do not consider other problems such as scheduling order, etc. These are two principles of the algorithm, the first one is making the disk I/O load balancing among all the physical nodes, in which to avoid the load is too high or too low among nodes. Second, to make the total disk I/O as high as possible in the unit of time. The Table1 is naming rules.
The maximum of instances in single physical node.
iI
2) To make the total disk I/O as high as possible in the unit of time (formula 3), which means that, summing up the disk I/O of all the instances in the unit of time. That can be approximately considered as the sum of the disk I/O load of all the physical nodes in the private cloud.
ALGORITHM DEFINITION AND MODEL DESCRIPTION
A
(1)
jJ iI
Lt
The matrix of disk I/O load. Nx;y describes that instance x consumes the disk I/O load of physical
60
i[i,TotalTimes]
IOti
Totaltimes
(6)
The Fig. 2 shows that after deleting all the edges which have no flow, all the in-stances had been placed and we got a feasible matrix M according to the FFDL algorithm.
If every physical node’s disk I/O load is strict balance, we can consider that the disk I/O load of every physical node is Lave. In production environment, to be strict balance is almost impossible. However, making the disk I/O of every physical node close to Lave infinitely is feasible solution. Adjusting the upper of disk I/O load in each physical node and making the upper close to Lave infinitely would lead to the disk I/O load balancing in the private cloud. C. Instances placement model Our FFDL describes the relationship between instances and physical nodes with the directed weighted graph [14], The twodimensional vector (S;R) can be used to show the weight value between instances and physical nodes, S = 1 means that there is an instances and it would consume R disk I/O resource. S shows how many instances the physical node contain and R shows the upper of the physical node. The FFDL algorithm solves the matrix of instances placement (M) making use of the Ford-Fulkerson method [14] which been used to solve the maximum-flow problem [14]. We add a source and a sink to the directed weighted graph relatively, The Fig. 1 is the directed weighted graph with source and sink. After modeling the instances placement, we transform the problem of solving matrix M into founding a maximum flow from Vsource to Vsink. If all the directed edges between the source and the instances are full flows, all the instances can be placed in physical nodes.
Fig. 2 The directed weight graph after deleting all the edges which have no flow IV.
EXPERIMENT AND PERFORMANCE ANALYSIS
Our experiment environment as follow: Hardware: IBM X3500M2 including 10 blade servers, each server has 2.2GHz Intel Xeon quad-core processor X 2, 24GB memory, 500GB SAS disk, 1000MB Ethernet. Software: OpenStack Essex 2012.1 deployed in Ubuntu Server 12.04.1 LTS running KVM as VMM. Instances: VCPU: 2 cores, Memory: 4 GB, Disk: 30 GB. We set up each physical node contains three instances at most which means A = 3, and we also set up eight physical nodes that |J| = 8. We compare FFDL algorithm with Greedy algorithm at three points: The disk I/O load balancing in private cloud. The running time according to the formula (1). The sum of disk I/O load of all physical nodes according to the formula (3).
Fig. 1 The directed weight graph with instances and physical nodes
The Greedy algorithm focus on the balance of the number of instances on physical nodes and it do not take disk I/O load or network load into consider. We run 16 instances on 8 physical nodes, among them, there 8 instances running CPI tasks and 8 instances running B++ tasks. ID 1-8 are B++ instances, ID 9-16 are CPI instances.
The Ford-Fulkerson method iteratively increases the value of the flow. We start with f (Vsource; Vsink) =0 for all Vsource, Vsink 2 V. At each iteration, we increase the flow value in G by finding an augmenting path in an associated residual network. We repeatedly augment the flow until the residual network has no more augmenting paths. The max-flow will show that upon termination.
A. Experimental result and analysis In the Table 2, we recorded the running time of all instances of the two algorithms relatively. The running time of CPI instances regardless of the Greedy or FFDL algorithm, they are basically the same efficiency. Obviously, the running time of B++ instances with Greedy algorithm consume more time than the FFDL algorithm, because the Greedy algorithm only consider the instances number on physical nodes and it cannot take other factors into account that affect system performance. Overall, B++ instances with Greedy algorithm spend almost twice the time than these instances with FFDL algorithm. Meanwhile, in the Table 3, we recorded the average
When we add the residual capacity to the augmenting path and solve a new residual network. The FFDL algorithm iteratively augments the flow along augmenting paths until it has found a maximum flow, in which if and only if it’s residual network contains no augmenting path. Finally, there are no augmenting path can be found and the FFDL algorithm terminates.
61
disk I/O load of each physical nodes during running time. At the first line, we can see that physical nodes 5-8 have no B++ instances, cause Greedy algorithm only focus on the balance of the number of instances on physical nodes, and almost all of the physical nodes have reached the upper of disk I/O. We can see that the total disk I/O load of the FFDL algorithm has reached 1000.06MB/s, and the total disk I/O load of the Greedy algorithm has only 513.99MB/s.
especially at the environment of big data processing. The instances placement strategy must be considered, otherwise, the performance of entire private cloud would be affected obviously. V.
In order to meet the needs of big data processing in private cloud. Firstly, we need to balance the disk I/O load in the private cloud, the disk I/O load balancing metric is the sum of absolute values which each physical node of disk I/O load deviate from the whole private cloud's average disk I/O load, and that can be represented by the formula (1). Secondly, to make the total disk I/O as high as possible in the unit of time formula (3), which means that, summing up the disk I/O of all the instances in the unit of time. According to the Table 2 and formula (2), we calculated that the Lave = 126.17 in the FFDL algorithm and this value is 64.24 in the Greedy algorithm. According to the formula (1) and Table 3, we can conclude that the Greedy algorithm do not take disk I/O into account, which leads to disk I/O load is unbalance in the entire cloud system. Conversely, the FFDL algorithm had taken CPU load and disk I/O load into consider which makes disk I/O load of each physical is fairly balance.
REFERENCES
Greedy FFDL
Greedy FFDL
345 .12 340 .71
ID2 545 .19 280 .11 ID1 0 343 .90 339 .97
ID3 546 .31 278 .39 ID1 1 350 .28 341 .10
ID4 543 .80 279 .43 ID1 2 350 .01 342 .03
ID5 540 .33 282 .56 ID1 3 347 .15 344 .85
ID6 541 .89 279 .24 ID1 4 347 .46 341 .27
ID7 539 .25 285 .60 ID1 5 346 .78 343 .36
FFDL
1 129 .58 122 .63
2 127 .44 125 .86
3 127 .44 126 .24
4 128 .06 124 .09
Barham, Paul, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield, "Xen and the art of virtualization," ACM SIGOPS Operating Systems Review 37, no.5, pp. 164-177, 2003.
[2] [3]
Pepple K. , Deploying OpenStack. O'Reilly, 2003.
[5]
ID8 545 .67 286 .85 ID1 6 348 .19 346 .69
[6]
[7]
[8]
[9]
Table 3 The average disk I/O load of physical nodes during running time (MB/s) NODE Greedy
[1]
[4]
Table 2 Comparison table of running time between Greedy algorithm and FFDL algorithm (sec.) ID1 546 .09 287 .05 ID9
CONCLUSION
We propose an algorithm FFDL which take disk I/O load balancing as strategy and use Ford-Fulkerson method for reference to solve the instances placement problem of private cloud in big data processing environment. According to our experimental result, the FFDL algorithm can improve the performance of disk I/O consuming instances in private cloud and get better performance at running time and disk I/O load balancing etc. than Greedy algorithm.
5 0
6 0
7 0
8 0
[10]
126 .24
122 .83
124 .09
121 .74
[11]
Finally, we can conclude that the FFDL algorithm has more balanced disk I/O load than the Greedy algorithm, which reached the first aim. At the same time, the total disk I/O load of the FFDL algorithm higher than the Greedy algorithm, which achieves the second aim. Overall, the performance of FFDL algorithm is better than the Greedy algorithm, it is 29.85% averagely. The experimental result shows that the disk I/O would become a bottleneck in private cloud when some instances share the disk I/O in the same physical node,
[12]
[13]
[14]
62
Evangelinos, Constantinos, and C. Hill. "Cloud computing for parallel scientific HPC applications: Feasibility of running coupled atmosphereocean climate models on Amazon’s EC2." ratio 2, no. 2.40, pp. 2-34, 2008. Chao Wang; Mueller, F.; Engelmann, C.; Scott, S.L., "Proactive process-level live migration in HPC environments," High Performance Computing, Networking, Storage and Analysis, 2008. SC 2008. International Conference for , vol., no., pp.1,12, 15-21 Nov. 2008. Clark, Christopher, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul, Christian Limpach, Ian Pratt, and Andrew Warfield. "Live migration of virtual machines." In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation-Volume 2, pp. 273-286. USENIX Association, 2005. Walters, John Paul, and Vipin Chaudhary, "A fault-tolerant strategy for virtualized HPC clusters." The Journal of Supercomputing 50, no. 3 pp. 209-239, 2009. Nagarajan, Arun Babu, Frank Mueller, Christian Engelmann, and Stephen L. Scott, "Proactive fault tolerance for HPC with Xen virtualization." In Proceedings of the 21st annual international conference on Supercomputing, pp. 23-32. ACM, 2007. Hermenier, Fabien, Xavier Lorca, Jean-Marc Menaud, Gilles Muller, and Julia Lawall, "Entropy: a consolidation manager for clusters." In Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, pp. 41-50. ACM, 2009. Chase, Jeffrey S., Darrell C. Anderson, Prachi N. Thakar, Amin M. Vahdat, and Ronald P. Doyle, "Managing energy and server resources in hosting centers." In ACM SIGOPS Operating Systems Review, vol. 35, no. 5, pp. 103-116. ACM, 2001. Pinheiro, Eduardo, Ricardo Bianchini, Enrique V. Carrera, and Taliver Heath. "Load balancing and unbalancing for power and performance in cluster-based systems." In Workshop on compilers and operating systems for low power, vol. 180, pp. 182-195. 2001. Rodero-Merino, Luis, Luis M. Vaquero, Victor Gil, Fermín Galán, Javier Fontán, Rubén S. Montero, and Ignacio M. Llorente, "From infrastructure delivery to service management in clouds." Future Generation Computer Systems 26, no. 8 pp. 1226-1240, 2010. WU S., WANG F., SHI X.H., “Network I/O load based virtual machine placement algorithm in HPC Cloud.” China Science: Information Science (in Chinese). 42.pp.290-302, 2012. Cardosa, M.; Korupolu, M.R.; Singh, A., "Shares and utilities based power consolidation in virtualized server environments," Integrated Network Management, 2009. IM '09. IFIP/IEEE International Symposium on , vol., no., pp.327,334, 1-5 June 2009 Thomas H C, Charles E L, Ronald L R and Clifford S, Introduction to Algorithm. 2nd ed. MIT Press. USA, 2001.
