Design and implementation of server cluster dynamic load balancing ...

Design and implementation of server cluster dynamic load balancing in virtualization environment based on OpenFlow Wenbo Chen

Hui Li

Qiang Ma

Zhihao Shang

Lanzhou University The School of Information Science and Engineering Lanzhou, Gansu, China

Lanzhou University Lanzhou University The School of Information The School of Information Science and Engineering Science and Engineering Lanzhou, Gansu, China Lanzhou, Gansu, China

Email: [email protected]

Email:[email protected] Email:[email protected] Email:[email protected]

Abstract This paper mainly studies dynamic load balancing in virtualization environment based on OpenFlow, where the Controller collected the server running status through virtual machine management techniques and calculated the aggregated load of the servers according to dynamic feedback aggregated load balancing scheduling algorithm; the OpenFlow switch will forward a client's request to the server whose aggregated load is the smallest, thus minimizing the transaction response time and maximize the throughput of the server. This method can implement flexible, powerful, easy to use, cost-effective higher load balancing products, and can be more reasonable use of resources to realize more effective load balancing.

Categories and Subject Descriptors C.2.1 [Network Architecture and Design]:

General Terms Design, Algorithms, Experimentation

Keywords Load balancing; OpenFlow; load balancing scheduling algorithm; virtualization

1.

Introduction

Currently the network traffic is very large, and is still growing rapidly, so network congestion and server overloading have become a serious problem for every service provider. This problem appears to be particularly important in the coming area of cloud computing and big data [1]. With the rapid growth of visiting traffic and data traffic, the processing ability of the core parts of the network should be increased correspondingly. In this case, if we tackle the problem by abandoning the existing equipment, and purchasing new equipment to upgrade relevant hardware, it will greatly increase the cost and waste resources. So load balancing technology emerges, by distributing a large amount of concurrent access or data traffic to more than one processing equipment to improve a server’s processing ability and reduce the response time of users. This technology is mainly applied to Web server, FTP server, enterprise key application servers and so on [2]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. CFI '14, June 18 - 20 2014, Tokyo, Japan Copyright 2014 ACM 978-1-4503-2942-2/14/06…$15.00. http://dx.doi.org/10.1145/2619287.2619288

Lanzhou University The School of Information Science and Engineering Lanzhou, Gansu, China

A traditional load balancer is very expensive (whose price is generally more than $50k), the policy sets of a load balancer need to be set in advance, its flexibility is very low, and it cannot deal with emergency situations very well. It also requires a specialized administrator to maintain and cannot make flexible strategies according to its own actual network conditions. Since the requests are transmitted by a single hardware load balancer, any failure on the load balancer will lead to the collapse of the entire site [3]. Virtualization technology can help users save money, integration servers, maximize use of the user's infrastructure. Maximize use of the expensive and the limited resource is the greatest driving force of virtualization technology. Data center virtualization wave has put forward new requires to the load balancing equipment. At present, the mainstream products of load balancing products are mostly focused on more features, which made load balancing products become increasingly complex, the applications of the load balancing are severely constrained in enterprise data center virtualization, so design a powerful, easy to use, cost-effective higher load balancing product is extremely urgent. In order to solve the above mentioned problems, we put forward the design and implementation of server cluster dynamic load balancing in virtualization environment based on OpenFlow [4], the architecture is not only low costs but also can provide flexible programmable modules which is used to achieve the policy sets that appropriate for your network in the Controller. With the flexible configuration ability of this architecture, internet application can achieve real-time monitor condition of loading and obtain the corresponding resources in a timely manner. With the arriving of the next generation of OpenFlow technology, an OpenFlow Switch can connect multiple Controllers; so we can improve the robustness of the architecture by using any server which connected to the switch as the Controller [5].

1.1 The technology of traditional load balancing Load balancing technology builds on the existing network structure, and provides an effective and transparent method to expand the bandwidth of network devices and servers, increase throughput, enhance network data processing capability, and enhance network flexibility and availability. It mainly completes the following tasks: to solve network congestion problem, provide service nearby, and achieve location independence. It provides a better user access to quality, improves the server’s responding speed and the utilization of a server’s resources [6]. The first load balancing technology is achieved through DNS, in which it configures a name for multiple IP addresses, so that a client who queries the name got one of the addresses, thus allowing different clients to access different servers and

achieve the purpose of load balancing. DNS load balancing is a simple and effective method, but it cannot distinguish the differences between servers, and can neither reflect the current running status of the servers [7].Once a server breaks down, even during the modification of the DNS settings in time, much time (refresh time) is needed to let it play a role again. Now many load balancer use NAT-based load balancing technology, which has an advantage in cost-reduction and efficiency. Because the address translation was relatively close to the bottom of the network, it is usually integrated in the hardware equipment, and will need to purchase additional hardware. Due to Ethernet itself has flaws in design, traditional load balancing technology mentioned above has failed to achieve the desired effect. The proposal of OpenFlow technology provides a new way for the implementation of load balancing and, it can well solve the deficiency of traditional load balancing.

1.2

OpenFlow technology

Because at the beginning of Ethernet designing, not enough take into account the future development of the network, Ethernet has now shown some disadvantages, especially for doing innovative network experiment on the existing protocol, and the impact of experimental data on the existing data become the biggest obstacle to the experiment. The generation of OpenFlow is to solve these problems, and OpenFlow has now become the representation of SDN and, opens up a new road for network innovation. Unlike traditional switch, OpenFlow switch separates the data forwarding and routing control, they accomplished by OpenFlow switch and controller respectively. OpenFlow switch forwards the packet that has already arrived at the switch according to the flow table, and the controller operates the flow table which in the switch through OpenFlow protocol, so as to achieve the goal of control data forwarding. OpenFlow switch maintains a flow table locally, after OpenFlow switch receives a packet, firstly it lookups flow entry that matches the packet in the flow table, if it can find a matching flow entry, then it performs the operation in flow entry. If there is no match, the packet is forwarded to the controller, and then the controller decides how to deal with the packet. The OpenFlow switch model is shown in figure 1.

founds equipment in the network; generates global network topology; and understands the network running status in real time. It uses module to load program and module that needs to run, and provides programming interface for users through the REST API, and then users can write extension program through custom modules, or REST API. In order to achieve the goal of controlling switch, we add or delete flow in OpenFlow switch through Floodlight.

1.3 Virtualization（OpenvSwitch and kvm） Virtualization technology first appeared in the IBM’s mainframe systems in the 1960s, the system of 370 series increasingly popular in the 1970s, these machines could generate many virtual machine instances on the physical hardware via something called virtual machine monitor program, which can run independently operating system software. With the recent years the widespread deployment of multi-core systems, clusters, grids and cloud computing, the advantages of virtualization technology increasingly reflect in commercial applications, not only reduces IT costs, but also enhances security and reliability of system, the concept of virtualization has gradually penetrated into people's daily work and life. With the popularity of virtualization, more virtual softswitches need to be deployed. But the expensive, closedsource virtual switch makes users overwhelmed, so a group of open source community devotees has developed a multilayered virtualization software switches Open vSwitch, which is a high-quality, multi-layer virtual switch licensed under the open source Apache 2.0 license, developed by the Nicira Networks, the main codes are portable C code. It purpose is to make large-scale network automation can be extended by programming, while still supporting standard management interfaces and protocols. KVM is the virtual machine that based on the Linux kernel. In the KVM model, each virtual machine is a standard process managed by a Linux scheduler, you can start the client operating system in user-space. A normal Linux process has two modes: kernel and user. KVM adds a third mode: client mode (with its own kernel and user mode). Client mode is used to perform all non-I / O client code, while the ordinary user mode support client I / O. KVM virtualization technology supports dynamic migration of virtual machines to achieve load balancing on the available hardware.

2.

Figure 1.The idealized OpenFlow switch model [8] OpenFlow switch doesn't operate the flow table actively, unless the controller sends instructions that operate the flow table to the OpenFlow switch, it will carry out the corresponding operations in accordance with the controller commands. That is to say, all operations of OpenFlow switch are controlled by the controller. The controller used in this paper is Floodlight [9], it is an open source OpenFlow controller which is developed by an open community of developers, it also achieves the control of OpenFlow network;

Related Work

OpenFlow can provide an effective method to control the packet in Ethernet and, it boasts of flexibility in implementing an arbitrary policy in software and decouple policy from the switch itself, hence the numerous studies in load balancing with OpenFlow in recent years. HardeepUppal and Dane Brandon implement and evaluate an alternative load balancing architecture using an OpenFlow switch connected to a NOX controller, which gains flexibility in policy and, costs less. [10] To achieve load balancing features, C++ modules were written that are executed by the NOX controller. The NOX executes the modules when a new flow arrives at the switch. The module defines the load balancing policies and adds new rules into the switch’s flow table. In “OpenFlow based Load Balancing for Fat-Tree Networks with Multipath Support”, Yu Li and Deng Pan presented a load

balancing algorithm and scheme for data center [11]. They argued that static load balancing has limits to schedule large numbers of randomly generated flows in data center networks. By comparison, dynamic load balancing routing can schedule network traffic according to updated traffic statistics on each network device. Their paper presented a load balancer for the fat-tree network with multipath support. They implement a dynamic load balancing routing algorithm in the load balancer, which is adaptive to network traffic, and schedules flows by examining current available bandwidth on all alternative links. The load balancer works as a module of the OpenFlow controlling program called Beacon [12], which runs on the controller host. The dynamic flow scheduling algorithm in the load balancer module determines the path for each flow. In the evaluation, they tested the load balancer using the Mininet network emulator [13] and compared the results of three flow scheduling algorithms: none load balancing (NLB) algorithm, static load balancing (SLB) algorithm and their dynamic load balancing (DLB) algorithm. The results showed that their DLB algorithm works effectively for fat-tree networks and provides network flows with more bandwidth and shorter packet delay on average than the other two algorithms under the same circumstances.

and forwards the packet using actions in the flow entry which is matched it. If has no flow entry that matched up with the packet, the switch will forward this packet to the Floodlight Controller, and then let the Controller determine how the switch forwards the packet. The Controller adds corresponding flow entry to switch through OpenFlow protocol. When the controller decides how to forward the client's request, it determines the server that provides service to client based on the load balancing strategy, the serve using the service in the shared storage via iSCSI protocol to provide corresponding services for client.

3.2

Load balancing strategies

3.2.1

Random algorithm

For each flow that is forwarded to Controller just a moment, the Controller randomly selected a server from the server list to process the client request [14].

3.2.2

3.

Proposed Research

3.2.3

3.1

Summary

(SBLB)

The load balancing architecture described in this paper consists of OpenFlow switch network with a controller and server clusters which is connected to OpenFlow switch network and, a SAN network and storage cluster which provide shared storage for servers cluster. Controller controls the OpenFlow switch at the same time obtains its running status from the servers regularly. As shown in figure 2:

Round Robin

For each flow that is forwarded to Controller just a moment, the Controller selected a server to process the client request according to a certain order [15].

Server-based load balancing algorithm

(1) Dynamic feedback the current server load, mainly collect CPU occupancy rate , memory occupancy rate , and response time ,but this information does not directly represent the load of a node, so it needs a function to convert these indicators, and then got the load of the server : (1) Due to there are different type service, which can have different influence on each part of the node, therefore, we introduced the parameter r, it is used to emphasize different degree of influence that this type service to various parts.

Figure 2.Load balance architecture based on OpenFlow As shown in the figure, each server that provides external service is equipped with a private IP address. The controller is equipped with a public static address for the clients to access. The controller maintains a server address pool and generates current network topology in real-time. From the client's perspective, the server reflects to be an IP-based system image by OpenFlow switch’s service, all servers share this virtual address and through this virtual address, the client can put the whole system as a host system that has independent legal IP address, all access from the clients are sent to the virtual IP address. When the client sends a request to the OpenFlow network, OpenFlow switch uses the packet header information to compare with flow entry in the switch, if the client packet header information matches up with the flow entry, and then increases the corresponding packet counters and byte counters,

(2) Processing ability computation of server node: When we compute load balancing, if the service nodes are heterogeneous, we should not only consider the node load, but also must consider the node's processing capacity. For the processing ability of node ,it is mainly considered from the following several indicators: CPU quantity ,CPU computation ability ,CPU type, memorycapacity ,disk I/O rate ,networkthroughput ,maximum process number . 0,1…,n-1;sum(ki)

=

1;

(2) (3) Calculating weight: When the server load balancing system comes into use, the administrator should set as initial weight to every server, as the server load changes, the weight to be adjusted. Controller runs SNMP module periodically to queries the load parameters and calculate the aggregate load value . And using a sigmoid function to map aggregate load to , if the result is 1, we think the server is in the best condition. We introduce the following formula, depending on the server aggregate load value to adjust its weight.

√

(3)

In the formula, 0.95 is what we want to achieve system utilization, and A is an adjustable coefficient (the default value is 5).If the aggregate load is 0.95 server weight will not change, if the aggregate load is greater than 0.95 , it will become smaller, and if the aggregate load is less than 0.95, it will becomes larger. As can be seen that this is a dynamic feedback formula, it will make the weights adjusted to a stable point, such as a system to achieve the desired state, the weight will be constant. In practice, if it is found the weights of all the servers are less than their , the server cluster should be overloaded, then it need to add new nodes to the server cluster. Conversely, if the weights of all servers are close to , then the current system load is very small. (4) For each new incoming request, it selects node to response the client by the following method. First, find out the server which has the maximum value of weight, and then find out a server , the weight of which is fitted where f is a constant, then add to collection J. After that we calculate the probability for each sever in collection using formula (4). (4) Then the system selects a server based on the probability.

3.3 OpenFlow Based Load Balancer Implementation Through adding flow, controller forwards the packet which accesses to the virtual host to a server whose aggregated load is smallest in the server pool. The specific works as follows: all clients use the virtual IP address which has released external as an address of the request service, when a request arrives OpenFlow switch, then the controller designates one server to serve it according to the set load balancing strategy, then adds the following flow to the switch table, using the selected server’s MAC and IP address modify the request packet’s destination MAC and IP address, after then the switch forward packet to the server. Meanwhile, adding flow for the response of server, this stream modifies the destination IP address of the client request with the packet’s source IP address. Table 1.The flow table entries that the controller sends to the switch

Flow table entry 1

Match

Actions

Flow table entry 2

Match

Actions

SRC_IP=10.0.1.2 DST_IP=10.0.1.10 DST_PORT=80 DST_IP=10.0.1.11 DST_MAC=52:54:00: ed: a3:e9 OUT_PORT=4 SRC_IP=10.0.1.11 DST_IP=10.0.1.2 DST_PORT=80 SRC_IP=10.0.1.10 SRC_MAC=52:54:00:27: b3: c2 OUT_PORT=3

As shown in Table 1 above, in our experiment, The MAC address of the cluster provides external service is 52:54:00:27: b3: c2, IP address is 10.0.1.10, and the port is 80. An user whose IP address is 10.0.1.2 access an request, and the user is

connected to the third port of the switch. Then the load balancing algorithm selected a server whose MAC address is 52:54:00: ed: a3:e9 and IP address 10.0.1.11 responds to the user’s request, and the server is connected in the fourth port of the switch. In this paper, our load balancer consists of three modules as following: virtual machine management, load balancing policy module, Floodlight, as shown in Figure 4. In the experiment, we use libvirt to realize virtual machine management module. This module is responsible for obtaining the running status of each virtual machine in fixed period, including CPU, memory, system I/O and response time of web server, and feeds the result back to the load balancing policy module. The responsibility of the load balancing policy module is when a client request arrives, selects a server for the client to process the client request according to SBLB algorithm. Load balancing policy module will calculates the aggregated load of each server according to the feedback information of virtual machine management module, then select a smallest loading server based on the calculate result, the selected server will process the client’s request. Floodlight module established a connection between the client and the server which serve the client through the way of adding flow to the OpenFlow switch, when a client request arrives, Floodlight module will notify the load balancing policy module, load balancing module will select a server deliver to Floodlight module, Floodlight module will re-establish network connection between the server and client, after then the communicate between the client and the server will be normal, then the whole request process is completed.

Figure 3.Functional units of Load Balancer

4.

Experiment Results

4.1

Experiment environment

The main purpose of this experiment is to compare the performance of using different load balancing strategy in virtualization environment based OpenFlow. As shown in figure 4, using Open vSwitch as OpenFlow switch connects three KVM virtual machines as the Web servers offer the same service external, choosing Floodlight as the controller to connect the Open vSwitch. Three Web servers in the network map to the same virtual address and provide the same service external. We must guarantee the consistency of the servers’ content because they would offer the same web service, so choose another KVM virtual machine as shared storage and choose it as the server-side that could provide disk, which is called iSCSI target. Three KVM servers can make an iSCSI target client that can mount use via the functions of iSCSI initiator, so that content can be published to a shared storage and Web server can access shared storage applications via the iSCSI protocol.

Table 3.Total experiments data when the maximum number of concurrent access is 100 SBLB

Round Robin

Random

Maximum Running Vusers

100

100

100

Total Throughput(bytes)

5808237

4777266

4486043

Average Throughput(bytes/second)

9980

10907

10242

Total Hits

1037

853

801

Average Hits per Second

1.782

1.947

1.829

Total Errors

434

364

365

Average Transaction Response Time

0.476

0.873

0.869

Algorithm

Figure 4.Load balancing architecture in virtualization environment using Open vSwitch Using MySQL as a database where the server to store data. The database restores the various performance parameters of the server real-timely. Floodlight gets data through the database for calculating the current load and adjusts accordingly real-timely. Client uses Windows XP system to install automatically load testing tool Loadrunner11.0 to simulate real users implement concurrent load and real-time performance monitoring.

4.2

Analysising experiment result

In this experiment, we use Loadrunner to simulated 80 and 1000 users respectively to concurrent access to web sites. When the number of users are 80, increasing the user at a rate of 2 users per 15 seconds until the concurrent users reach 80 and then persistence access for 5 minutes, after that decreasing the user at a rate of 5 users per 30 seconds to over the access gradually. The experiment lasted 23 minutes each time. When the number of users are 100, increasing the user at a rate of 10 users per 10 seconds until the concurrent users reach 100 and then persistence access for 5 minutes, after that decreasing the user at a rate of 5 users per 10 seconds to over the access gradually. The table 2 shows when the number of client access increase to 80 gradually, the total experimental data we had obtained by using SBLB algorithms, Round Robin algorithm and Random algorithm respectively.

From the table, we can visually see that compare Round Robin and Random algorithms using the SBLB algorithms have a great advantage, the average transaction response time is the half of other two algorithms and the total throughput of the SBLB algorithms has some advantages. For more visualized to see the advantages of SBLB load balancing strategy, we analyzed the Transaction Response Time of using different load balancing strategies separately. Transaction Response Time consists of network response time and server response time. Figure 5 shows the Transaction Response Time of using three different load balancing strategy separately when the maximum number of concurrent access is 80.

Table 2.Total experiments data when the maximum number of concurrent access is 80 Algorithm

SBLB

Round Robin

Rando m

Maximum Running Vusers

80

80

80

Total Throughput(bytes)

1526832 6

1507659 0

1448294 0

Average Throughput(bytes/second)

11411

11268

10824

Total Hits

2726

2692

2586

Average Hits per Second Total Errors

2.037

2.012

1.933

570

571

570

Average Transaction Response Time

0.342

0.652

0.664

The table 3 shows when the number of client access increase to 100 gradually, the total experimental data we had obtained by using these three algorithms respectively.

Figure 5.Transaction Response Time when the maximum number of concurrent access is 80. Figure 6 shows the Transaction Response Time of using three different load balancing policy separately when the maximum number of concurrent access is 100.

Figure 6.Transaction Response Time when the maximum number of concurrent access is 100. In the figure x-axis represents the data number of 35 times Transaction Response Time which we had extracted from LoadRunner, y-axis represents the server response time in second. As we can see from the figure, comparing the Round Robin and Random algorithm, SBLB algorithm has a great

advantage. Meanwhile we can see with the number of user increasingly, it takes basically the same time when using SBLB to complement each transaction, the curve basically become to be a straight line. While the Transaction Response Time of the other two algorithms jitter larger. Using SBLB load balancing strategy as the server’s dynamic load balancing technology in virtualized environments based on OpenFlow not only saves money, has a flexible strategy set, but also can significantly reduce the response time of the server, provide a better user experience. To illustrate SBLB load balancing strategy compared Round Robin and Random the utilization of network resources is more reasonable, we analyze the Throughput of the entire load balancing architecture when use different load balancing strategies. Throughput refers to the number of client requests that had processed per unit time. Figure 7 shows when the maximum number of concurrent access is 80, the Throughput chart of using three different load balancing strategies. Throughput graph shows the value of the amount of received data from Web Server every second during the scenario running.

Figure 7.Throughput when the maximum number of concurrent access is 80.

use the approach of software defined to achieve different load balancing strategies according to different network experiment. Experimental results show that, compared traditional Random algorithm and Round Robin algorithm, the SBLB load balancing strategy we designed not only can improve the Transaction Response Time, provides a better user experience, but also has greater Throughput, can use the network resources more rational and has achieved a more effective load balancing.

6.

References

[1]

Theophilus, Benson. Understanding data center traffic characteristics[J]. ACM SIGCOMM Computer Communication Review, 2010, 40(1): 92-99.

[2]

Cardellini, V, Rome, Univ, Italy, Colajanni, M,Yu, P, S. Dynamic load balancing on Web-server systems[J]. Internet Computing, IEEE , 2002, 3(3): 28-39.

[3]

Foundry ServerIron Load Balancer. http://www.foundrynet.com/products/webswitches/server iron/.

[4]

T.O.Consortium, “Openflow switch specification/version 1.1.0,”February 2011,http://www.openflow.org/wp/documents/.

[5]

M. Koerner, O. Kao. Multiple service load-balancing with OpenFlow.IEEE HPSR, 2012.

[6]

Ruud, Schoonderwoerd, Owen, E, Holland, Janet, L, Bruten, Leon, J, M, Rothkrantz. Ant-Based Load Balancing in Telecommunications Networks[J]. SAGE, 1997, 5(2): 169-207

[7]

DNS Support for Load Balancing:http://xml2rfc.tools.ietf.org/html/rfc1794

[8]

N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,J. Rexford, S. Shenker, and J. Turner, “Openflow: Enabling innovation in campus networks,” ACM SIGCOMM Computer Communication Review,April 2008.

[9]

Hardeep, Uppal, Dane, Brandon. OpenFlow Based Load Balancing[Z]. Washington: University of Washington,2010.

Figure 8 shows when the maximum number of concurrent access is100, the Throughput chart of using three different load balancing strategies.

[10] Li, Yu, and Deng Pan. OpenFlow based Load Balancing for Fat-Tree Networks with Multipath Support.

Figure 8.Throughput when the maximum number of concurrent access is 100. In the figure x-axis represents the data number of 35 times Transaction Response Time which we had extracted from LoadRunner, and y-axis represents the number of client requests that had processed per unit of time, which is in bytes/sec increments. As can be seen from the figure, when using SBLB algorithm they received the amount of data from Web Server is larger than the other two algorithms and the curve is basically on the rise with the increasing of users.

5.

Conclusion

This paper present a design approach of server cluster dynamic load balancing in virtualization environment based on OpenFlow to solve the load balancing problem that data center network virtualization encountered. Experimental results show that using the approach of OpenFlow, we can achieve flexible, powerful and higher cost-effective load balancing products. OpenFlow technology provides flexibility for the realization of different load balancing strategies, which is convenience us to

[11] Beacon OpenFlow Controllerhttps://OpenFlow.stanford.edu/display/Beacon/ Home. [12] B. Lantz, B. Heller, and N. McKeown. A Network in a Laptop: Rapid Prototyping for Software-Definded Networks. ACM SIGCOMM, 2010. [13] Murata,Y., Inaba,T., Takizawa,H., and Kobayashi H.Adistributed and cooperative load balancing mechanism for largescale P2P systems.SAINT-W, 2006. [14] Adabala,5.,Chadha,V.,Chawla,P.,and Figueiredo, R.From virtualized resources to virtual computing grids: the invigo system. Future Generation Computer Systems,6(21),2005.