Received: 25 April 2017
Revised: 18 September 2017
Accepted: 20 December 2017
DOI: 10.1002/dac.3530
RESEARCH ARTICLE
An SDN-based scalable routing and resource management model for service provider networks Mahmud Rasih Celenlioglu1 1 Department of Computer Engineering, Gebze Technical University, Gebze, Kocaeli, Turkey 2
Department of Computer Engineering, Harran University, Sanliurfa, Turkey 3
Department of Electrical and Computer Science, Syracuse University, Syracuse, New York, USA Correspondence Mahmud Rasih Celenlioglu, Department of Computer Engineering, Gebze Technical University, Gebze, Kocaeli, Turkey. Email:
[email protected] Funding information The Scientific and Technological Research Council of Turkey (TUBITAK), Grant/Award Number: 113E253
Mehmet Fatih Tuysuz2
Haci Ali Mantar1,3
Summary Volume of the Internet traffic has increased significantly in recent years. Service providers (SPs) are now striving to make resource management and considering dynamically changing large volume of network traffic. In this context, software defined networking (SDN) has been alluring the attention of SPs, as it provides virtualization, programmability, ease of management, and so on. Yet severe scalability issues are one of the key challenges of the SDN due to its centralized architecture. First of all, SDN controller may become the bottleneck as the number of flows and switches increase. It is because routing and admission control decisions are made per flow basis by the controller. Second, there is a signaling overhead between the controller and switches since the controller makes decisions on behalf of them. In line with the aforementioned explanations, this paper proposes an SDN-based scalable routing and resource management model (SRRM) for SPs. The proposed model is twofold. SRRM performs routing, admission control, and signaling operations (RASOs) in a scalable manner. Additionally, resource management has also been accomplished to increase link use. To achieve high degree of scalability and resource use, pre-established paths (PEPs) between each edge node in the domain are provided. The proposed controller performs RASOs based on PEPs. The controller also balances the load of PEPs and adjusts their path capacities dynamically to increase resource use. Experimental results show that SRRM can successfully perform RASOs in a scalable way and also increase link use even under heavy traffic loads. K E Y WO R D S controller design, OpenFlow, pre-established paths, resource management, software-defined networking, virtualization
1
I N T RO DU CT ION
Over the past few decades, the way we use the Internet has evolved because of the proliferation of wireless networks and mobile devices. Consumers now demand for brand new services and applications at anywhere and anytime. Service providers (SPs) purchase new hardware and software to meet needs of customers. This ends up with more complex networking architecture. Thus, management of such networks becomes complex and costly. For this reason, service provider networks (SPNs) look for network management systems that make the management both simpler and efficient.
Int J Commun Syst. 2018;31:e3530. https://doi.org/10.1002/dac.3530
wileyonlinelibrary.com/journal/dac
Copyright Β© 2018 John Wiley & Sons, Ltd.
1 of 22
2 of 22
CELENLIOGLU ET AL.
Software-defined networking (SDN)1 is an emerging network architecture. It allows network operators to perform disjoint network operations (eg, load balancing and resource management) in a single framework by separating the control and forwarding functions. Briefly, Figure 1 depicts 3 layers of SDN. The infrastructure layer (data plane) is the layer at the bottom. Entities in this layer usually performs actions of forwarding, dropping, and altering packets but not limited to them. Control layer (control plane) is logically centralized and performs decision making on behalf of the entities in data plane. The application layer contains applications and services that use the services provided in control layer. The separation of control and forwarding functions of SDN allows the network control to be programmable, brings intelligence to network with centralized management, simplifies network design and management through open standards, and so on. Therefore, SDN allures the attention of SPNs. Although SDN has many benefits, there are several limitations as well.2 First of all, SDN controller is the only entity that makes decision on behalf of all the underlying switches. As SDN controller makes per-flow basis online decisions, it may become bottleneck in terms of decision making.3,4 In addition, SDN controller has to communicate with all of the underlying switches to keep network state up-to-date, install new rules, update or delete the existing rules, etc. This procedure may also result in signaling scalability issue.5 As a result, using SDN-based network management models in SPNs is challenging because of aforementioned scalability problems. There are several studies in the literature that focus on scalability in control plane. Some of these studies6-8 have distributed control plane where several physical controllers are responsible for the management of a portion of the data plane. Other studies9-12 have hierarchical control plane that has 2 or more levels of controllers, each of which has different responsibilities. There are also some works13-16 that address SDN-based resource management. However, there are only few works, such as in Luo et al,17 Tuncer et al,18 and Wang et al,19 that address both scalability and resource management. In this paper, we describe the design, implementation, and evaluation of our SDN-based scalable routing and resource management model for SPNs. Our model relies on a central controller and several pre-established multipaths (PEPs) between edge nodes. Central controller performs disjoint network operations such as routing, admission control, and resource use by means of the modular design of the controller. Pre-established multipaths between each ingress-egress pair (IEP) make an abstraction, and the complex physical network turns into a simple virtual one. The virtual network consists of ingress, egress switches, and PEPs. The proposed controller performs routing, admission control, signaling, and resource management operations based on the virtual network. To increase resource use, the controller
FIGURE 1 Overview of SDN architecture
CELENLIOGLU ET AL.
3 of 22
adaptively balances load of paths that belong to the same IEP. Additionally, the controller resizes capacities of virtual paths dynamically. Apart from the existing works, we look for an answer whether single controller can achieve routing, admission control, and signaling scalability for SPNs. Second, we propose an adaptive load-balancing method for resource use. Third, we develop another resource use method, namely, PEP resizing, for further improvement. Finally, we provide the implementation details of our work to show that such system can be easily applied. The rest of this paper is organized as follows. Section 2 discusses related works. Section 3 describes design of the proposed model and explains the operations that the controller performs. Section 4 provides implementation details of the proposed model. Section 5 presents the experimental results of the evaluation. Finally, the work is concluded in Section 6.
2
RELATED WORKS
Centralized architecture of SDN has many benefits regarding to efficiency, resilience, virtualization, and etc. Yet central control plane may lead to severe scalability issues depending on dynamic network reconfigurations and size of the network. Several works address either scalability issues of centralized controller6-12 or resource use.13-16 However, only a few works17-19 address both. The works in Tootoonchian and Ganjali6 and Koponen et al7 suggest distributed control plane structure to achieve scalability in control plane. HyperFlow6 aims to decrease controller response time through logically centralized but physically distributed control plane. Controllers are distributed over the data plane. Authors use publish/subscribe model for state distribution among controllers. In other words, whenever a state change occurs, associated controller (publisher) notifies other controllers (subscribers). Onix7 is a control platform runs over distributed servers. It provides control logic that basically allows applications to communicate with network elements. Onix also disseminates network state for consistency among servers. Thus, Onix allows ease of implementation for application developers using Onix API and considering consistency, durability, and scalability trade-offs. ZeroSDN8 addresses to the flexibility of distributed controllers with respect to distribution of control logic. Control logic consists of lightweight control modules that are called as controllers and pushes control logic to switches. This way, they enable local processing and achieve control plane scalability. Aside from distributed control plane approach, works proposed in Hassas Yeganeh and Ganjali9 and Ahmed and Boutaba10 provide hierarchical structure within their control planes to achieve scalability. Kandoo9 has 2 levels of controllers. First level of controllers handles local events. A controller in this level can control one or more switches and handle frequent events. On the other hand, top level logically centralized controller handles rare events that require global network view. In Ahmed and Boutaba,10 network is divided into zones, each of which has a zone manager. A zone manager is responsible for management operations (eg, flow setup) within its domain. Network manager is at the top of zone managers and responsible for network wide operations such as abstract topology calculation, statistic collection, flow management, credential management for applications, and configuration. Unlike previous studies, Lopez-Rodriguez and Campelo11 propose a network architecture for SPs to reduce controller response time and provide routing scalability. In the proposed architecture, incoming flows are classified into 4 categories: (1) intrarouting, (2) Multi Protocol Label Switching (MPLS), (3) QoS, and (4) interrouting. The controller proactively installs rules for intrarouting and MPLS to improve scalability. The controller steps in only for QoS-based routing. The work of Wang et al12 aims to achieve scalability on multidomain, multivendor SDN-based networks. In this study, there are several domains where each of which is controlled by a local controller (LC). There is also a controller at the top, which is called as coordinator controller. Coordinator controller has the global network view and allows different LCs to cooperate with each other. Northbound API is unified to achieve this. While aforementioned works focus on control plane scalability issues, there are also SDN-based resource management works proposed in the literature. Google reports their experience on SDN-based inter data center resource management (B4) in Jain et al.13 In B4, each data center site has a set of controllers. Traffic engineering server on top of site controllers gathers topological data from them and splits the traffic of source-destination site pair over k-shortest paths. The controller in Wang et al14 allocates joint bandwidth for interactive, elastic, and background traffic classes in inter data center networks. The controller forces proportional fairness within the same traffic class to increase resource use. On the other hand, Cammarata et al15 propose a load-balancing algorithm (A4SDN) derived from well-known ant colony algorithm. Each packet and switch is treated as an ant and pheromone, respectively. Ants in A4SDN do not converge to a single optimum path but instead explore the paths with the weakest pheromone. Similar to the A4SDN, Tomovic et al16 also aim to increase resource use through load balancing. In this work, authors modify 3 different routing algorithms
4 of 22
CELENLIOGLU ET AL.
called Shortest Widest Path (SWP),20 Minimum Interference Routing Algorithm (MIRA),21 and, Dynamic Online Routing Algorithm (DORA)22 for best-effort traffic. Each of these algorithms performs routing based on estimated available bandwidth in links. MIRA and DORA are suitable for backbone networks as they take location of ingress and egress switches into account. All the works mentioned up to now mainly focus on either control plane scalability problems or resource management. The works that address control plane scalability have distributed or hierarchical (or both) multicontroller architectures. In these works, controller plane scalability is achieved through cooperation and coordination of multiple controllers. However, there appears controller-to-controller communication overhead. Besides, most of these works also do not provide admission control, signaling scalability, and resource management mechanisms. The works that focus on resource management achieve either load balancing with cost-based routing or adjust traffic split ratios within switches. Load balancing through cost-based routing is not suitable since the controller makes decision per-flow. The controller may become the bottleneck as the number of flows increase. On the other hand, load balancing through traffic splitting in switches requires modification of switches since OpenFlow does not support this currently. There are also works that address both control plane scalability and resource management for SDN-based networks. Luo et al17 propose Adaptive Dynamic Multi-Path Computation Framework (ADMPCF) that computes adaptive multipaths for centrally controlled networks. ADMPCF uses an existing path as long as the path satisfies requirements of a flow request. Otherwise, it establishes a new path by computing and assessing several disjoint multipath finding algorithms in the literature. ADMPCF performs parallel execution of those algorithms to achieve scalability. ADMPCF improves resource use through adaptive multipath establishment based on topology, link cost, application traffic and network state. Tuncer et al.18 propose an SDN-based management and control framework for fixed backbone networks. The proposed framework achieves control plane scalability due to its both hierarchical and distributed architecture. It has local managers (LMs) at the top. Each LM works with one or more LCs. Local managers are responsible for implementing logic of the management applications such as resource use. Local controllers perform sequence of actions based on the configurations given by LMs. Authors also propose an adaptive load-balancing mechanism by adjusting traffic split ratios in switches. The work of Wang et al19 aims to achieve both control plane scalability via distributed control plane and load balancing through switch migration. The proposed work has a set of controllers distributed over switches. A controller is responsible for several switches that reside in the data plane. The proposed work migrates a switch from one controller (source) to another (destination) in case when the source controller load reaches to its maximum capacity. Apart from most of the works in the literature, this study also takes migration efficiency into account and provides a greedy algorithm. The works Luo et al,17 Tuncer et al,18 and Wang et al19 address both control plane scalability and resource management. ADMPCF uses number of algorithms that work in concert with an adaptive fashion to perform global routing and optimum resource allocation. This makes ADMPCF complex in terms of implementation. Besides, ADMPCF achieves control plane scalability through parallel execution of algorithms used in the framework. In other words, control plane scalability of ADMPCF depends on processing power of the system (eg, cluster of servers). ADMPCF also does not perform admission control, and this may cause overuse of resources. Moreover, ADMPCF has partial signaling scalability because of the fact that the controller has to deploy multipath rules to all the switches along paths if existing paths do not satisfy flow constraints. The framework presented in Tuncer et al18 also has several drawbacks. First of all, the proposed framework requires several number of controllers and also managers to achieve control scalability due to both hierarchical and distributed control plane architecture. This may increase capital expenditures (CAPEX) and operating expenses of SPs. Additionally, there appear different communication overheads such as Local Manager Orchestrator (LMO) Local Controller Orchestrator (LCO), LMO-LM, LCO-LC, and LM-LC apart from controller to switch communication, although controller-switch communication scalability is achieved by reducing number of switches per controller. Authors also state that communication overhead in management substrate depends on its topology. Moreover, the proposed framework includes a simple communication protocol, yet the details are not provided. Thus, there may appear integration problems in case of an environment that contains heterogeneous LCs and managers. Finally, the proposed framework requires modification of switches for adjusting traffic split ratios based on a hashing scheme. This increases the deployment cost of the proposed framework. In Wang et al,19 it is unclear that which decision making entity (eg, source controller and external server) performs switch migration operation. In case of source controller, several controllers may perform migration to a single controller at the same time. This may dramatically increase load of the destination controller. Apparently, resource management is crucial for SPNs because of today's dynamically changing network traffic and reduction of costs both for maintenance, capital, and operational expenditures. Software-defined networking is suitable for such needs, but it has control plane scalability issues in terms of routing, admission control, and signaling. Unlike aforementioned works, in this paper, we aim to perform routing, admission control, and signaling in a scalable manner
CELENLIOGLU ET AL.
5 of 22
using centralized controller in SDN-based SPNs. We achieve these objectives owing to the PEPs. Additionally, we propose 2 resource management methods for SPNs to increase resource use. The proposed load-balancing method performs adaptive cost equalization of paths between edge pairs. The proposed path-resizing method adjusts path capacities based on edge pair traffic loads.
3
SCALABLE ROUTING AND RESOURCE MANAGEMENT MODEL
Software-defined networking is an emerging network architecture that allures the attention of ISPs regarding to resilience, virtualization, ease of network resource management, reduction of costs, etc. Software-defined networkingβbased resource management systems typically operate on global network state, and the controller makes optimization regarding to resource use. Although SDN concept seems to be adequate for resource management on its own, a new algorithm is in fact required because of the scalability issues* that the SDN controller can experience. Since the SDN controller is the only entity that makes decision on behalf of all the switches, it may become the bottleneck with the increase in number of flows and switches. To overcome aforementioned limitations of SDN, we exploit PEPs to reduce the heavy workload of the controller. In the proposed scheme, these paths are fixed for certain period (eg, 15 min), and the controller performs routing, admission control, and signaling based on these paths. In addition to the scalable routing scheme, we also propose 2 resource management methods that depend on PEPs. While the first method focuses on load balancing between overused paths and less-used paths, the second method focuses on resizing capacity of PEPs to increase resource use even further. Further details of the proposed methods are presented in this section.
3.1
Architecture
Figure 2 depicts architecture of the proposed model. The layer at the bottom is data plane. Traditionally, data plane of an SPN consists of provider edge and core switches. Provider edge switches are connected to the customer switches. Hence, network traffic arrives at/departs from the network through provider edge switches. In this work, we name the switches that customer traffic arrives at the network are ingress software-defined networking switches (ISSes) and the switches that traffic leaves SPN are egress software-defined networking switches (ESSes). Note that any edge switch can be ISS and ESS at the same time. In SPNs, there are also one or more provider core switches that connect edge switches to each other. These switches solely perform forwarding of packets between edge switches. In this work, we call these switches as core software-defined networking switches (CSSes). Additionally, the layer at the top is the control plane. The proposed controller is responsible for routing, admission control, and resource management operations. It consists of several modules where each of which performs a distinct task. The 2 layers in Figure 2 communicate with each other via a signaling protocol (eg, OpenFlow23 ). In this work, we use PEPs between each provider edge switch pair, namely, IEP, such as ISS1-ESS in Figure 2. Basically, PEPs turn the complex physical network into a simple virtual one. Virtual network consists of edge switches and PEPs. The controller performs routing, admission control, signaling, and resource use operations based on virtual network in a scalable manner. First of all, the controller does not perform routing and admission control operations per flow basis. It performs these operations based on PEPs. Thus, scalable routing and admission control are achieved. Secondly, proposed model achieves signaling scalability owing to aggregation of flows to PEPs. To be clearer, instead of signaling with all the switches along the path of a new flow, the controller makes signaling only with corresponding edge switches. This is because core switches already have path rules in their forwarding tables. Finally, as explained in sections 3.5 and 3.6 in detail, the controller performs load balancing and PEP capacity resizing operations to increase resource use. Table 1 summarizes main acronyms used in this work for clarification purposes.
3.2
Pre-established paths and virtual network
We model our network with a set L of unidirectional links indexed by i and set IE of ingress-egress switch pairs indexed by j. Whole network is denoted as G = (IE, L). Each IEj pair has a set Pj of PEPs indexed by k, where |Pj | β₯ 2. Note that any 2 paths in the network are distinct from each other since Pkπ β 2L . Thus, a PEP may share some of its links with some * Routing-, admission controlβ, and signaling-based scalability issues.
6 of 22
CELENLIOGLU ET AL.
FIGURE 2 Overview of the proposed model
other PEPs of different pairs. We name these PEPs as neighbor paths. Any 2 pair that have neighbor paths are neighbor pairs as well. Unlike most of the works in the literature, the proposed controller does not compute paths per flow request but, instead, establishes PEPs for a certain period such as hours, days, or even weeks. The proposed model requires at least 2 mutually disjoint paths between edge pairs. However, there is no specific upper limit for them. Number of paths of a pair depends on both topology and domain administrator(s). Within the scope of this work, we assume that PEPs are pre-established. We do not have any contribution regarding to multipath computation. Any existing algorithm (eg, k-shortest path algorithm17,24,25 ) that finds multipaths between 2 specific nodes can be used. Selection of paths does not necessarily depend on bandwidth. A network administrator may choose to select paths with lowest latency based on requirements of customers. In the proposed model, we convert a physical link to one or more virtual links to make the network management easier. Number of virtual links of a physical link is equal to the number of paths pass through that physical link. Thus, sum of the virtual link capacities is equal to the capacity of the physical link (π(Li )). A PEP becomes one or more virtual links. The proposed controller calculates initial virtual link capacities for each path, defined as π(Lki ), at first. Value of the π(Lki ) is equal to the division of π(Li ) to the number of paths that Li serves. After determination of virtual link capacities for all paths, the controller sets π(Pkπ ) to the capacity of virtual link, which has a minimum capacity (ie, π(Pkπ ) = min(π(Lki ))). For instance, the link L6 in Figure 2 serves to 2 paths P21 and P12 . Therefore, L6 turns into 2 virtual links, which are L26 and L16 as depicted in Figure 3. Assume that π(L6 ) is 2 Gbps. Thus, both π(L26 ) and π(L16 ) becomes 1 Gbps. The proposed model takes advantage of PEPs to convert the complex physical network into a simple virtual one. The virtual network consists only of IEj and their paths Pkπ . For example, the network illustrated in Figure 2 consists of 3
CELENLIOGLU ET AL.
7 of 22
TABLE 1 Table of acronyms Acronym
Explanation
SRRM
Software-defined networkingβbased scalable routing and resource management model
RASO
Routing, admission control, and signaling operations
SP
Service provider
SPN
Service provider network
PEP
Pre-established multipaths
ISS
Ingress software-defined networking switch
ESS
Egress software-defined networking switch
CSS
Core software-defined networking switch
IEP
Ingress-egress pair
LSDB
Link-state database
PSDB
Path-state database
HMA
Hash-based modulo-π assignment operation
FIGURE 3 Illustration of the virtualized network in Figure 2. CSS, core software-defined networking switch; ESS, egress software-defined networking switch; ISS, ingress software-defined networking switch
edge switches (ie, ISS1, ISS2, and ESS) and 6 core switches (ie, CSS11, CSS12, CSS13, CSS221, CSS222, and CSS223). This network turns into a simpler network as depicted in Figure 3. The simplified network consists only of 3 edge switches. The controller performs routing, admission control, signaling, and also resource management operations based only on this virtual network.
3.3
Controller design
The controller at the top of the network performs routing, admission control, load balancing, and PEP capacity resizing operations for network management purposes. Our controller maintains 2 databases called link-state database (LSDB) and path-state database (PSDB) to store temporary network view. These databases are periodically updated in the background without interfering other operations. Whenever the controller performs a management operation, it accesses to one or both of these databases. As soon as the controller gets the required information, it starts performing the operation. Link state database keeps track of connection map of underlying switches. This part of the database changes only when topology changes. However, link statistics stored in this database are updated periodically. Link information is structurally similar to the OSPF routing information base. Path state database maintains information related to paths that are established between each IEP. Path state database is actually the abstraction of physical network. The controller manages the
8 of 22
CELENLIOGLU ET AL.
network based on the abstract network maintained in PSDB. Some parts of the PSDB are updated right after the update of the LSDB. We will explain these updates later in Section 4. The controller performs routing, admission control, load balancing, and PEP capacity resizing based on the information maintained in LSDB and PSDB. Routing allows a flow to be routed over the network. Admission control prevents network resources from overloading. Load balancing increases the efficiency of resource use by equalizing cost of paths of IEPs. Pre-established path capacity resizing adjusts path capacities based on loads to increase resource use. Obviously, forming control plane with single physical controller is not suitable, as the controller is the single point of failure. In this work, we do not address this issue extensively, as there are proposed works mentioned in Chen et al26 regarding to this issue. A simple solution for our model is that there can be 2 or more controllers within control plane. One of them acts as a master. The other controller, aka backup controller, periodically checks if master controller is alive. In case of failure, backup controller takes over. As the proposed model maintains network state in databases, the backup controller can directly connect to these databases to manage the whole network. Thus, there is a minimal overhead of taking control in case of controller failure.
3.4
Routing and admission control
Basically, routing in SDN occurs as follows. As soon as a request freq for a new flow f comes from a switch to the controller, controller has to compute a shortest path between source and destination nodes. Afterwards, it has to install corresponding rules for f to the associated switches along the path. Therefore, the controller may become bottleneck, as the number of flows and switches increase. Similar to the routing, the controller has to check whether there are enough amount of resources (ie, bandwidth) in the network to accept the request during the admission control process. In our model, we simplify routing and admission control processes using PEPs to provide scalability. In routing, the proposed controller does not compute a path-per-flow request. Instead, it simply identifies the corresponding edge pair IEj for each freq at first. Then, it assigns each f to a pre-established path Pkπ of corresponding pair IEj . Apart from the routing operation, our controller checks available resources of IEj to determine if acceptance of freq causes resource overload. Edge pair identification step that mentioned above in both routing and admission control is simple. The controller already knows the ISS as soon as it gets freq . It identifies corresponding ESS by querying PSDB based on the information in freq . Flow assignment is more complex. There are several paths between IEP to assign, and also path capacities are dynamic because of path resizing operation. We develop hash-based modulo-π assignment operation (HMA) for the flow assignment step. k,π k,π and πΏmax . Assignment range depends on the HMA requires each Pkπ to have an assignment range specified with πΏmin π number of paths of IEj , path capacity π(Pk ), and the maximum range value π. The controller calculates assignment ranges using the formula defined below. { k,π πΏmin
=
k,π πΏmax
1 kβ1,π πΏmax +1
=
k,π πΏmin
k=1 , βπ β IE , 2 β€ k β€ |Pπ |
π(Pkπ ) β π +β π , βπ β IE. |P | π(Pkπ )
(1)
(2)
k=1
To explain assignment range computation in detail, let us give an example. Suppose that there are 2 distinct paths, paths 1 and 2 with capacities 2 and 3 Gbps, respectively. The assignment ranges become 1 to 40 and 41 to 100 for paths 1 and 2, respectively, if π is 100. In HMA process, the controller calculates the hash value of freq using some of its fields (eg, source and destination IP addresses) at first. Any hash function (eg, MD5 and SHA-1) can be used for this step. Result of modulus π of hash f, k,π k,π β€ π(π ) β€ πΏmax ). Note that π is an arbitrary number which is π(f), lies within the assignment range of a path (ie, πΏmin to specify assignment range for pair. Value of π must be greater than 1 and divisible to total capacity of pair, denoted as π(IEj ). Thus, assignment ranges of paths are static unless path capacities do not change. Also note that the controller calculates assignment ranges either after path resetup that is caused by link or switch failure or resizing operation that we will explain later. This implies that the controller does not frequently perform this action.
CELENLIOGLU ET AL.
9 of 22
Algorithms both for routing and admission control are provided in Algorithms 1 and 2, respectively. In Algorithm 1, the controller identifies edge pairs whenever a request comes. Then, it performs HMA operation to determine which path the flow will be assigned. Finally, the controller assigns the flow to the associated path. Algorithm 2 resembles to Algorithm 1. In admission control algorithm, the controller checks if there is enough amount of resource in edge pair before HMA operation. After assignment of f for both processes, forwarding of packets starts in data plane. First of all, ISS marks packets of f based on associated path identifier as stated in matching flow rule. Subsequently, it forwards them to the related core switch. Then, packets are forwarded over the core network. Finally, they arrive to the corresponding ESS. ESS extracts the path identifiers, and packets exit the domain. Note that data traffic can be forwarded over the paths in reverse direction as well. In this case, an ESS acts as an ingress switch and an ISS acts as an egress switch. How packet forwarding is done will be explained in Section 4. Also, notice that we adopt single-path routing approach to avoid out-of-order packet delivery that happens in multipath routing.
3.5
Load balancing
Load balancing is essential to increase resource use, which in turn reduces both capital expenditures (CAPEX) and operating expenses (OPEX) of SPs. Aside from this, proposed model must perform load balancing due to the HMA. It is because the controller may assign all the flows of an IEj to only one of its path Pkπ . To be clearer, this occurs when the result of hashing operation of flow requests corresponds mostly to only several paths. Hence, some of the paths become highly loaded when the remaining ones are lightly loaded.
10 of 22
CELENLIOGLU ET AL.
The proposed controller tries to balance load of paths of a single edge pair by shifting flows in overloaded paths to lightly loaded paths. One way of achieving this is to equalize cost of paths. Here, cost of a path must reflect its load/use/congestion π π as π(Fm ). Therefore, level. We define set Fj of flows belong to IEj pair indexed by m and denote load of a single flow Fm we formulate total traffic load of a pair as follows: π
π(IEπ ) =
|F | β
π π(Fm ),
βπ β IE.
(3)
m=1
In the proposed model, several links form a path between IEP. Thus, cost of a path is related to the cost of it's links. Cost of Pkπ is the sum of the costs of the links that form Pkπ . Denoting the cost of a path Pkπ and one of its link Lki as π(Pkπ ) and π(Lki ), respectively, path cost computation is defined as follows: π
π(Pkπ ) =
|L | β
π(Lki ),
i=1
We define the following equation for link cost computation.
βk β Pπ .
(4)
CELENLIOGLU ET AL.
11 of 22
π(Lki ) =
Qi 1β
π(L )β1 ( π(Li ) ) i
,
βi β Lk
(5)
Network operators may want to adjust link costs manually because of physical condition of the link or some other reasons. That is the reason why we introduce an adjustable parameter Qi to Equation 5. We denote link load as π(Li ) and its capacity as π(Li ). Division of π(Li ) to π(Li ) gives us the use of link. We subtract one from π(Li ) to avoid the case where denominator of Equation 5 is zero. Notice that π(Lki ) increases exponentially while π(Li ) increases linearly. This implies that cost of a link is sensitive to its load changes. The proposed controller performs pair-based load balancing. This, in turn, converges to a global load balancing in time owing to the physically shared links. Load balancing algorithm is defined in Algorithm 3. Whenever the controller initiates load-balancing process for a pair IEj , it calculates mean path cost value π(IEj ) of the pair. Costs of all the paths of IEj must eventually be equal (or close) to π(IEj ) after load-balancing process. Afterwards, the controller identifies highly and lightly used paths of pair. Consequently, the controller requests flows belong to paths of IEj from the corresponding ISS. Finally, it shifts some of the flows in highly used paths to lightly used ones. The controller selects which flow to be shifted in a greedy manner (eg, heavily to lightly loaded flow). We choose to perform adaptive load balancing for 2 reasons. First of all, the controller has limited computational resources. Although optimal network-wide load balancing can be done, any computation intensive task may degrade performance of the controller. Secondly, traffic load is highly dynamic. Thus, global load-balancing optimization does not last a long time.
3.6
Path resizing
In the proposed model, PEPs are virtual so their capacity is resizable. This prevents the network from congestion and increases resource use. Assume that there are 2 neighbor pairs, IEa and IEb , and their paths have certain capacity. IEa may suffer from congestion, while IEb is lightly loaded because of the fact that IEb can not give some of its capacity to IEa . In this case, resource use decreases. To avoid this, we develop path-resizing method to adjust path capacities whenever a path exceeds a certain threshold (eg, 80% of its maximum capacity). Note that this process is performed on PSDB, as paths are virtual. For this reason, the controller does not explicitly alter rule of paths on switches.
As aforementioned, several different paths of different edge pairs share a physical link. The controller views a single physical link Li as one or more virtual links. Each virtual link Lki is a part of a path Pπk . Thus, capacity of a path π(Pkπ )
12 of 22
CELENLIOGLU ET AL.
can change if capacity of its virtual link with minimum capacity changes. The proposed controller computes new virtual link capacity π(Lki ) for all paths Pki proportional to their path loads, denoted as π(Pki ), where those paths use Li . The corresponding equation is provided below. π(Lki ) = π(Li ) Γ
π(Pki ) π(Li )
,
βπ β IE, k β IEπ , i β L
(6)
As soon as the controller performs this computation per virtual link, new path capacity becomes the capacity of its virtual link with minimum capacity. To be more clear about path resizing, consider a scenario based on the network in Figure 3. Let capacities of paths 2 and 3 be 6 and 4 Mbps, respectively. L et also loads of paths 2 and 3 be 5.7 and 1 Mbps, respectively. In this case, path 2 exceeds the threshold, and the controller initiates resizing process. Therefore, new capacities of paths 2 and 3 become 8.5 and 1.5 Mbps, respectively. Path resizing is a heavy process although new π(Lki ) computation is simple. Since the proposed controller resizes PEPs only if a pair exceeds the resizing threshold, it is not expected to update path capacities too often. In addition, cost of some paths increase after resizing process. However, the controller performs load balancing to prevent these paths to be congested.
3.7
Signaling
The controller has to communicate with underlying switches so as to manage the whole network in SDN. Number of control messages increase as the size of network increases. Thus, there may occur heavy control traffic between the controller and switches. In terms of routing and admission control signaling, usually, an SDN controller sends forwarding rules to all of the switches along the path per flow. Therefore, the number of messages to be sent becomes the number of switches along the path per flow. Compared to this, the proposed controller sends only 2 messages independent from number of switches along the route, which are ingress and egress switches. Any CSS already contains path rules as paths are pre-established. Thus, signaling scalability is achieved regarding to the routing and admission control. As the proposed controller requires up-to-date network state to manage the whole network, it updates link states periodically. Edge switches are busy with sending flow requests to controller, inserting new flow rules to their tables, and updating existing rules during load-balancing process. For these reasons, the proposed controller does not communicate with edge switches to get link states. Instead, it collects link states from corresponding core switches. Considering the load-balancing process, suppose that there are no PEPs, the controller treats the flow to be shifted as a new flow and has to send forwarding rules to new switches between source and destination. Besides, old flow rules are deleted to maintain switch tables. Therefore, signaling scalability issue may arise as the number of flows to be shifted and network size increase. In the proposed model, the controller sends a flow update message to an ingress switch for shifting a flow from one path to another during load-balancing process. Edge switches only update their output ports to shift a flow from one path to another. Note that the proposed controller has to communicate with all the core switches along the new path in case of path creation. It also has to remove old path rules in core switches as well. In case of path alteration, the controller communicates with switches that are affected by alteration. Finally, the controller only communicates with core switches along the path in case of path deletion.
4
IMPLEMENTAT ION D ETAILS
In this section, we provide implementation details of SRRM demonstrated in Figure 2. Our controller maintains 2 databases called LSDB and PSDB. Any database system can be used to implement them. We built our controller upon Floodlight.27 Floodlight is an open source controller software, developed by a group of developers including several engineers who work at Big Switch Networks company. We implemented 5 modules so that controller performs aforementioned operations.
CELENLIOGLU ET AL.
4.1
13 of 22
Databases
The proposed controller maintains LSDB to keep track of links and the whole network topology. Basically, they are identifier of Li , switch datapath identifiers (DPID) and ports that Li connects, π(Li ), rate of Li , and π(Lki ). The latter two information changes in time based on the traffic passes through the link. In addition to LSDB, the proposed controller maintains PSDB to keep track of PEPs. In the proposed model, we assume that paths are pre-established. For this reason, we provide path information to the controller via a network definition file ndf instead of performing a multipath routing algorithm. During the lifetime of PEPs, controller maintains this database whenever any path-related information changes such as path costs. Since we determine path cost based on link costs, controller updates PSDB whenever link costs in LSDB are updated. Some static information of a path maintained in PSDB are identifier (PID), DPIDs of ISS, and ESS, links that form the path. Path-state database also has dynamic data that are path capacity and π(Pkπ ).
4.2
Path management module
We implemented the path management module so that the proposed controller can create, update, and remove them. In this part of the section, we will explain implementation details of PEPs and packet forwarding over paths in data plane.
4.2.1
Path establishment
At the beginning of the path establishment process, controller parses ndf to learn about details of paths. In our implementation, we used virtual LAN (VLAN) to establish paths using OpenFlow v1.0.28 Each VLAN has a globally distinct PID. The PID is just an integer value. After parsing the definition file, controller sends OFP_FLOW_MOD_ADD messages to corresponding core switches using Static Flow Pusher Module of Floodlight. Note that length of VLAN identifier is 16 bits. Thus, there are 216 unique paths. MPLS can be used for paths instead of VLAN,29 but this requires OpenFlow v1.3. Controller populates only tables of core switches with path rules during path establishment process. In other words, there is no specific rule for a single flow in a CSS. For instance, consider path 1 in the Figure 2. It consists of ISS, CSS11, CSS21, and ESS. According to this example, controller sends path establishment messages only to CSS11 and CSS21. Edge switches (eg, ISS1 and ESS) are empty. Thus, there is no full path. Now, let us explain how we create a complete path for a flow f. Suppose that f comes to ISSi for the first time and it should depart the network from ESSj . In this case, establishment of full path for f occurs as follows: 1. ISSi receives the first packet of f. 2. ISSi looks for a match with the packet regarding to existing forwarding rules. Since flow is new, incoming packets do not match with any of the existing flow rules. 3. ISSi prepares an OFP_PACKET_IN message and sends it to the controller. 4. Controller extracts destination address of packets from the OFP_PACKET_IN message and determines corresponding egress switch, which is ESSj . 5. Controller queries the PSDB to obtain paths of the pair ISSi - ESSj . 6. Controller determines the path for f in HMA step. 7. Controller prepares OFP_FLOW_MOD_ADD message for ISSi . In the match part of the OpenFlow (OF) message, there are source and destination IP addresses. In the action part of the same message, there are 2 actions. The first action marks the packet with associated PID specified in OFPAT_SET_VLAN_VID action. The second action sends the packet to the core switch from the physical port defined in the OFP_ACTION_OUTPUT action. The same applies for egress, but this time, controller exchanges source and destination addresses. 8. Controller prepares another OFP_FLOW_MOD_ADD message for ESSj . In the match part of the OF message, source and destination IP addresses are swapped compared to the message described in the previous step. In the action part of the this message, there are 2 actions. The first action removes the PID with OFPAT_STRIP_VLAN action. The second action sends the packet out of network from the physical port defined in the OFP_ACTION_OUTPUT action. Note that distance between an edge pair can be 1 hop. Even in this case, controller generate and install routing rules described as above.
14 of 22
4.2.2
CELENLIOGLU ET AL.
Packet forwarding
Packet forwarding starts in an ISS and ends in an ESS after controller install flow rules to corresponding edge switches. Forwarding of a packet pkt of flow f coming from ISSi and destined to ESSj is performed as follows: 1. pkt arrives to the ISSi , and ISSi finds the matching rule regarding to f. 2. ISSi sets VID part of the 802.1Q header (within Ethernet frame) of pkt with OFP_SET_VLAN_VID action of the associated rule to assign pkt to a path. 3. ISSi forwards pkt from the out-port specified in the actions of the same rule. 4. The first core switch, which is the neighbor of ISSi , receives pkt. It finds the matching path rule by checking PID of pkt. Subsequently, it forwards pkt to the next core switch along the path. 5. Each core switch along the path forwards pkt the same way as the first core switch and pkt eventually reaches to ESSj . 6. ESSj takes pkt and removes VID of the packet by OFP_STRIP_VLAN action. Finally, it forwards the packet out of network over the specified out-port. Response message for the pkt is routed back in the same way. However, this time, egress acts as ingress and vice versa.
4.3
Admission control module
If a packet does not match with any rules in an OF switch, the packet is called as unknown. As soon as an OF switch receives such a packet, it sends a message, namely, OFP_PACKET_IN, to the controller as a default action in OF v1.0. As soon as the controller receives OFP_PACKET_IN message, it takes an action such as creating a new route or rejecting the request. In our implementation, only ingress switches send such requests to the controller due to the structure of SPNs. As soon as controller receives a request from an ISS, Admissin Control Module (ACM) steps in. Briefly, ACM handles admission control in 2 steps. First of all, it determines corresponding IEP based on source and destination IP addresses. Secondly, ACM queries PSDB to check if there exists enough amount of bandwidth. If so, the controller assigns the flow to a path and sends new rules to associated edge switches. Otherwise, the request is rejected.
4.4
Statistic handling module
The proposed controller only needs link loads to balance cost of paths. In OpenFlow, port statistics provide cumulative transmitted and received byte counts. Statistic handling module (SHM) periodically sends (OF) port statistics message to corresponding core switches per link. Upon receiving OF reply messages from the CSSs, controller computes rates of links as follows: 1. Controller subtracts previously obtained byte counts from current ones. 2. Result of subtraction is divided to the time interval between consecutive measurements. Therefore, SHM obtains approximate link rates. Consequently, controller calculates link costs as described in Equation 5. Finally, SHM updates LSDB in the background.
4.5
Load-balancing module
In the proposed model, load balancing has a key role to increase resource use. Controller can initiate load balancing process in two ways. In the first way, load-balancing module (LBM) periodically balances each pair (eg, every 5 min). In the second way, the controller checks cost difference between all paths ( of)a pair. Assuming that there are |IE| pairs and |Pπ | j |P | paths, the total number of cost difference check becomes |IE| Γ . If the difference between any 2 paths exceed 2 a certain threshold, LBM initiates load-balancing process. Threshold value can be constant or adaptive. In this work, we implemented the first way. LBM follows the steps below during load-balancing process of a pair. 1. LBM retrieves path costs from PSDB. 2. LBM calculates π(IEj ). 3. LBM calculates difference between π(IEj ) and path costs per path.
CELENLIOGLU ET AL.
15 of 22
TABLE 2 The path-link matrix of Figure 3 L1
L2
L3
L4
L5
L6
L7
L8
L9
L10
Path1
10
β
β
β
10
β
β
10
β
β β
Path2
β
10
β
β
β
6.25
β
β
6.25
Path3
β
β
10
β
β
3.75
β
β
3.75
β
Path4
β
β
β
10
β
β
10
β
β
10
4. If the result is positive for a path, LBM marks the path as heavily loaded. If not, it marks the path as lightly loaded. 5. LBM obtain flow statistics from corresponding ISS for overloaded paths. 6. LBM selects subset of flows in a greedy manner (ie, lightly loaded to heavily loaded). It stops when cost of lightly loaded path is equal or close to the π(IEj ). 7. LBM sends path update message for each flow in the subset.
4.6
Path-resizing module
Path-resizing module (PRM) is responsible for updating virtual path capacities. Path-resizing module periodically accesses to PSDB to check path use. If a path exceeds a certain threshold, PRM invokes resizing process. Path-resizing module does resizing in 3 steps. First of all, it computes virtual path capacity portions for each link. Results are stored in a matrix called path-link matrix. In this matrix, rows are paths, and columns are links. As soon as PRM fills the whole matrix, it sets virtual capacity of each path to minimum value of the row which is minimum virtual capacity portion. To be clearer, consider the Table 2. There is a path-link matrix for the topology illustrated in Figure 3. In this topology, there are 10 links and 4 paths. Whenever PRM invokes resizing process, it iteratively calculates each cell in the matrix. In this example, capacity of paths 1 and 4 does not change because there is no link shared with other paths. Thus, controller does not perform any calculation for paths 1 and 4. Unlike paths 1 and 4, paths 2 and 3 share L6 and L9. Suppose that capacity of all the links is 10 Mbps and loads of paths 2 and 3 are 5 and 3, respectively. New capacities of paths 2 and 3 for L6 and L9 become 6.25 and 3.75, respectively. Path resizing enables to increase admissions of more flow requests and also reduces congestion. However, it becomes heavier as the number of links and paths in the network increases. For this reason, our controller resizes paths on demand (eg, exceeding 80% of path size). Although this prevents controller to perform resizing more than necessary, controller may perform resizing consecutively. This happens when the threshold is exceeded even after the previous path-resizing operation. To avoid this case, we introduce periodic resizing parameter. If number of on-demand resizing exceeds a certain threshold in a specific time interval (eg, 3 times in a minute), controller switches to periodic resizing mode. In this mode, controller performs resizing periodically (eg, 3 min). In periodic resizing mode, controller continues to access PSDB to check whether it is still necessary to perform periodic resizing. If there is no need to perform periodic resizing, controller switches to on-demand resizing mode. Additionally, notice that path resizing is actually a virtual operation. Thus, controller and databases are the parts of this operation. Physical links are not affected by path resizing, so signaling protocol is not used.
5
EXPERIMENTAL RESULTS
This section presents the experimental results of our model. We have implemented aforementioned modules27 of our model upon Floodlight v0.91. For creating test networks, we have used both Mininet30 v2.1 and Open vSwitch31 v2.5. The controller and switches communicate via OpenFlow v1.0.28 We used TG32 as a traffic generator. Throughout this section, several scenarios under three topologies are tested to evaluate the proposed model. These topologies are NSFNET T3, Net-M, and Net-L. NSFNET T3 is a real network33 and is chosen as it provides connectivity to several regional networks. Apart from the NSFNET T3, two custom topologies, namely, Net-M and Net-L, as illustrated in Figure 4, are also used. The reason behind creating these custom topologies is to test the proposed model in terms of topology size, number of disjoint paths, connectivity among edge, and core switches.
16 of 22
CELENLIOGLU ET AL.
FIGURE 4 Illustration of custom experimental topologies. ESS, egress software-defined networking switch; ISS, ingress software-defined networking switch 5
Chicago A Chicago B San Jose A San Jose B
Gigabits/s
4 3 2 1 0
Sun 12:00 Sun 18:00 Mon 00:00 Mon 06:00
Time (Hours)
FIGURE 5 Traffic pattern of March 29, 2015, to March 30, 2015, Coordinated Universal Time (UTC) taken from the Center for Applied Internet Data Analysis34-37
To create a realistic test environment, several real-time traffic traces are taken from the Center for Applied Internet Data Analysis34-37 as depicted in Figure 5. We have generated various traffic loads (ie, light, moderate, and heavy) by scaling these traffic traces and assigned 1 of 4 traffic patterns arbitrarily to each pair in each test run. The test scenarios are performed to analyze (1) effect of path number on resource management performance, (2) load-balancing period, (3) threshold-based load balancing, (4) resource management performance under various traffic loads, (5) resource management for different topologies, (6) comparison of the proposed model with the work proposed in Jiang et al,38 (7) investigation of admission control time, and (8) investigation of signaling scalability.
5.1
Effect of path number on resource management performance
Resource management performance of the proposed framework depends on the number of paths. In this context, we have investigated the performance of load balancing under various path numbers, namely, P2, P3, P4, and P5. P2, P3, P4, and P5 has 2, 3, 4, and 5 paths per pair, respectively. Net-M is chosen as the test topology since it allows to create 5 disjoint paths unlike others. The same traffic trace combination is applied to the same pairs of Net-M for 10 minutes. There are 3 test runs. During each test run, maximum core link uses are measured and average of them is calculated per measurement time. The result is illustrated in Figure 6. We have observed that performing load balancing with 2 paths has the worst performance as it has the highest use. In P2 case, controller performs load balancing among 2 paths per pair. As the proposed framework performs adaptive
CELENLIOGLU ET AL.
17 of 22
Max Utilization (%)
100
P2 P3 P4 P5
80 60 40 20 0 0
100
200
300
400
500
600
Time
FIGURE 6 The mean of maximum core link uses in time for different number of paths
load balancing, convergence of cost equalization among pairs takes more time. As the path number increases, max use decreases. For the same reason, P5 has better performance since more pairs are neighbors.
5.2
Investigation of load-balancing period
Resource management performance of the proposed framework depends on load-balancing performance. In this context, we have investigated the performance of load balancing under various periods to understand how frequent the load balancing should be performed depending on the path number. Load-balancing periods are set to 5, 10, 20, and 30 seconds. Net-M is chosen as the test topology since it allows to create 5 disjoint paths unlike others. The same traffic trace combination is applied to the same pairs of Net-M for 10 minutes. There are 3 test runs for both P2 and P5. During each test run, maximum core link uses are measured, and average of them is calculated per measurement time. The result is illustrated in Figure 7. We have observed that performing load balancing in short (eg, 5 s) or large (eg, 30 s) intervals both for P2 and P5 causes to a performance degradation as shown in Figure 7. Performing load balancing with short period shifts flows from one path to another frequently. This causes unnatural fluctuation in traffic. Load balancing with large period has low performance, as it equals costs infrequently. This implies that load balancing with larger periods makes the case like there is no load balancing, especially for highly dynamic network traffic.
5.3
Investigation of threshold based load balancing
The proposed load-balancing method can also be initiated depending on the cost difference among paths of pairs. In this context, we have investigated the performance of load balancing under various cost differences, in other words, 100
5 Seconds 10 Seconds 20 Seconds 30 Seconds
80
Mean Utilization (%)
Mean Utilization (%)
100
60 40 20 0
0
100
200
300
Time
400
500
600
5 Seconds 10 Seconds 20 Seconds 30 Seconds
80 60 40 20 0
0
100
200
300
400
500
Time
FIGURE 7 The mean of maximum core link uses in time for various number of paths and load-balancing periods
600
18 of 22
CELENLIOGLU ET AL.
threshold-based load balancing (TLB), that are 0.1, 0.25, 0.5, and 1.0. Net-M is chosen as the test topology since it allows to create 5 disjoint paths. The same traffic trace combination is applied to the same pairs of Net-M for 10 minutes. There are 3 test runs for both P2 and P5. During each test run, maximum core link uses are measured, and average of them is calculated per measurement time. The result is illustrated in Figure 8. Similar to the periodic load-balancing case, we have observed that large TLBs (eg, 1.0) has little impact on cost equalization. Threshold-based load balancing with 1.0 has the lowest performance as the controller have initiated load-balancing process for a number of times. Additionally, performing load balancing for very small cost differences (eg, 0.1) causes lower use compared to TLB with 0.25 and 0.5. This is because the controller frequently performs flow shifting among paths of pairs. We have observed that TLB with 0.25 has the best performance regardless of path numbers among the threshold values we chose. Similar to the periodic load balancing, the proposed model achieve better use with larger number of paths. It must be noted that TLB should be adjusted based on the traffic fluctuation.
5.4
Resource management performance under various traffic loads
We have investigated how the proposed framework performs under various traffic loads (eg, 20%, light; 50%, moderate; and 80%, heavy). To do this, the same set of traffic traces are applied on Net-M with 2 paths per pair by scaling them. There are 3 test runs. Duration of test run is set to 10 minutes, and periodic (ie, 10 s) load balancing is performed by the controller. During each test run per traffic load, maximum core link uses are measured, and average of them is calculated per measurement time. The result is illustrated in Figure 9. To generate light traffic load, flows with small sizes are generated by clients. In case of moderate traffic load generation, there is a mixture of flows with small and large sizes. In the heavy traffic load case, almost all flows are large. Under these circumstances, we have observed that shifting flows from one path to another causes small cost changes in light traffic load case. Fluctuation in moderate traffic load case is more compared to the previous case. In heavy traffic load case, impact of shifting flows from one path to another causes greater fluctuation. We believe that, in real world, there are many flows with various sizes. Hence, load balancing can be achieved in a smoother way.
5.5
Resource management performance for various topologies
We have investigated performance of the proposed model for different networks. In this context, we applied the same set of traffic traces to all test topologies for 10 minutes. The applied traffic load is moderate. We have measured the change in costs of 2 randomly chosen paths. Obtained results, illustrated in Figure 10, show that path costs get close to each other after cost equalization process. Therefore, the proposed model successfully balances loads regardless of topology in an adaptive manner.
0.1 0.25 0.5 1.0
80 60 40 20 0
0
100
200
300 Time
400
500
600
100
Mean Utilization (%)
Mean Utilization (%)
100
0.1 0.25 0.5 1.0
80 60 40 20 0
0
100
200
300
400
500
Time
FIGURE 8 The mean of maximum core link uses in time for various number of paths and load-balancing thresholds
600
CELENLIOGLU ET AL.
19 of 22
100
Light Moderate Heavy
Max Utilization (%)
80
60 40 20
0
0
100
200
300
400
500
600
Time
FIGURE 9 The mean of maximum core link uses in time for various traffic loads
10
10
Path 1 Path 2
8
6 4
6 4
0
100
200
300
400
500
6 4 2
2
2
Path 1 Path 2
8
Path Costs
Path Costs
8
Path Costs
10
Path 1 Path 2
0
100
Time (Seconds)
200
300
400
0 0
500
Time (Seconds)
100
200
300
400
500
Time (Seconds)
FIGURE 10 Equalization of path costs under moderate traffic load in all test topologies
100
SRRM - P5 WPF Max Utilization (%)
80
60
40
20
0
0
100
200
300
400
500
600
Time
FIGURE 11 Comparison of resource management performance of models. SRRM, software-defined networkingβbased scalable routing and resource management model
5.6
Resource management performance comparison
The works Luo et al17 and Tuncer et al18 address both resource management and control plane scalability in SDN-based networks. However, they are complex to implement due to their both hierarchical and distributed structures. They also contain multiple controllers. Therefore, instead of comparing our model with the proposed works in Luo et al17 and Tuncer et al,18 we compare our model SRRM with the Widest Path First Model (WPF), which is implemented in Jiang et al.38 WPF also has a single controller that performs both resource use and routing. In this test scenario, we applied the same set of traffic traces to both of the models for 10 minutes. Net-M is chosen as the test topology. The applied traffic
20 of 22
CELENLIOGLU ET AL.
(B) 120
SRRM - P5 WPF
100 80 60 40 20 0
0
50
10 0
15 0
Rate of Flow Acceptance
Admission Control Time (msec)
(A)
200
WPF SRRM - P2 SRRM - P5
100 80 60 40 20 0
0
Number of Flow Requests/Second
20
40
60
80
100
Network Load (%)
FIGURE 12 Performance comparison of models in terms of admission control time and flow acceptance rate. SRRM, software-defined networkingβbased scalable routing and resource management model
load is moderate. There are 3 test runs. During each test run, maximum core link uses are measured, and average of them is calculated per measurement time. The result is illustrated in Figure 11. Note that WPF calculates the shortest path in a graph where both links and nodes have had weighted. WPF establishes a path once for a flow and does not change the path in time. Thus, link use does not change drastically in time. However, the proposed model, where each pair has 5 paths, successfully performs resource use and outperforms (WPF).
5.7
Performance of admission control mechanism
In this test scenario, we have investigated the performance of admission control mechanism implemented in the proposed model. In the first part of this test case, we have measured the admission control time for both SRRM and WPF and illustrated the results in Figure 12A. We have observed that WPF consumes more time to find a suitable path as the network load increases. The proposed model, on the other hand, performs admission control very quickly. Besides, increase of admission control time with respect to increasing traffic load is negligible. This proves that our model performs admission control in a scalable manner. We have conducted another test on Net-M to examine effect of resource management operations on flow acceptance rate. We sent periodic flow requests to only 1 arbitrarily chosen pair and measured the rate of flow acceptance and run this scenario both for P2 and P5. Our controller have distributed network load among paths and performed resizing to reduce congestion, if necessary. Thus, flow acceptance rates of SRRM is close to WPF as illustrated in Figure 12B even P2 case.
6
CO N C LUSION AN D FU T U R E WORKS
To put in a nutshell, this paper proposes a scalable routing and resource management model for SDN-based SPNs. In the proposed model, multipaths are pre-established between each pair of edge switches. These paths allow controller to perform scalable routing, admission control, and signaling. Controller also performs adaptive load balancing by equalizing path costs of individual pairs and adjusts path capacities based on network congestion. The proposed model shows that even if there is a single controller at the top of network, it can successfully perform routing, admission control, and signaling in a scalable manner. The proposed controller is able accept or reject flow requests very quickly due to its pre-established paths structure. Instead of computing flow routes per request, paths that are established in offline mode prevents controller to become a bottleneck. We also witness that the amount of signaling between controller and data plane is also reduced both in statistic collection and path establishment processes. Our model also successfully achieves adaptive load balancing. Additionally, we observe that path resizing is a heavy process. Controller performs path resizing until it is not bearable any more. Therefore, it does not affect scalability of controller significantly. Finally, obtained results prove that load balancing and path resizing increases flow acceptance rate dramatically.
CELENLIOGLU ET AL.
21 of 22
As a future work, we aim to perform load balancing by estimating incoming traffic. This way, we hope to achieve a better load-balancing performance. We also aim to develop moving load-balancing period that changes in time depending on the network traffic. Our another goal is to provide energy efficiency without degrading the performance, making use of PEPs.
ACKNOWLEDGMENTS This research is supported by The Scientific and Technological Research Council of Turkey (TUBITAK) under grant no. 113E253.
ORCID Mahmud Rasih Celenlioglu http://orcid.org/0000-0001-8061-419X http://orcid.org/0000-0002-8955-9710 Mehmet Fatih Tuysuz
REFERENCES 1. Kim H, Feamster N. Improving network management with software defined networking IEEE Commun Mag. 2013;51(2):114-119. 2. Yeganeh SH, Tootoonchian A, Ganjali Y. On scalability of software-defined networking. IEEE Commun Mag. 2013;51(2):136-141. 3. Luo H, Cui J, Chen G, Chen Z, Zhang H. On the applicability of software defined networking to large scale networks. In: 23rd International Conference on Computer Communication and Networks (ICCCN); 2014; Shanghai, China. 1-6. 4. Leguay J, Maggi L, Draief M, Paris S, Chouvardas S. Admission control with online algorithms in sdn. In: 2016 IEEE/IFIP Network Operations and Management Symposium; 2016; Istanbul, Turkey. 718-721. 5. Bianco A, Giaccone P, Mahmood A, Ullio M, Vercellone V. Evaluating the SDN control traffic in large ISP networks. In: 2015 IEEE International Conference on Communications (ICC); 2015; London, UK. 5248-5253. 6. Tootoonchian A, Ganjali Y. Hyperflow: a distributed control plane for openflow, In: 2010 Internet Network Management Conference on Research on Enterprise Networking, INM/WREN'10. Berkeley, CA, USA: USENIX Association; 2010:3-3. 7. Koponen T, Casado M, Gude N, et al. Onix: a distributed control platform for large-scale production networks. In: 9th USENIX Conference on Operating Systems Design and Implementation; 2010; Vancouver, BC, Canada. 351-364. 8. Kohler T, DΓrr F, Rothermel K. Zerosdn: a highly flexible and modular architecture for full-range network control distribution. In: 2017 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS); 2017; Beijing, China. 25-37. 9. Hassas Yeganeh S, Ganjali Y. Kandoo: a framework for efficient and scalable offloading of control applications. In: First Workshop on Hot Topics in Software Defined Networks, HotSDN '12; New York, NY, USA: ACM; 2012:19-24. 10. Ahmed R, Boutaba R. Design considerations for managing wide area software defined networks. IEEE Commun Mag. 2014;52(7):116-123. 11. Lopez-Rodriguez F, Campelo DR. A robust SDN network architecture for service providers. In: 2014 IEEE Global Communications Conference (GLOBECOM); 2014; Austin, TX, USA. 1903-1908. 12. Wang J, Shou G, Hu Y, Guo Z. A multi-domain SDN scalability architecture implementation based on the coordinate controller. In: 2016 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC); 2016; Chengdu, China. 494-499. 13. Jain S, Kumar A, Mandal S, et al. B4: experience with a globally-deployed software defined wan. SIGCOMM Comput Commun Rev. 2013;43(4):3-14. 14. Wang JM, Wang Y, Dai X, Bensaou B. SDN-based multi-class QoS-guaranteed inter-data center traffic management. In: 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet); 2014; Luxembourg. 401-406. 15. Cammarata G, Stefano AD, Morana G, Zito D. Evaluating the performance of A4SDN on various network topologies. 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW); 2016; Chicago, IL, USA. 801-808. 16. Tomovic S, Lekic N, Radusinovic I, Gardasevic G. A new approach to dynamic routing in SDN networks. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW); 2016; Chicago, IL, USA. 1-6. 17. Luo M, Zeng Y, Li J, Chou W. An adaptive multi-path computation framework for centrally controlled networks. Comput Networks. 2015;83:30-44. 18. Tuncer D, Charalambides M, Clayman S, Pavlou G. Adaptive resource management and control in software defined networks. IEEE Trans Netw Serv Manage. 2015;12(1):18-33. 19. Wang C, Hu B, Chen S, Li D, Liu B. A switch migration-based decision-making scheme for balancing load in SDN. IEEE Access. 2017;5:4537-4544. 20. Wang Z, Crowcroft J. Routing algorithms for supporting resource reservation. IEEE J Sel Areas Commun. 1996. 21. Kodialam M, Lakshman TV. Minimum interference routing with applications to MPLS traffic engineering. In: Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies; 2000; Tel Aviv, Israel. 884-893. 22. Boutaba R, Szeto W, Iraqi Y. Dora: Efficient routing for MPLS traffic engineering. J Network Syst Manage. 2002;10(3):309-325. 23. McKeown N, Anderson T, Balakrishnan H, et al. Openflow: enabling innovation in campus networks. SIGCOMM Comput Commun Rev. 2008;38(2):69-74.
22 of 22
CELENLIOGLU ET AL.
24. Eppstein D. Finding the k shortest paths. SIAM J Comput. 1999;28(2):652-673. 25. Abe JO, Mantar HA, Yayimli AG. k -maximally disjoint path routing algorithms for SDN. 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery; 2015; Xi'an, China. 499-508. 26. Chen J, Chen J, Xu F, Yin M, Zhang W. When software defined networks meet fault tolerance: a survey. Cham: Springer International Publishing; 2015:351-368. 27. Izard R. Floodlight v1.0 release notes. 2014. https://floodlight.atlassian.net/wiki/spaces/floodlightcontroller/pages/1343555/ Floodlight+v1.0. Accessed April 25, 2017. 28. Foundation Open Networking. Openflow specifications v.1.0.0. 2009. https://www.opennetworking.org/images/stories/downloads/ sdn-resources/onf-specifications/openflow/openflow-spec-v1.0.0.pdf. Accessed April 25, 2017. 29. Foundation Open Networking. Openflow specifications v.1.3.3. 2013. https://www.opennetworking.org/images/stories/downloads/ sdn-resources/onf-specifications/openflow/openflow-spec-v1.3.3.pdf. Accessed April 25, 2017. 30. Lantz B, Heller B, McKeown N. A network in a laptop: rapid prototyping for software-defined networks. In: 9th ACM SIGCOMM Workshop on Hot Topics in Networks (Hotnets-IX); New York, NY, USA: ACM; 2010:19:1-19:6. 31. Pfaff B, Pettit J, Koponen T, et al. The design and implementation of open vswitch. In: 12th USENIX Symposium on Networked Systems Design and Implementation, NSDI'15. Berkeley, CA, USA: SENIX Association; 2015:117-130. 32. International SRI, Center USC/ISI Postel. Tg traffic generator tool. 2002. http://www.postel.org/tg/. Accessed April 25, 2017. 33. Claffy KC, Braun HW, Polyzos GC. Tracking long-term growth of the nsfnet. Commun ACM. 1994;37(8):34-45. 34. The cooperative association for internet data analysis. passive network monitors - Chicago monitor A (current). 2015. http://www.caida. org/data/realtime/passive/?monitor=equinix-chicago-dirA/. Accessed March 29, 2015. 35. The cooperative association for internet data analysis. passive network monitors - Chicago monitor B (current). 2015. http://www.caida. org/data/realtime/passive/?monitor=equinix-chicago-dirB/. Accessed March 29, 2015. 36. The cooperative association for internet data analysis. passive network monitors - San Jose monitor A (current). 2015. http://www.caida. org/data/realtime/passive/?monitor=equinix-sanjose-dirA/. Accessed March 29, 2015. 37. The cooperative association for internet data analysis. passive network monitors - San Jose monitor B (current). 2015. http://www.caida. org/data/realtime/passive/?monitor=equinix-sanjose-dirB/. Accessed March 29, 2015. 38. Jiang JR, Huang HW, Liao JH, Chen SY. Extending Dijkstra's shortest path algorithm for software defined networking. In: The 16th Asia-Pacific Network Operations and Management Symposium. Hsinchu, Taiwan: National Chiao Tung University; 2014:1-4.
How to cite this article: Celenlioglu MR, Tuysuz MF, Mantar HA. An SDN-based scalable routing and resource management model for service provider networks. Int J Commun Syst. 2018;31:e3530. https://doi.org/10.1002/dac.3530
