JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
329
Performances of OpenFlow-Based SoftwareDefined Networks: An overview Fouad Benamrane, Mouad Ben mamoun, and Redouane Benaini LRI, Faculty of Sciences of Rabat, Mohammed V University, Rabat, Morocco Email:
[email protected], {ben_mamoun, benaini}@fsr.ac.ma
Abstract—Software Defined Networking (SDN) has gained significant attention from network researchers and industry in recent years. Indeed, the SDN concept provides many advantages such as programmability and easy management of the network. However, it generates new challenges as scalability and performances issues, understanding in-depth the performances and limitations of the SDN concept is a prerequisite to its implementation and deployment in real networks. In this paper, we aim to present in a comprehensive way, the most important works that focus on performances of SDN. As SDN separates the control plane from the data plane, we first present research efforts made to enhance the performances of data plane devices, then, we give an overview of different solutions proposed to improve controller performances. We provide also an overview on recent control plane architectures with multiple controllers that have been proposed to meet performances and scalability constraints. Finally, we present the different techniques and tools used in literature to evaluate the performances of software defined networks.
I.
INTRODUCTION
Software-Defined Networking (SDN) is a new concept of network architecture that has emerged in last years. Contrary to traditional architectures where the control plane and the data plane coexist in network devices such as routers and switches, this concept basically consists to move the control plane outside network devices and leaves only the data plane inside. The control plane is supported by a software application called: “Controller”. Network devices become simple packet forwarding devices that can be programmed through rules that are set by the controller. In SDN, communication between the control plane (the controller or controllers) and the data plane (networking devices) is mainly established today by the OpenFlow protocol. After the proposal of the SDN concept and the implementation of OpenFlow by researchers from Stanford University towards 2008, SDN has aroused great interest and it becomes an important topic for research, development, and standardization in the networking area. This is due, on the one hand to the limitations of current network architectures to meet the requirements of new IT trends such as cloud computing, virtualization, Internet of things and the explosion of mobile devices and on the other hand, to the potential and the various benefits, which Software-Defined Networks may offer.
© 2015 ACADEMY PUBLISHER doi:10.4304/jnw.10.6.329-337
In fact, the coupling between the control plane and the data plane in network elements prevents quick and easy deployment of new network functionalities and services because they must be implemented directly in infrastructure. However, this depends on the production cycles of manufacturers which may be very long. Thus, traditional networks are relatively static and lack flexibility to adapt to the dynamic nature of traffic, applications, cloud and virtual environments. The interest of SDN is the ability to program network devices in a unified and centralized way through software applications. This gives a large flexibility to administrators to dynamically control, manage, secure and optimize their network resources. Moreover, the ability to program the network by operators, enterprises and users will accelerate innovation in networks. The introduction of new network services will be done at software development speed without need to wait for new hardware products from manufacturers. The SDN approach certainly has several advantages, as explained above. However, some questions and issues have occurred with this approach. In fact, the first propositions and deployments of SDN networks use a single centralized controller. This poses the problem of having a single point of failure, a concern about the performance of the controller and its capacity to handle a large amount of flow requests and by consequence the ability to scale when the network size grows. The first Openflow controller has been NOX, it has a flow install time less than 10 ms and can handles up to 30000 flow requests per second [1]. Many other Openflow controllers were developed after with considerably better performance like Floodlight [2], OpenDayLight [3]. In this article we provide an overview of the progress that has been made regarding the performance of the controller and techniques that have enabled this progress. On the other hand, to avoid the problem of a single point of failure and address the scalability issue, control planes with multiple controllers have been proposed. But this generates new questions and issues such as: the number of needed controllers, the placement of controllers and the communication between controllers. In this context, we can distinguish two main propositions of control plane designs: logically centralized and logically distributed control planes. Both propositions have their advantages and disadvantages. Later in this paper, we present these
330
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
propositions and discuss their advantages and disadvantages. Since the emergence of Software Defined Networking concept, a lot of works have investigated this topic in the networking area. Several recent papers [4], [5], [6], and [7] provide interesting surveys about SDN and the performed works. These papers present the different contributions that have helped to promote programmable networks before SDN. They explain in details the concept, the motivations and the architecture of SDN. They give all information about the Openflow implementation and its different releases. They present and discuss the current deployments and challenges and also the future directions for SDN. Our goal in this paper is different, we aim to give to the reader especially interested in the performances of SDN networks, an overview of the main studies and enhancement propositions that have been made in the literature regarding the performances of SDN networks. In fact, on the one hand, the survey papers mentioned above cover all aspects of SDN networks and do not focus on the performance issues and the improvement solutions. The information on performance is embedded in a large amount of information and details about SDN. On the other hand, some articles are exclusively devoted to performances but they are specific to a controller or a performance aspect. For instance, the paper [8] is dedicated to performances of the NOX-MT controller, the paper [9] analyses the impact of the latency between an openflow switch and its controller, moreover, authors [10] compare the throughput of different controller, and shows the impact of multi-threading on their performances. The paper is organized as following: the first section describes briefly the concept of SDN and OpenFlow protocol. The second section is devoted to the performances of OpenFlow switches. The third section treats the performances of Openflow controllers. The fourth section surveys the emerging propositions based on multiple controllers. Section V describes the various techniques that have been used to evaluate the performances of OpenFlow networks. The last section is dedicated to a conclusion and some ideas for future works. II.
SDN AND OPENFLOW
In this section we review the SDN architecture and the Openflow protocol, then we present briefly the main performance metrics associated to an OpenFlow-Based Software-Defined Network. A. SDN Architecture As illustrated in figure 1, SDN separates the network architecture into three layers: Transmission layer (or data plane): composed by soft switches, its main function is to execute the orders provided by the remote controller. Control layer (or control plane): composed by one or more controllers which constitute the core of the infrastructure and concentrate all the intelligence of the network. This layer offers a unified and centralized view of the network, and it hides the complexity of the underlying physical network to the applications. © 2015 ACADEMY PUBLISHER
Application layer: as a result of the centralized view provided by the control plane, this layer constitutes a platform for the implementation of all kinds of new services and applications designed specifically for users needs. B. OpenFlow Protocol The communication between the control plane and the data plane, in the SDN architecture is essentially performed by the OpenFlow protocol. It’s an open source protocol normalized by the Open Networking Foundation (ONF) [32], funded by the leaders of IT industry like Cisco, Facebook, Google, HP, Microsoft, … Before presenting the components of the Openflow protocol, note that there is several versions of this protocol. Indeed, since its first release and thanks to the growth of interest for SDN/OpenFlow, several versions have been developed with the objective to improve the capabilities of previous versions. The main additions proposed in each version are summarized below:
Figure 1. SDN architecture
1.1: Support for MPLS, Qos, VLANs, multipath, multiple tables, virtual ports, port groups 1.2: Support for extensible headers (in match, packet_in, set_field), IPv6 1.3: Support for tunneling, per-flow traffic meters, Provider Backbone Bridging, and notion of a group, multiple controllers (Master, Slave or Equal). 1.4: Provides better management of switches by controllers by adding a number of new interactions between them. “Role status” keeps controllers synchronized with regard to which is primary versus secondary. Group/meter change notifications sent to all controllers associated with a switch. In OpenFlow-Based Software-Defined Networks, the traffic is treated as flows. Each OpenFlow-compliant switch contains one or more flow tables that include informations about flows. Each flow table entry has the following fields: Match field: defines all information about the flow that are used to match incoming packets. Statistics field: maintains the number of packets and bytes for each flow.
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
Action field: defines how the packets belonging to a flow should be processed and forwarded. With the OpenFlow protocol, the switches use flow tables to forward packets, the controller may set up new forwarding rules, if necessary, into a switch flow table at any time, the communication between the controller and switches is established over a secure channel. Figure 2 illustrates the communication process between an OpenFlow switch and a controller when a new rule must be implemented in the flow table: when a packet arrives to a switch, its header fields are examined and compared with the match fields in the flow table entries. If a Match is identified, the switch executes the action specified in the corresponding flow table entry. Otherwise, if there is no match (1), a flow request is sent to the controller (2), then the controller decides an action and sends a new forwarding rule to the switch (3). Finally, the switch's flow table is updated (4).
331
More recent controllers like beacon could achieve a throughput of 7 million flows per second. To achieve these good results, many studies, propositions and solutions have been made in the literature to understand and improve the performances of OpenFlow-Based Software-Defined Networks. Our goal in this paper is to present and discuss the main performed studies. In the remainder of this paper, we classify them in three categories: Performance studies of openflow switches Performance studies of openflow controllers Distributed controllers studies We note that the distributed controllers’ studies may be treated under the performance studies of Openflow controllers’ category, but we prefer to discuss them in a separate section due to their importance. III.
PERFORMANCES OF OPENFLOW SWITCHES
Recently, the flow-based switch models (OpenFlow) emerge with more flexibility in comparison to Ethernet switches. It is a promising technology that can enhance network virtualization towards a more flexible future Internet architecture. Two types of OpenFlow switches can be identified. The first one is the hardware-based commercial switch, which typically uses TCAMs and flow table to store all information about flows. The second one is the software-based switch that uses Linux systems to perform the OpenFlow switch operations. In this section, we mention a study that compares the OpenFlow switches and the traditional switches in terms of performances, and then present some interesting contributions that have been made to the data plane of an OpenFlow network. We classify them in two categories: software-based enhancements and hardware-based enhancements. Figure 2. Switch-Controller communication process
Obviously, the traffic for which the intervention of the controller is necessary to install a new rule will be penalized in terms of delay compared to the traffic that can be transmitted directly by the switch. In addition, if the controller is extremely solicited by the switches, it could become a bottleneck and affects the performances of the whole network. C. Key Performance Metrics From the above subsection, it appears that the SDN/OpenFlow architecture raises the problem of controller performance and its capacity to handle the requests of a large number of network devices. Two performance metrics are from primary importance: Throughput: number of flow requests handled per second. Latency: delay to respond to flow requests. Currently, there are several available OpenFlow controllers, for example, NOX, Floodlight, Ryu and ODL (OpenDayLight). The main difference between these controllers lies on their performances. For instance, NOX the first developed controller could handle just 30K flows per second and has a flow install time (latency) of 10 ms. © 2015 ACADEMY PUBLISHER
A. Comparison with Traditional Switches Since the arrival of the OpenFlow switch model, a comparison in terms of performance with traditional switches has been needed. In their paper [11], authors evaluate the achievable performance of an OpenFlow switch and a traditional L2 and L3 switch by comparing the results of single flows against multiple flows experiment. Two parameters are considered for the evaluation, throughput and latency. In single-flow case, authors focus on the exact match packet with different size, they find that starting from 256-byte packets, all three forwarding techniques are able to achieve the full wire speed, maximum throughput and the latencies are quite small (less than 30 us). However in multi-flow case they consider the effects of the forwarding table size, the performance degradation of L3 and OpenFlow switches is quite limited, and the effect on L2 switch is disastrous. In all the experiments, the OpenFlow switch offers very good performance in both cases compared to traditional switches. B. Software-based Enhancements OpenFlow switches have two types of lookup tables: hash and linear. The hash table called also the exact match table uses a hashing function, and it contains up to
332
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
131072 entries. The linear table exploits wildcards in the packet header fields to match packets to flows; it’s a small table and contains only 100 entries. In [12], the authors introduce the Flow Director technique. It is used for directing incoming packets to received queues. The Flow Director is considered as an additional lookup table, which stores a mapping between receiving queue and output port. They observe that unlike the linear table size, the hash table size as well as Flow Director table size does not affect OpenFlow switching performance, and the modified OpenFlow switch improves switching throughput up to 25% compared to the throughput of regular software-based OpenFlow switching. Another contribution that we mention in this subsection concerns the counters and statistics that remain in an OpenFlow switch. Mogul et al. [13] have announced that among advantages of SDN network is that it allows a fine-grained flow control by maintaining per-flow counters (e.g.; Received packets, Received bytes, and Duration) in the data plane, and making them visible to the external controllers. They propose SoftwareDefined Counters (SDC) to migrate all SDN counters from the ASIC to switch-local CPU, consequently they propose theoretically SDC for increasing flexibility, efficiently access to counters, and reduction in ASIC space and complexity.
SDN controllers are special control software which interacts with switching devices and provides a programmatic interface. Nowadays there are more than 30 different OpenFlow controllers created by different groups, written in different languages and having different performances. The performance of an SDN controller is characterized by several metrics, but, throughput and latency are the most considered. Since the first developed controller NOX, several studies and improvement solutions have been done concerning the performances of controllers. These solutions are essentially based on the use of virtualization and multithreading techniques. We began this section by mentioning some interesting performance studies, after we present the improvement solutions.
C. Hardware-Based Enhancements An OpenFlow switch makes generally two important decisions: queue-management and scheduling operations. The first concerns how long can the queue grow? and the second consists on deciding which packet should be sent next when an outgoing link is free. An interesting work in [14] treats these two questions. The authors propose to extend SDN’s flexibility to cover queuing and scheduling decisions made in the data plane, this by adding a small FPGA to the fast path of a hardware switch, with simple interface to the switch’s packet queues. This solution allows queuing and scheduling to be reconfigured by the network operator, and it adjusts queuing and scheduling behavior with application objectives. As a result they achieve both high performance and flexibility to the data plane. At least two studies have proposed additional ways to take profit of a CPU being connected to the switch. Authors in [15] attempt to increase the size of both forwarding table and packet buffer, by proposing some modifications to the current OpenFlow switch design. The main idea focuses on combining ASIC and CPU processing. In their solution, the switch is not a dummy component; it swaps the flows, monitors the queues and it decides the traffic redirection. They propose to use CPU as a traffic co-processor in switches to address the limited size of forwarding table and packet buffer. This makes the network devices more programmable and offer more network functions. A prototype is developed and a 3.9Gb/s throughput is achieved. In the same direction, Luo et al in [16] implement network processor based acceleration cards to perform OpenFlow switching. They show a 20 percent reduction on packet delay compared to conventional designs.
A. Some Performance Studies To evaluate the best controller in term of latency, the authors of [10] calculate the latency considered as the average response time with one connected switch and 105 hosts. They show that the smallest latency has been demonstrated by MuL and Maestro controllers, while the largest latency is typical of python-based controllers: POX and Ryu. In [9], the authors consider FloodLight which is a high performing OpenFlow controller that can handle a large amount of flows from a large amount of equipments, and then show that the link between a switch and its controller is of primary importance for the whole performances of the network. In fact, the controller cannot process requests faster than it receives them. In this study, the authors focus on how the controller and the network perform with bandwidth and latency issues on the control link. They adjust the bandwidth with a traffic shaper and the latency by increasing the time a packet has to wait in the egress queue to reach the controller. Increasing the bandwidth leads to a more reactive network due to the controller working at full capacity. Conversely, a low bandwidth will cause some packet losses as the egress queues fill up. The latency has a very different effect on the performances. In particular, the authors show that having a high throughput with a high latency causes bad performances. As each switch has to do a handshake with the controller before being able to send requests, no operation is possible during a time window. For instance, when the latency increases, the time to complete the handshake phase increases until 7s for a latency of 100ms in a 32 switches network. Hence, with a high latency, the packets have to wait longer, thus increasing the probability of
© 2015 ACADEMY PUBLISHER
From the previous studies, we see that various parameters may influence the performance of an OpenFlow-enabled switch, including the processing time in the modern all-in-one ASIC, lookup table delay, queue-management and scheduling operations. The proposed enhancements lead to better performances of OpenFlow switches, and they encourage the enterprises to use them in production networks. In the next section, we present performance studies of Openflow controllers. IV.
PERFORMANCES OF OPENFLOW CONTROLLERS
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
333
packet loss, and it reduces the overall performance of the network. In conclusion, the bandwidth arbitrates how many flows the controller can process, as well as the loss rate if the system is under heavy load, while latency drives the overall behavior of the network. B. Virtualization-Based Improvements Network virtualization is a particular abstraction of a physical network that allows supporting multiple logical networks to run on a common physical substrate. SDN effectively separates the data plane and the control plane, whereas virtual networks separate logical and physical networks. The two concepts are certainly distinct, but SDN is a useful tool for implementing virtual networks. To benefit from SDN and network virtualization, and improve the performance of an SDN network, researchers add new layer between control-plane and data-plane called Flowvisor [17]. Flowvisor basically virtualizes network control by letting experimental traffic run in parallel on the production network with the real user, and real production traffic. Flowvisor specifies some subset of the traffic that he is willing to let run over that experimental network control. This is achieved using a concept called flow space, where some subset of traffic flows, based on IP address, port, might be specified as being controlled by an experimental network controller, as opposed to the production network control. Whereas FlowVisor can be compared to a full virtualization technology, FlowN [18] is a virtualization solution where each tenant (instance) has the illusion of having its own address space, topology, and controller. This enables the tenants to deploy any network abstraction and application on top of the controller platform, and otherwise it allows a unique shared controller platform to be used for managing multiple domains in a cloud environment. By comparing FlowN and Flowvisor in a scalable virtual networks, they conclude that FlowN has a higher overhead (Latency) due to the database but scales better than FlowVisor (in case of 100 virtual networks or greater). C. Multithreading-Based Improvements Authors in [8] attempt to understand the implication of using multi-threading techniques in SDNs. They have introduced a new multi-threaded controller called NOXMT (new release of the NOX OpenFlow controller). The main conclusion is that NOX-MT improves the controller throughput by more than 30 times. Indeed, NOX-MT takes full advantage of multi-core processors using multithreading techniques that can run multi-tasks TABLE I. Language Developer OpenFlow Throughput in kilo flows/s Multi-threaded OpenStack Distributed Learning curve
© 2015 ACADEMY PUBLISHER
NOX C++ Nicira 1.0, CPQD:1.1, 1.2, 1.3 30 No No No Moderate
concurrently and handling different tasks at the same time, hence making optimal use of the available resources. An important study that investigates the impact of multithreading on performances is given [7]. The authors perform a comparison between different controllers in term of performance, especially the throughput. In fact, two major factors distinguish controllers, the first one is the algorithm of distributing incoming messages between threads, and the second one is the mechanism or the libraries used for network interaction. Using different number of threads (from 1 up to 12) shows that single threaded controllers (Nox and Ryu) are very limited regarding the throughput because they cannot handle a large number of flows, however the Beacon controller which is multi-threaded can handle a large number of flows per second. Knowing that Python controllers (POX and Ryu) do not support multi-threading, they show no scalability despite a number of CPU cores. Maestro's scalability is limited to 8 cores as the controller does not run with more than 8 threads. The performance of Floodlight increases steadily in line with the increase in the number of cores. Finally, Beacon achieves a throughput of nearly 7 million flows per second and shows the best scalability. From the previous studies, we conclude that the characteristics and performances of SDN controllers may be very different. So, it's important to carefully choose the OpenFlow controller. At the end of this section, we give a summary of some famous controllers and their characteristics in the table 1. In particular, we specify the OpenFlow version that they support, their throughput, if they are multi-threaded or not, if they can be distributed or not (distributed controllers will be considered in the next section). Also we classify all controllers by their ability to support OpenStack, which is an open source platform for creating and managing public or private cloud. V.
PERFORMANCES IMPROVEMENT USING MULTIPLE CONTROLLERS
With the application of SDN/OpenFlow in large networks, the controller could become a performance bottleneck due to the large amount of incoming flow requests. To prevent bottlenecks and more generally to improve performances, architectures with distributed controllers have been proposed for SDN networks. In this section, we describe various design options for a control plane with multiple controllers.
SDN CONTROLLER’S CHARACTERISTICS POX Python Nicira 1.0 30 No No No Easy
Ryu Python NTT, OSRG group 1.0,1.1,1.3,1.4 33 No Yes Yes Moderate
Floodlight Java BigSwitch 1.0, Floodlight plus:1.3 1500 Yes Yes Yes Steep
ODL Java Linux Foundation 1.0, 1.3 – Yes Yes Yes Steep
334
A. Number and Location of Controllers Using multiple controllers offers many advantages, but also raises several design questions such as the number of controllers needed in a given topology and their placement. In [19], the authors focus on the WAN deployments where latency dominates. The best controller placement relies on propagation latency that bounds the control reactions that can be executed at reasonable speed and stability with a remote controller. They conclude that there are no placement rules that applies to every network and that the controllers’ number depends on the network topology and the metric choice. The authors of [10] focus on the reliability of the network, by measuring the number of failures during long-term testing and under a given workload profile. The experiments have shown that most of the controllers successfully cope with the test load, although two controllers, MuL and Maestro, start to drop PacketINs after several minutes of work. The controller responsiveness is the primary factor to decide if additional controllers should be deployed. For wide-area SDN deployments, multiple controllers are often required, and the placement of these controllers influences every aspect of an SDN. Authors in [20] search for the most reliable controller placements for SDNs, they first present a novel metric that reflects the reliability of the SDN control network, called expected percentage of valid control paths when network failures happen. They show that placement of controllers should be carefully chosen and depends on the specific algorithm used, and a greedy algorithm provides solutions that are close to optimal. Another similar study [21] attempts to optimizes the reliability of SDN control network. They define reliability metric as the same defined previously, where the control path loss is the number of broken control paths due to network failures. The optimization target is then to minimize the expected percentage of control path loss. To evaluate reliability, they use the brute force algorithm, and they measure the cost of each placement of K controllers, and they also keep the best one. They conclude that using a few controllers reduces reliability. However, for all topologies past a certain amount of controllers, adding more controllers has an adverse effect. Hence the best controller number, K, is in between [0.035n, 0.117n] where K is the number of controllers and N the number of nodes. Recently, proposals have been made to physically distribute controllers. We present next two main classes of solutions that aim to avoid having a Single Point Of Failure (SPOF) for the entire network and allowing a scalable architecture. The first category proposes a physically distributed control-plane, but, it is logically centralized by synchronizing all network information and balancing load among several controllers. The second category suggests a logically distributed control-plane, where each controller manages its domain and distribute useful information to other instances within the cluster and communicate if necessary with the neighboring domain.
© 2015 ACADEMY PUBLISHER
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
B. Logically Centralized Control Plane This kind of solutions is focused on improving the performance of SDN networks by sharing the charge between controllers. Each controller must synchronize all information about its portion of the network. We can benefit from these solutions in data centers where the controller instances share a huge amount of information to ensure fine-grained network wide consistency. The first attempt that suggests using a physically distributed control-plane is Hyperflow [22]. In this project, all controllers synchronize their network wide view using a distributed file system. To facilitate cross-controller communication, they employ the Publish/Subscriber messaging communication. As a result, Hyperflow can handle more flow events while keeping the flow setup latency minimal. With the same concept, Onix [23] provides a distributed system which runs on a cluster. It's responsible for control logic programmatic access to the network, moreover it distributes network state to other instances within the cluster, the Onix team designs four components in a network that is controlled by Onix, physical infrastructure (switch and router), connectivity infrastructure (control channel), Onix and Control logic. Similarly, KANDOO [24] introduces a hierarchical distribution controller based on two layers of controlplane, the bottom layer contains a group of local controllers that manage local applications with no knowledge of network wide state, and a centralized root controller builds the top layer, its role is to determine the desired network behavior and to run non-local applications. As a result they reduce the number of events that are received at the control plane of the network. All these solutions are physically distributed but logically centralized, they propose a different architecture and layer of a SDN network, though they offer a simplified central view of the network and decrease the look-up overhead by enabling communication with local controllers. However, there are not adapted to large networks with several Autonomous Systems (AS), and they require extensive traffic between controllers to keep a network-wide view. C. Logically Distributed Control Plane Another approach to use multiple controller is logically distribute the control plane, Thales Communications proposes a logically DIstributed SDN COntrollers (DISCO) [25], DISCO controllers administrate their own network domain and communicate with each other using a unique manageable control channel called Agent. Hence, it provides an open distributed control plane for multi-domain network based on a unique messageoriented communication. The general success behind DISCO is that they separate control plane on two domains, the intra-domain part, round up the main functionalities of the controllers, and the inter-domain piece, manages the communication with other controllers by sending selected Agent type necessary for reservation, topology state and disruption operations. This solution has been tested in inter-domain topology interruption, end-to-end priority service request and VM migration
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
335
Figure 3. Distributed architecture of SDN networks
cases. For instance, the VM migration process can be feasible using Reachability agent between domains. Currently, the IETF is working on developing an eastwest protocol called SDNi to achieve interconnection between SDN controllers. In this project the authors describe the interfaces for exchanging information among multiple SDN domains in order to synchronize network database and coordinate their decisions (Figure 3). Like any distributed architecture, each controller collects the local physical state and exchanges its network information between neighboring controllers in order to distribute its domain state to another controller domain. The main advantage of this type of solutions is its adaptation to large distributed networks that contain several AS such as Internet. But, many open questions remain, like the protocol of communication between controllers, synchronization cost, AS policy agreement. In this context, we work on a special interface that could answer some above questions. This interface is implemented between SDN controllers and share useful information based on the communication modes of each controller in the network. These mode depend on the desirable behavior of the controller, its critical position, and its performance. VI.
PERFORMANCE EVALUATION TECHNIQUES OF OPENFLOW NETWORKS
The three general techniques to evaluate the performances of a network are analytical models, simulations and measurements on the physical network or an experimentation platform. In this section, we give an overview on the techniques that were effectively used in the particular case of OpenFlow Networks. We have already mentioned that the three techniques were applied in this context, but the most and common used technique is to consider a simulation or emulation tool. A. Analytical Models To the best of our knowledge, there are a very few performances studies of SDN/OpenFlow networks based on analytical models. We mention here two works. The first work based on queuing theory [26] allows to evaluate the probability of dropping packets and the packet delay in the system, however, the model is quite basic and does not capture all the complexity of the SDN architecture. The second work [27] is based on network © 2015 ACADEMY PUBLISHER
calculus formalism and it allows analyzing delay and queue size boundaries of SDN switches and controllers. B. Experimentation Platforms In order to help researchers to test and validate new mechanisms and applications in the domain of SDN/OpenFlow, many experimentation platforms with real hardware have been developed such as GENI [28] in USA, AKARI [29] in Japan, FEDERICA [30], NOVI and OFELIA [31] in Europe. For example, OFELIA community creates real-world experimental networking substrate, and it provides for external experimenters a lot of island equipment (i2CAT, IBBT, ETHZ) for free, where each island supplies different OpenFlow switches and servers capabilities. Performances metrics measured on this kind of large testbeds are close to reality and allow having a good idea on the behavior of the studied system before eventually moving to the stage of its industrialization. Many projects are tested under these platforms such us Openflow protocol [32] in GENI and data plane performance [11] in FEDERICA. C. Simulation and Emulation Tools Several simulation and emulation tools have been developed to implement OpenFlow-based networks on a single machine and to test new applications. For example, Mininet [33] an SDN emulation environment developed by the Stanford University, it can be used to deploy a virtual network and estimate performance metrics for various network topologies and sizes. Another option is to use the NS3 [34] network simulator that supports OpenFlow within its environment. For performance analysis of OpenFlow switches, an open framework called OFLOPS [35] has been developed. It permits the development of tests for OpenFlow switches such as CPU utilization, packet counters, then it determines the bottlenecks between the switch and the remote control application. On the other hand, for performance analysis of controllers, an OpenFlow controller benchmarker called Cbench [36], has also been developed. Cbench emulates a bunch of Open-Flow switches connect to a controller then computes the performance metrics like throughput, response time, and latency of an SDN controller. Another tool proposes a software-defined traffic measurement architecture called OpenSketch [37], it collects measurement information
336
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
from data plane and the control plane. In the data plane, OpenSketch provides a simple three-stage pipeline (hashing, filtering, and counting). However, in the control plane, OpenSketch provides a measurement library that automatically configures the pipeline and it allocates resources for different measurement task. VII. CONCLUSION In this paper, we attempt to highlight the most important studies that provide a solution to improve the performances of an SDN network for both data plane and control plane. In fact, since the emergence of Software Defined Networks, several works have focused on their performance. This does not mean they pose more performance problems than traditional networks, but, it rather reflects a great interest towards this new technology which allows programmability and ease management of networks. Through the different presented studies, we note that the CPU processing rate, throughput, and the processing delays are the key factor that enhances the performance of the data plane. Also, we find that control plane which is the core layer in SDN architecture, received many attentions regarding the performance, and recent controllers are becoming more efficient. However, we think that multiple controller deployments, especially for WAN, require deeply analysis and more investigations: The impact of coordination tasks across multiple controllers on performance, scalability and elasticity, must be studied in-depth. New sharing data mechanisms are needed to allow controllers to share information concerning their networks more completely with more useful information such as the traffic charge of neighboring domains. Ensuring end-to-end Quality of Service (QoS) in distributed architecture. The SDN paradigm and OpenFlow protocol are advantageous for WAN networks as shown by Google that has deployed a centralized Traffic Engineering using SDN/OpenFlow in its inter-data center [38]. This may be an important future research direction. REFERENCE: [1] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, and S. Shenker, “NOX: towards an operating system for networks,” ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 3, pp. 105–110, 2008. [2] “Floodlight OpenFlow Controller -Project Floodlight.” [Online]. Available: http://www.projectfloodlight.org/floodlight/. [Accessed: 27-Apr-2015]. [3] “OpenDaylight - An Open Source community and Meritocracy for Software-Defined Networking.” [4] B. A. A. Nunes, M. Mendonca, X.-N. Nguyen, K. Obraczka, and T. Turletti, “A Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks,” IEEE Commun. Surv. Tutor., vol. 16, no. 3, pp. 1617–1634, Third 2014. [5] W. Xia, Y. Wen, C. H. Foh, D. Niyato, and H. Xie, “A Survey on Software-Defined Networking,” IEEE Commun. Surv. Tutor., vol. 17, no. 1, pp. 27–51, Firstquarter 2015.
© 2015 ACADEMY PUBLISHER
[6] D. Kreutz, F. M. Ramos, P. Esteves Verissimo, C. Esteve Rothenberg, S. Azodolmolky, and S. Uhlig, “Softwaredefined networking: A comprehensive survey,” Proc. IEEE, vol. 103, no. 1, pp. 14–76, 2015. [7] A. Lara, A. Kolasani, and B. Ramamurthy, “Network Innovation using OpenFlow: A Survey,” IEEE Commun. Surv. Tutor., vol. 16, no. 1, pp. 493–512, First 2014. [8] A. Tootoonchian, S. Gorbunov, Y. Ganjali, M. Casado, and R. Sherwood, “On Controller Performance in Software-defined Networks,” in Proceedings of the 2Nd USENIX Conference on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services, Berkeley, CA, USA, 2012, pp. 10–10. [9] K. Phemius and M. Bouet, “OpenFlow: Why latency does matter,” in 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), 2013, pp. 680–683. [10] A. Shalimov, D. Zuikov, D. Zimarina, V. Pashkov, and R. Smeliansky, “Advanced Study of SDN/OpenFlow Controllers,” in Proceedings of the 9th Central & Eastern European Software Engineering Conference in Russia, New York, NY, USA, 2013, pp. 1:1–1:6. [11] A. Bianco, R. Birke, L. Giraudo, and M. Palacin, “OpenFlow Switching: Data Plane Performance,” in 2010 IEEE International Conference on Communications (ICC), 2010, pp. 1–5. [12] V. Tanyingyong, M. Hidell, and P. Sjodin, “Improving PCbased OpenFlow switching performance,” in 2010 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), 2010, pp. 1–2. [13] J. C. Mogul and P. Congdon, “Hey, You Darned Counters!: Get off My ASIC!,” in Proceedings of the First Workshop on Hot Topics in Software Defined Networks, New York, NY, USA, 2012, pp. 25–30. [14] A. Sivaraman, K. Winstein, S. Subramanian, and H. Balakrishnan, “No Silver Bullet: Extending SDN to the Data Plane,” in Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks, New York, NY, USA, 2013, pp. 19:1–19:7. [15] G. Lu, R. Miao, Y. Xiong, and C. Guo, “Using CPU As a Traffic Co-processing Unit in Commodity Switches,” in Proceedings of the First Workshop on Hot Topics in Software Defined Networks, New York, NY, USA, 2012, pp. 31–36. [16] Y. Luo, P. Cascon, E. Murray, and J. Ortega, “Accelerating OpenFlow Switching with Network Processors,” in Proceedings of the 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems, New York, NY, USA, 2009, pp. 70–71. [17] R. Sherwood, M. Chan, A. Covington, G. Gibb, M. Flajslik, N. Handigol, T.-Y. Huang, P. Kazemian, M. Kobayashi, J. Naous, S. Seetharaman, D. Underhill, T. Yabe, K.-K. Yap, Y. Yiakoumis, H. Zeng, G. Appenzeller, R. Johari, N. McKeown, and G. Parulkar, “Carving Research Slices out of Your Production Networks with OpenFlow,” SIGCOMM Comput Commun Rev, vol. 40, no. 1, pp. 129– 130, Jan. 2010. [18] D. Drutskoy, E. Keller, and J. Rexford, “Scalable Network Virtualization in Software-Defined Networks,” IEEE Internet Comput., vol. 17, no. 2, pp. 20–27, Mar. 2013. [19] B. Heller, R. Sherwood, and N. McKeown, “The Controller Placement Problem,” in Proceedings of the First Workshop on Hot Topics in Software Defined Networks, New York, NY, USA, 2012, pp. 7–12. [20] Y. HU, W. WANG, X. GONG, X. QUE, and S. CHENG, “On the placement of controllers in software-defined
JOURNAL OF NETWORKS, VOL. 10, NO. 6, JUNE 2015
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28] [29]
[30]
[31] [32]
networks,” J. China Univ. Posts Telecommun., vol. 19, Supplement 2, pp. 92–171, Oct. 2012. Y. Hu, W. Wendong, X. Gong, X. Que, and C. Shiduan, “Reliability-aware controller placement for SoftwareDefined Networks,” in 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), 2013, pp. 672–675. A. Tootoonchian and Y. Ganjali, “HyperFlow: A distributed control plane for OpenFlow,” in Proceedings of the 2010 internet network management conference on Research on enterprise networking, 2010, pp. 3–3. T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, R. Ramanathan, Y. Iwata, H. Inoue, T. Hama, and S. Shenker, “Onix: A Distributed Control Platform for Large-scale Production Networks,” in Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, Berkeley, CA, USA, 2010, pp. 1–6. S. Hassas Yeganeh and Y. Ganjali, “Kandoo: A Framework for Efficient and Scalable Offloading of Control Applications,” in Proceedings of the First Workshop on Hot Topics in Software Defined Networks, New York, NY, USA, 2012, pp. 19–24. K. Phemius, M. Bouet, and J. Leguay, “DISCO: Distributed multi-domain SDN controllers,” in 2014 IEEE Network Operations and Management Symposium (NOMS), 2014, pp. 1–4. K. Mahmood, A. Chilwan, O. N. Østerbø, and M. Jarschel, “On the Modeling of OpenFlow-based SDNs: The Single Node Case,” ArXiv Prepr. ArXiv14114733, 2014. S. Azodolmolky, R. Nejabati, M. Pazouki, P. Wieder, R. Yahyapour, and D. Simeonidou, “An analytical model for software defined networking: A network calculus-based approach,” in 2013 IEEE Global Communications Conference (GLOBECOM), 2013, pp. 1397–1402. “GENI.” . H. Harai, “AKARI architecture design for new generation network,” in Summer Topical Meeting, 2009. LEOSST ’09. IEEE/LEOS, 2009, pp. 155–156. M. Campanella and F. Farina, “The FEDERICA Infrastructure and Experience,” Comput Netw, vol. 61, pp. 176–183, Mar. 2014. “Ofelia - Home.” [Online]. Available: http://www.fp7ofelia.eu/. [Accessed: 24-Jun-2015]. N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner, “OpenFlow: enabling innovation in campus networks,” ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 2, pp. 69–74, 2008.
© 2015 ACADEMY PUBLISHER
337
[33] B. Lantz, B. Heller, and N. McKeown, “A Network in a Laptop: Rapid Prototyping for Software-defined Networks,” in Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks, New York, NY, USA, 2010, pp. 19:1–19:6. [34] Carneiro, G. J., “NS3.” . [35] C. Rotsos, N. Sarrar, S. Uhlig, R. Sherwood, and A. W. Moore, “OFLOPS: An Open Framework for Openflow Switch Evaluation,” in Proceedings of the 13th International Conference on Passive and Active Measurement, Berlin, Heidelberg, 2012, pp. 85–95. [36] R. Sherwood and Y. KOK-KIONG, Cbench: an open-flow controller benchmarker. 2010. [37] M. Yu, L. Jose, and R. Miao, “Software Defined Traffic Measurement with OpenSketch,” in Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation, Berkeley, CA, USA, 2013, pp. 29–42. [38] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Hölzle, S. Stuart, and A. Vahdat, “B4: Experience with a Globally-deployed Software Defined Wan,” in Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM, New York, NY, USA, 2013, pp. 3–14.
Fouad Benamrane received his B.Sc. degree (2009) in physics at the Faculty of Sciences of Rabat (FSR), Morocco, and his M.Sc. degree (2011) in telecommunication sciences at the National School of Applied Sciences of Fez, Morocco (ENSAF). He is currently a Ph.D. candidate at the FSR. His current research focuses on the performance and scalability of softwaredefined networks and the OpenFlow protocol. Mouad Ben Mamoun is currently a Professor in the Department of Computer Science at Mohammed V University, Rabat, Morocco. He received his M.Sc. and Ph.D. degrees in Computer Science from the University of Versailles, France, in 1998 and 2002. His research interests are about performance evaluation of networks. Redouane BENAINI is an Assistant Professor in Computer Networking at Mohamed V University, Rabat, Morocco, since 2006. Before that, he served as a Temporary Assistant Professor at Marne-la-Vallée University in Paris and in the National School of Engineers at Bourges, France. He received a PhD degree in Computer Science from TELECOM SudParis, in 2005. His main activities are focused on network protocols.
