Applying Software-defined Networks to Cloud Computing - SBRC 2015

Capítulo

1 Applying Software-defined Networks to Cloud Computing Bruno Medeiros de Barros (USP), Marcos Antonio Simplicio Jr. (USP), Tereza Cristina Melo de Brito Carvalho (USP), Marco Antonio Torrez Rojas (USP), Fernando Frota Redígolo (USP), Ewerton Rodrigues Andrade (USP), Dino Raffael Cristofoleti Magri (USP)

Abstract Network virtualization and network management for cloud computing systems have become quite active research areas in the last years. More recently, the advent of the Software-Defined Networks (SDNs) introduced new concepts for tackling these issues, fomenting new research initiatives oriented to the development and application of SDNs in the cloud. The goal of this course is to analyze these opportunities, showing how the SDN technology can be employed to develop, organize and virtualize cloud networking. Besides discussing the theoretical aspects related to this integration, as well as the ensuing benefits, we present a practical a case study based on the integration between OpenDaylight SDN controller and OpenStack cloud operating system.

1.1. Introduction The present section introduces the main topics of the course, providing an evolutionary view of network virtualization in cloud computing and distributed systems. We present the main changes occurred in the field in the latest years, focusing in the advent of Softwaredefined Networks (SDN) and its implications in the current research scenario. 1.1.1. The Role Networking in Cloud Computing Cloud computing has ushered the information technology (IT) field and service providers into a new era, redefining how computational resources and services are delivered and consumed. With cloud computing, distinct and distributed physical resources such as computing power and storage space can be acquired and used in an on-demand basis, empowering applications with scalability and elasticity at low cost. This allows the creation of different service models, generally classified as [Mell and Grance 2011]:

Infrastructure-as-a-Service (IaaS), which consists in providing only fundamental computing resources such as processing, storage and networks; Platform-as-a-Service (PaaS), in which a development platform with the required tools (languages, libraries, etc.) is provided to tenants; and Software-as-a-Service (SaaS), in which the consumer simply uses the applications running on the cloud infrastructure. To actually provide cost reductions, the cloud needs to take advantage of economies of scale, and one key technology for doing so is resource virtualization. After all, virtualization allows creation of a logical abstraction layer above the pool of physical resources, thereby enabling a programmatic approach to allocate resource wherever needed while hiding the complexities involved in their management. The result is potentially very efficient resource utilization, better manageability, on-demand and programmatic resource instantiation, and resource isolation for better control, accounting and availability. In any cloud environment, the network is a critical resource that connects various distributed and virtualized components, such as servers, storage elements, appliances and applications. For example, it is the network that allows aggregation of physical servers, efficient virtual machine (VM) migration, and remote connection to storage systems, effectively creating the perception of large, monolithic resource pool. Furthermore, it is also the network that enables delivery of cloud based applications to end users. Yet, while every component in a cloud is getting virtualized, the physical network connecting these components is not. Without virtualization, the network is one physical common network, shared by all cloud end-users and cloud components. Without virtualization, the network likely becomes a single complex system in the cloud as the cloud evolves to provide new services with diverse requirements while trying to sustain the scale. 1.1.2. The Advent of Software-Defined Networks (SDNs) The term SDN originally appeared in [Greene 2009], referring to the ability of OpenFlow [McKeown et al. 2008] to support the configuration of table flows in routers and switches using software. However, the ideas behind SDNs come from the goal of having a programmable network, whose research started short after the emergence of the Internet, led mainly by the telecom industry. Today, the networking industry has shown enormous interest in the SDN paradigm, given the expectations of reducing both capital and operational costs with service providers and enterprise data centers with programmable, virtualizable and easily partitionable networks. Actually, programmability is also becoming a strategic feature for network hardware vendors, since it allows a many devices to be programmed and orchestrated in large network deployments (e.g., data centers). In addition, discussions related to the future Internet has led to the standardization of SDN-related application programming interfaces (API), with new communication protocols being successfully deployed on experimentation and real scenarios [Kim et al. 2013, Pan et al. 2011]. These features of SDNs make them highly valuable for cloud computing systems, where the network infrastructure is shared by a number of independent entities and, thus, network management becomes a challenge. Indeed, while the first wave of innovation in the cloud focused on server virtualization technologies and on how to abstract com-

putational resources such as processor, memory and storage, SDNs are today promoting a second wave with network virtualization [Lin et al. 2014]. The emergence of large SDN controllers focused on ensuring availability and scalability of virtual networking for cloud computing systems (e.g., OpenDayLight [Medved et al. 2014] and OpenContrail [OpenContrail 2014]) is a clear indication of this synergy between both technologies. Besides the cloud, SDNs have also been adopted in other computing scenarios, with device vendors following the SDN path and implementing most of control logic in software over standard processors. This has led to the emergence of software-defined base stations, software defined optical switches, software-defined radios, software-defined routers, among others.

1.2. Cloud Computing and Network Virtualization This section aims to introduce the concepts and technologies related to network virtualization in cloud computing systems. We start by describing the main virtualization technologies used to implement multitenant networks. Then, we present an architectural view of virtual networks in the cloud, discussing the main components involved, their responsibilities and existing interfaces. Finally, we focus on security, scalability and availability aspects of the presented solutions. 1.2.1. Cloud Computing and Resource Virtualization Virtualization is not a new concept in computing, having in fact appeared in the 70’s [Jain and Paul 2013a, Menascé 2005]. The concept of virtualization has evolved with time, however, going from virtual memory to processor virtualization (e.g., Hyper-V, AMD-V, Hyper-threading) up to the virtualization of network resources (e.g., SDN, OpenvSwitch, etc). With the advent of cloud computing and the demand of virtualizing entire computing environments, new virtualization techniques were developed, among them [Amazon 2014]: • Full Virtualization or Hardware VM: all hardware resources are simulated via software. The hardware itself is not directly accessed by VMs, as the hypervisor translates all interruptions and calls between the virtual and physical appliances. Obviously, this technique incurs performance penalties due to I/O limitations, so it is less efficient than its counterparts. However, it offers high flexibility, as systems running on VMs do not need to be altered if there is a change on the underlying physical hardware; • Para-Virtualization: the hardware is not simulated, but divided in different domains so they can be accessed by VMs. Systems running on VMs need to be adapted so that they can directly access the physical machine’s hardware resources. Performance here is close to the performance on the physical machine (bare-metal), with the drawback of limited flexibility, as hardware upgrades may demand changes on VMs. • Para-virtualized drivers (Para+Full Virtualization): a combination of the previous techniques. As para-virtualized storage and networking devices have a much better performance than their full-virtualized counterparts [Amazon 2014], this is the technique applied to these devices, while full-virtualization (and the consequent flexibility brought

by it) is applied to devices whose performance is not critically affected. This approach allows minimum changes when physical hardware upgrades are needed. Several studies highlight the benefits of virtualization on a computing environment. Among them, the following can be cited [Menascé 2005, Kotsovinos 2010]: • Resource sharing: when a device has more resources than what can be consumed by a single entity, those resources can be shared among different users or processes for better usage efficiency. For example, the different user applications or VMs running on a server can share its multiple processors, storage disks or network links. If properly executed, the economy achieved in small server consolidation onto VMs, for example, can be from 29 % to 64 % [Menascé 2005]; • Resource aggregation: devices with a low availability of resources can be combined to create a larger-capacity virtual resource. For example, with an adequate file management system, small-size magnetic disks can be combined to create the impression of a large virtual disk. • Ease of management: one of the main advantages of virtualization is that it facilitates maintenance of virtual hardware resources. One reason is that virtualization usually provides standard software interfaces that abstract the underlying hardware (except for para-virtualization). In addition, legacy applications placed in virtualized environments can keep running even after being migrated to a new infrastructure, as the hypervisor becomes responsible to translate old instructions into those comprehensible by the underlying physical hardware. • Dynamics: with the constant changes to application requirements and workloads, rapid resource realocation or new resource provisioning becomes essential for fulfilling these new demands. Virtualization is a powerful tool for this task, since virtual resources can be easily expanded, realocated, moved or removed without concerns about which physical resources will support the new demands. As an example, when a user provisions a dynamic virtual disk, the underlying physical disk does not need to have that capacity available at provisioning time: it just needs to be available when the user actually needs to use it. • Isolation: multiple users environments may contain users that do not trust on each other. Therefore, it is essential that all users have their resources isolated from other users, even if this is done logically (i.e., in software). When this happens, malicious users are unable to monitor and/or interfere with other users’ activities, preventing a vulnerability or attack to a given machine from affecting other users. Despite their benefits, there are also disadvantages of virtualized environments, such as [Kotsovinos 2010]: • Performance: even though there is no single method for measuring performance, it is intuitive that the extra software layer of the hypervisors leads to higher processing costs than a comparable system with no virtualization. • Management: virtual environments abstract physical resources in software and files, so they need to be instantiated, monitored, configured and saved in an efficient and

auditable manner, which is not always an easy task. • Security: whereas isolation is a mandatory requirement for VMs in many real case scenarios, completely isolating a virtualized resource from another, or applications running on the physical hardware from virtualized ones, are involved (if not impossible) tasks. Therefore, it is hard to say whether or not a physical server hosting several virtualized applications is monitoring them with the goal of obtaining confidential information, or even whether a VM is somehow attacking or monitoring another VM. 1.2.2. Mechanisms for Network Virtualization To understand the mechanisms that can implement network virtualization, first we need to understand which resources can be virtualized in a network. In terms of resources, networks are basically composed by network interface cards (NICs) connected to a layer 2 network through a switch. These layer 2 networks can be connected through routers to form a layer 3 network, which in turn can be connected via routers to compose the Internet. Each of these network components — NIC, L2 network, L2 switch, L3 networks, and L3 routers — can be virtualized [Jain and Paul 2013b]. However, there are multiple, often competing, mechanism for virtualizing these resources, as discussed in what follows: • Virtualization of NICs: Every networked computer system is composed by at least one NIC. In virtualized environments with multiple VMs, it becomes, thus, necessary to provide every VM with its own virtual NIC (vNIC). This need is currently satisfied by the hypervisor software, which is able to provide as many vNICs as the number of VMs under its responsibility. The vNICs are connected to the physical NIC (pNIC) through a virtual switch (vSwitch), just like physical NICs can be connected through a physical switch to compose layer 2 networks. This NIC virtualization strategy has benefits such as transparency and simplicity and, thus, is generally proposed by software vendors. Nevertheless, there is an alternative design proposed by pNIC (chip) vendors, which is to virtualize NIC ports using single-route I/O virtualization (SR-IOV) [PCI-SIG 2010] on the peripheral-component interconnect (PCI) bus. This approach directly connects the VMs to the pNICs, potentially providing better performance (as it eliminates intermediary software) and resource isolation (as the traffic does not go through a shared vSwitch). A third design approach, promoted by physical switch vendors, is to provide virtual channels for inter-VM communication using a virtual Ethernet port aggregator (VEPA) [IEEE 2012b], which in turn passes VM frames to an external switch that implements inter-VM communication. This approach not only frees up server resources, but also provides better visibility and control over the traffic between any pair of VMs. • Virtualization of L2 Switches: The number of ports in a typical switch is limited, and possibly lower than the number of physical machines that need to be connected on an L2 network. Therefore, several layers of L2 switches need to be connected to address network scalability requirements. To solve this issue, IEEE Bridge Port Extension standard 802.1BR [IEEE 2012a] proposes a virtual bridge with a large number of ports using physical or virtual port extenders (like a vSwitch). • Virtualization of L2 Networks: In a multitenant data center, VMs in a single physical machine may belong to different clients and, thus, need to be in different

virtual LANs (VLANs) [IEEE 2014]. VLANs implement package tagging for allowing L2 devices to isolate clients’ traffic in different logical L2 networks, so different virtual networks can use the same addressing space for different clients. • Virtualization of L3 Networks: When the multitenant environment is extended to a layer 3 network, there are a number of competing proposals to solve the problem. Examples include: virtual extensible LANs (VXLANs) [Mahalingam et al. 2014b]; network virtualization using generic routing encapsulation (NVGRE) [Sridharan et al. 2011]; and the Stateless Transport Tunneling (STT) protocol [Pan and Wu 2009]. • Virtualization of L3 Router: Multicore processors allow the design of networking devices using software modules that run on standard processors. By combining many different software-based functional modules, any networking device (L2 switch, L3 router, etc.) can be implemented in a cost-effective manner while providing acceptable performance. Network Function Virtualization (NFV) [Carapinha and Jiménez 2009] provides the conceptual framework for developing and deploying virtual L3 routers and other layer 3 network resources. 1.2.3. Virtual Network Applications in Cloud Computing As discussed in Section 1.1.1 the interest surrounding network virtualization has been fueled by cloud computing and its isolation and scalability requirements. All the network virtualization mechanisms presented in Section 1.2.2 can be applied to solve specific network issues in cloud computing, in especial for the implementation of multitenant data centers. Specifically, as depicted in Figure 1.1, a data center consist mainly of servers in racks interconnected via a top-of-rack Ethernet switch, which in turn connects to an aggregation switch, also known as an end-of-rack switch. The aggregation switches then connect to each other, as well as the other servers in the data center. A core switch connects the various aggregation switches and provides connectivity to the outside world, typically through layer 3 networks. In multitenant data centers, client VMs are commonly placed in a different server, connected through the L2 network composed by this switch-enabled infrastructure. The virtualization of L2 switches via mechanisms such as VLAN enables the abstraction of tenant L2 networks on the distributed cloud data center, allowing traffic isolation of tenant networks with a different logical addressing space. Similarly, the virtualization of L3 routers using technologies such as VXLAN and GRE tunneling enable the abstraction of layer 3 networks, connecting multiple data centers and allowing tenant networks to be distributed in different sites. Another inherent characteristic of multitenant data centers is the virtualization of servers, enabling the instantiation of multiple VMs. VMs deployed in the same cloud server commonly belong to different tenants and share the same computing resources, including the network interface (NIC). Mechanisms to virtualize server NICs such as virtual switches (i.e., in software) and SR-IOV (i.e., in hardware) are necessary to address multi-tenancy. Besides virtual switches, other software-based virtualization mechanisms are enabled by the NFV approach. NFV consists on the virtualization of network functional classes, such as routers, firewalls, load balancers and WAN accelerators. These appliances take the form software-based modules that can be deployed in one or more VMs running on top of servers.

Figure 1.1. Cloud data center network.

1.2.4. Security, Scalability and Availability aspects The widespread adoption of cloud computing has emerged important security concerns inherited from multi-tenant environments. The need for isolating the server and network resources consumed by different tenants is an example of the security requirements introduced by cloud computing. Virtualization technologies such as the presented in section 1.2.2 play a crucial role in these scenario as mechanisms to enforce the resource isolation. Considering the cloud networking scenario, virtualization technologies should provide secure network domains for the cloud tenants, enabling secure connectivity for the services running inside the cloud. Understand the nature of security threats is a fundamental component of managing risks and implementing secure network services for cloud tenants. According to [Barros et al. 2015], there are three threat scenarios in cloud computing networking. The scenarios are explained below. • Tenant-to-Tenant: Threats related to attacks promoted by a legitimate tenant targeting another legitimate tenant exploiting security vulnerabilities in the cloud networking infrastructure. • Tenant-to-Provider: Threats related to cloud vulnerabilities that allow a legitimate tenant to disrupt the operation of the cloud infrastructure, preventing the cloud provider from delivering the service in accordance with the service level agreements (SLAs) established with other legitimate tenants. • Provider-to-Tenant: Threats related to vulnerabilities in the cloud provider infrastructure, which allows malicious insider attacks from employees and other agents with

direct access to the cloud infrastructure. Different scenarios can also originate different groups of security threats in cloud networking scenarios. Consequently, different groups of security solutions built upon network virtualization mechanisms should be applied to ensure secure cloud services provision. Also according to [Barros et al. 2015], the sources of security threats concerned to cloud networking scenario are described as follow. • Physical isolation: Security threats originated from shared phyisical resources in the underlying network infrastructure, such as server NICs, switches and routers. Attacks are commonly related to hijacking and analyzing tenant data from shared resources or even causing resource exhaustion on shared physical network elements. • Logical isolation: Security threats originated from shared virtual resources such as virtual switches, Linux bridge and virtual routers. Security attacks commonly exploit vulnerabilities in software-based virtualization mechanisms to access unauthorized data and to reduce the quality of cloud network services. • Authentication: Security threats originated from authentications vulnerabilities related to inadequate authentication, which allow attackers to mask their real identities. This can be accomplished by exploiting authentication protocols, acquiring credentials and/or key materials by capturing data traffic, or via password recovery attacks (e.g., brute force or dictionary attacks). • Authorization: Security threats originated by vulnerabilities related to authorization problems, allowing granting or scaling rights, permissions or credentials to or from an unauthorized user. For example, the attacker can exploit a vulnerability in the cloud platform authorization modules, or even in the victim computer, to create or change it’s credentials in order to obtain privileged rights. • Insecure API: Security threats related to failures, malfunctions and vulnerabilities in APIs that compose the cloud system. Attacks of this class try to exploit insecure interfaces for accessing or tampering with services running in other tenants or cloud administrative tools. Following the principles of cloud computing, the cloud networking should be highly scalable. Tha scalability of cloud networks are directly related to the features provided by the network virtualization mechanisms. The use of technologies such as VLAN and SR-IOV have intrinsic scalability limitations related to the number of VMs hosted in the same node. The capacity to replicate and migrate virtual domains in cloud computing are fundamental keys to ensure the availability of cloud services. Redundant links in the underlying infrastructure, as well as eliminating single point of failure in physical and virtual network resources, are good practices for network availability in multi-tenant environments.

1.3. Software-defined Networks (SDNs) The current section introduces the concept of SDNs, as well as its importance in the network virtualization scenario. Introducing the conceptual and practical division between

control plane and data plane, we explore the opportunities to apply SDN technologies in different network architectures, focusing on the role of SDN control layer in network virtualization deployments. We also present a reference architecture to implement virtual networks in real scenarios. This section also present an evolutionary view of the SDN controllers currently available in the market, aiming to support network professionals and decision maker to adopt the right SDN approach in his deployment. We finish by focusing on security, scalability and availability aspects of the presented solutions. 1.3.1. Creating Programmable Networks: a Historical Perspective Recently, there has been considerable excitement surrounding the SDN concept, which is explained by the emergence of new application areas such as network virtualization and cloud networking. However, the basic ideas behind the SDN technology are actually a result of more than 20 years of advances in the network field, in especial the interest of turning computer networks into programmable systems. Aiming to give an overview of this evolution, we can divide the historical advancements that culminated in the SDN concept into the three different phases [Feamster et al. 2013], as follows: 1. Active Networks (from the mid-1990s to the early 2000s): This phase follows the historical advent of the Internet, a period in which the demands for innovation in the computer networks area were met mainly by the development and tests of new protocols in laboratories with limited infrastructure and simulation tools. In this context, the so-called “active networks” appeared as a first initiative aiming to turn network devices (e.g., switches and routers) into programmable elements and, thus, allow furthers innovations in the area. This programmability could then allow a separation between the two main functionalities of networking elements: the control plane, which refers to the device’s ability to decide how each packet should be dealt with; and the data plane, which is responsible for forwardind packets at high speed following the decisions made by the control plane. Specifically, active networks introduced an new paradigm for dealing with the network’s control plane, in which the resources (e.g., processing, storage, and packet queues) provided by the network elements could be accessed through application programming interfaces (APIs). As a result, anyone could develop new functionalities for customizing the treatment given to the packets passing by each node composing the network, promoting innovations in the networking area However, the criticism received due to the potential complexity it would add to the Internet itself, allied to the fact that the distributed nature of the Internet’s control plane was seen as a way to avoid single points of failure, reduced the interest and diffusion of the active network concept in the industry. 2. Control- and data-plane separation (from around 2001 to 2007): After the Internet became a much more mature technology in the late 1990’s, the continuous growth in the volume of traffic turned the attention of the industry and academic communities to requirements such as reliability, predictability and performance of computer networks. The increasing complexity of network topologies, together with concerns regarding the performance of backbone networks, led different hardware manufacturers to develop embedded protocols for packet forwarding, promoting the high integration between the control and data planes seen in today’s Internet. Nevertheless, network operators and Internet Service Providers (ISPs) would still seek new management models to meet the needs from net-

work topologies ever larger and more complex. The importance of a centralized control model has become more evident, as well as the need of a separation between the control and data planes. Among the technological innovations arising from this phase, we can cite the creation of open interfaces for communications between the control and data planes such as ForCES (Forwarding and Control Element Separation)[Yang et al. 2004], whose goal was to enable a locally centralized control over the hardware elements distributed along the network topology [Caesar et al. 2005, Lakshman et al. 2004]. To ensure the efficiency of centralized control mechanisms, the consistent replication of the control logic among the data plan elements would play a key role. The development of such distributed state management techniques is also among the main technological contributions from this phase. There was, however, considerable resistance from equipment suppliers to implement open communication interfaces, which were seen as a factor that would facilitate the entry of new competitors in the network market. This ended up hindering the widespread of the separation of data and control planes, limiting the number and variety of applications developed for the control plane in spite of the possibility of doing so. 3. OpenFlow and Network Operating System (from 2007 to 2010): The ever growing demand for open interfaces in the data plane led researchers to explore different clean slate architectures for logically centralized network control [Casado et al. 2007, Greenberg et al. 2005, Chun et al. 2003]. In particular, the Ethane project [Casado et al. 2007] created a centralized control solution for enterprise networks, reducing switch control units to programmable flow-tables. The operational deployment of Ethane in the Stanford computer science department, focusing on network experimentation inside the campus, was indeed huge success, and resulted in the creation of OpenFlow protocol [McKeown et al. 2008]. OpenFlow enables fully programmable networks by providing a standard data plane API for existing packet switching hardware. The creation of the OpenFlow API, on its turn, allowed the emergence of SDN control platforms such as NOX [Gude et al. 2008], thus enabling the creation of a wide range of network applications. OpenFlow provided an unified abstraction of network devices and its functions, defining forwarding behavior through traffic flows based on 13 different instructions. OpenFlow also led to the vision of a network operating system that, different from the node-oriented system preconized by active networks, organize the network’s operation into three layers: (1) a data plane with an open interface; (2) a state management layer that is responsible for maintaining a consistent view of the overall network state; and (3) control logic that performs various operations depending on its view of network state [Koponen et al. 2010]. The need for integrating and orchestrating multiple controllers for scalability, reliability and performance purposes also led to significant enhancements on distributed state management techniques. Following these advances, solutions such as Onix [Koponen et al. 2010] and its open-source counterpart, ONOS (Open Network Operating System) [Berde et al. 2014], introduced the idea of a network information base that consists of a representation of the network topology and other control state shared by all controller replicas, while incorporating past work in distributed systems to satisfy state consistency and durability requirements. Analyzing this historical perspective and the needs recognized in each phase, it becomes easier to see that the SDN concept emerged as a tool for allowing further network innovation, helping researchers and network operators to solve longstanding prob-

lems in network management and also to provide new network services. SDN has been successfully explored in many different research fields, including areas such as network virtualization and cloud networking. 1.3.2. SDNs and the Future Internet Today’s Internet was designed more than 30 years ago, with specific requirements to connect, in a general and minimalist fashion, the (few) existing networks at the time. After it was proven to be very successful at this task, the TCP/IP model became widely adopted, in especial due to the possibility of running many distinct applications over its infrastructure while keeping the core of the network as simple as possible. However, the increase in the number of applications, users and devices making intense use of the network resources would bring many (usually conflicting) requirements with each new technology developed, turning the Internet into a completely different environment filled with disputes regarding its evolution [Moreira et al. 2009]. While in the early days of the Internet the simplicity of the TCP/IP model was considered one of its main strengths, enabling the rapid development of applications and the growth of the network itself, it became a weakness because it would imply an unintelligent network.

Figure 1.2. Ossification of the Internet

That is the main reason why TCP/IP’s simplicity is sometimes accused of being responsible for the “ossification of the Internet” (See Figure 1.2): without the ability of adding intelligence to the core of the network itself, many applications had to take corrective actions on other layers; many patches would be sub-optimal, imposing certain restrictions on the applications that could be deployed with the required levels of security, performance, scalability, mobility, maintainability, etc. Therefore, even though the TCP/IP model displays a reasonably good level of efficiency and is able to meet many of the original requirements of the Internet, many believe it may not be the best solution for the future [Alkmim et al. 2011]. Many of the factors pointed out as the cause of the Internet’s ossification are related to the strong coupling between the control and data planes, so the decision on how to treat the data flow and the execution of this decision are both handled by the same

device. In such environment, new network applications or features have to be deployed directly into the network infrastructure, a cumbersome task given the lack of standard interfaces for doing so in a market dominated by proprietary solutions. Actually, even when a vendor does provide interfaces for setting and implementing policies into the network infrastructure, the presence of heterogeneous devices with incompatible interfaces ends up hindering such seemingly trivial tasks. This ossification issue has led to the creation of dedicated appliances for tasks seem as essential for the correct network’s operation, such as firewalls, intrusion detection systems (IDS), network address translators (NAT), among others [Moreira et al. 2009]. Since such solutions are many times seen as palliative, studies aimed at changing this ossification state became more prominent, focusing especially in two approaches. The first, more radical, involved the proposal of a completely new architecture that could replace the current Internet model, based on past experiences and identified limitations. This “clean state” strategy has not received much support, however, not only to the high costs involved in its deployment, but also because it is quite possible that, after years of effort to build such specification, it might become outdated after a few decades due to the appearance of new applications with unanticipated requirements. The second approach suggests evolving the current architecture without losing compatibility with current and future devices, thus involving lower costs. By separating the data and control planes, thus adding flexibility to how the network is operated, the SDN paradigm gives support to this second strategy [Feamster et al. 2014]. According to [Open Networking Foundation 2012], the formal definition of an SDN is: “an emerging architecture that is dynamic, manageable, cost-effective, and adaptable, making it ideal for the high-bandwidth, dynamic nature of today’s applications. This architecture decouples the network control and forwarding functions enabling the network control to become directly programmable and the underlying infrastructure to be abstracted for applications and network services.” This definition is quite comprehensive, making it clear that the main advantage of the SDN paradigm is to allow different policies to be dynamically applied to the network by means of a logically centralized controller, which has a global view of the network and, thus, can quickly adapt the network configuration in response to changes [Kim and Feamster 2013]. At the same time, it enables independent innovations in the now decoupled control and data planes, besides facilitating the network state visualization and the consolidation of several dedicated network appliances into a single software implementation [Kreutz et al. 2014]. This flexibility is probably among the main reasons why companies from different segments (e.g., device manufacturers, cloud computing providers, among others) are increasingly adopting the SDN paradigm as the main tool for managing their resources in an efficient and costeffective manner [Kreutz et al. 2014]. 1.3.3. Data and Control Planes Given that the separation between data and control planes is at the core of the SDN technology, it is important to discuss them in some detail. Figure 1.3 shows a simplified SDN architecture and its main components, showing that the data and control planes are connected via a well-defined programming interface between the switches and the SDN controller.

Figure 1.3. SDN architecture overview

The data plane corresponds to the switching circuitry that interconnects all devices composing the network infrastructure, together with a set of rules that define which actions should be taken as soon as a packet arrives at one of the device’s ports. Examples of common actions are to forward the packet to another port, rewrite (part of) its header, or even to discard the packet. The control plane, on its turn, is responsible for programming and managing the data plane, controlling how the routing logic should work. This is done by one or more software controllers, whose main task is is to set the routing rules to be followed by each forwarding device through standardized interfaces, called the southbound interfaces. These interfaces can be implemented using protocols such as OpenFlow 1.0 and 1.3 [OpenFlow 2009, OpenFlow 2012], OVSDB [Pfaff and Davie 2013] and NETCONF [Enns et al. 2011] The control plane concentrates, thus, the intelligence of the network, using information provided by the forwarding elements (e.g., traffic statistics and packet headers) to decide which actions should be taken by them [Kreutz et al. 2014]. Finally, developers can take advantage of the protocols provided by the control plane through the northbound interfaces, which abstracts the low-level operations for controlling the hardware devices similarly to what is done by operating systems in computing devices such as desktops. These interfaces can be provided by remote procedure calls (RPC), restful services and other cross-application interface models. This greatly facilitates the construction of different network applications that, by interacting with the control plane, can control and monitor the underlying network. This allows them to customize the behavior of the forwarding elements, defining policies for implementing functions such as firewalls, load balancers, intrusion detection, among others.

1.3.4. The OpenFlow Protocol The OpenFlows protocol is one of the most commonly used southbound interfaces, being widely supported both in software and hardware, and standardized by the Open Networking Foundation (ONF). It works with the concept of flows, defined as groups of packets matching a specific (albeit non-standard) header [McKeown et al. 2008], which receive may be treated differently depending how the network is programmed. OpenFlow’s simplicity and flexibility, allied to the high performance at low cost, ability to isolate experimental traffic from production traffic, and to cope with vendors’ need for closed platforms [McKeown et al. 2008], are probably among the main reasons for this success. Whereas other SDN approaches take into account other network elements, such as routers, OpenFlow focus mainly on switches [Braun and Menth 2014]. Its architecture comprises, then, three main concepts [Braun and Menth 2014]: (1) the network’s data plane is composed by OpenFlow-compliant switches; (2) the control plane consists of one or more controllers using the OpenFlow protocol; (3) the connection between the switches and the control plane is made through a secure channel. An OpenFlow switch is basically a forwarding device endowed with a Flow Table, whose entries define the packet forwarding rules to be enforced by the device. To accomplish this goal, each entry of the table comprises three elements [McKeown et al. 2008]: match fields, counters, and actions. The match fields refer to pieces of information that identify the input packets, such as fields of its header or its ingress port. The counters, on their turn, are reserved for collecting statistics about the corresponding flow. They can, for example, be used for keeping track of the number of packets/bytes matching that flow, or of the time since the last packet belonging to that flow was seen (so inactive flows can be easily identified) [Braun and Menth 2014]. Finally, the actions specify how the packets from the flow must be processed, the most basic options being: (1) forward the packet to a given port, so it can be routed to through the network; (2) encapsulate the packet and deliver it to a controller so the latter can decide how it should be dealt with (in this case, the communication is done through the secure channel); or (3) drop the packet (e.g., for security reasons). There are two models for the implementation of an OpenFlow switch [McKeown et al. 2008]. The first, consists in a dedicated OpenFlow switch, which is basically a “dumb” device that only forwards packets according to the rules defined by a remote controller. In this case (See Figure 1.4), the flows can be broadly defined by the applications, so the network capabilities are only limited by how the Flow Table is implemented and which actions are available. The second, which may be preferable for legacy reasons, is a classic switch that supports OpenFlow but also keeps its ability to make its own forwarding decisions. In such hybrid scenario, it is more complicated to provide a clear isolation between OpenFlow and “classical” traffic. To be able to do so, there are basically two alternatives: (1) to implement one extra action to the OpenFlow Table, which forwards packets to the switches normal processing pipeline, or (2) to define different VLANs for each type of traffic. Whichever the case, the behaviors of the switch’s OpenFlow-enabled portion may

Figure 1.4. OpenFlow switch proposed by [McKeown et al. 2008].

be either reactive or proactive. In the reactive mode, whenever a packet arrives at the switch, it tries to find an entry in its Flow Table matching that packet. If such an entry is found, the corresponding action is executed; otherwise, the flow is redirected to the controller, which will insert a new entry into the switch’s Flow Table for handling the flow and only then the packet is forwarded according to this new rule. In the proactive mode, on the other hand, the switch’s Flow Table is pre-configured and, if an arriving flow does not math any of the existing rules, the corresponding packets are simply discarded [Hu et al. 2014a]. Operating in the proactive mode may lead to the need of installing a large number of rules beforehand on the switches, one advantage over the reactive mode is that in this case the flow is not delayed by the controller’s flow configuration process. Another relevant aspect is that, if the switch is unable to communication with the controller in the reactive mode, then the switch’s operation will remain limited to the existing rules, which may not be enough for dealing with all flows. In comparison, if the network is designed to work in the proactive mode from the beginning, it is more likely that all flows will be handled by the rules already installed on the switches. As a last remark, it is interesting to notice that implementing the controller as a centralized entity can provide a global and unique view of the network to all applications, potentially simplifying the management of rules and policies inside the network. However, as any physically centralized server, it also becomes a single point of failure, potentially impairing the network’s availability and scalability. This issue can be solved by implementing a a physically distributed controller, so if one controller is compromised, only the switches under its responsibility are affected. In this case, however, it would be necessary to implement synchronization protocols for allowing a unique view of the whole network and avoid inconsistencies. Therefore, to take full advantage of the benefits from a distributed architecture, such protocols must be efficient enough not to impact the overall network’s performance.

1.3.5. SDN Controllers An SDN controller, also called a network operating system, is a software platform where all the network control applications are deployed. SDN controllers commonly contain a set of modules that provide different network services for the deployed applications, including routing, multicasting, security, access control, bandwidth management, traffic engineering, quality of service, processor and storage optimization, energy usage, and all forms of policy management, tailored to meet business objectives. The network services provided by the SDN controller consist of network applications running upon the controller platform, and can be classified as follows: • Basic Network Service: Basic network applications that implement protocol, topology and device essential functions. Examples of basic network services are topology management, ARP handling, host tracking, status management and device monitoring. Basic network services are commonly used by other network services deployed in the controller platform to implement more complex control functionalities. • Management Services: Management network applications that make use of basic functions to implement business-centric management functionalities. Examples of management services are authentication and authorization services, virtual tenant network coordination, network bandwidth slicing and network policy management. • Core Services: Core network applications oriented to manage and orchestrate the operation of the control platform, including managing communication between other network services and shared data resources. Examples of core services are messaging, control database managing and service registering. • Custom Application Services: Custom network applications consist of any application developed by the platform users. The applications commonly use other network services deployed in the same SDN control platform to implement different network solutions. Examples of custom application services oriented toward security are DDoS prevention, load balancing and firewalling. Custom application services can also target areas such as QoS implementation, enforcement of policies and integration with cloud computing orchestration systems. Open source controllers have been an important vector of innovation in the SDN field. The dynamics of the open source community led the development of lots of SDN projets, including software-based switches and SDN controllers [Casado 2015]. To evaluate and compare different open-source controller solutions and their suitability to each deployment scenario, one can employ the following metrics: • Programming Language: The programming language used to build the controller platform. The controller language will also dictate the programming language used to develop the network services, and can directly influence other metrics such as performance and learning curve. Moreover, some operating systems may not provide full support for all programming languages. • Performance: The performance of the controller can be determinant when choosing the correct platform for production purposes. The performance of an SDN con-

troller can be influenced by many factors, including the programming language, design patterns adopted and hardware compatibility. • Learning Curve: The learning curve of the control platform is a fundamental metric to consider when starting a project. It measures the experience necessary to learn the SDN controller platform and build the necessary skills. The learning curve directly influences the time to develop a project and also the availability of skilled developers. • Features: The set of network functions provided by the SDN controller. In addiction to basic network services, control platforms can also provide specialized services related to controlling and managing network infrastructures. Two important groups of features are the set of protocols supported in the southbound API of the controller (e.g., OpenFlow, OVSDB, NETCONF), which will determine the supported devices in the underlying network infrastructure, and the support for integration with cloud computing systems. • Community Support: The support provided by the open source community is essential to measure how easy it would be to solve development and operating questions, as well as the frequency in which new features are released. Some open source SDN projects are also supported or maintained by private companies, which is likely to accelerate releases and lead to better support for specific business demands. To give a concrete example on the usefulness of these metrics, we can apply them to some of the most popular open source SDN controller projects, namely: NOX [Gude et al. 2008, NOXRepo.org 2015], POX [NOXRepo.org 2015], Ryu [Ryu 2015], Floodlight [Floodlight 2015] and OpenDaylight [Medved et al. 2014, Linux Foundation 2015]. NOX Controller: The NOX controller is part of the first generation of OpenFlow controllers, being developed by Nicira Networks side-by-side with the OpenFlow protocol. As the oldest OpenFlow controller, it is considered very stable by the industry and the open source community, and is largely deployed in production and educational environments. The NOX controller has two versions. The first, NOX-Classic, was implemented in C++ and Phyton, and supports the development of network control application using both languages. This cross-language design was later proved to be less efficient than designs based on a single language, since it ended up leading to some inconsistency in terms of features and interfaces. Possibly due to these issues, NOX-Classic is no longer supported, being superseded by the second version, called simply NOX or “new NOX”. This second version of NOX was implemented using the C++ programming language, supporting network application services developed with the same language using an eventoriented programming model. The code NOX was also reorganized to provide better performance and programmability compared with NOX-Classic, introducing support for both 1.0 and 1.3 versions of the OpenFlow protocol. A modern NOX SDN controller is recommended when: users know the C++ programming language; users are willing to use low-level facilities and semantics of the OpenFlow protocol; users need production level performance. POX Controller: POX is a Python implementation of the NOX controller, being created to be a platform for rapid development and prototyping of network control soft-

ware. Taking advantage of Python’s flexibility, POX has been used as basis for many SDN projects, being applied for prototyping and debugging SDN applications, implementing network virtualization and designing new control and programming models. The POX controller has also official support from the NOX community. POX support the version 1.0 only of the OpenFlow protocol and provides better performance when compared with Python applications deployed on NOX-Classic. However, since Python is an interpreted instead of compiled language, POX does not provide production level performance as NOX controller do. Therefore, a POX SDN controller is recommended when: users know the Python programming language; users are not much concerned with the controller’s performance; users need a rapid SDN platform for prototyping and experimentation, e.g., for research, experimentation, or demonstrations purposes; users are looking for an easy way to learn about SDN control platforms (e.g., for educational purposes). Ryu Framework: Ryu is a Python component-based SDN framework that provides a large set of network services through a well-defined API, making it easy for developers to create new network management and control applications for multiple network devices. Differently from NOX and POX SDN controllers, which support only OpenFlow protocols in their southbound API, Ryu supports various protocols for managing network devices, such as OpenFlow (versions 1.0 and 1.2 – 1.4), Netconf and OF-config. Another important feature of the Ryu framework is the integration with the OpenStack cloud orchestration system [OpenStack 2015], enabling large deployments on cloud data centers. Even though Ryu was implemented using Python, its the learning curve is moderated, since it provides a large set of service components and interfaces that need to be understood before it can be integrated into new applications. As a result, the Ryu SDN framework is recommended when: users know the Python programming language; users are not much concerned with the controller’s performance; the control applications require versions 1.3 or 1.4 of the OpenFlow protocol or some of the other supported protocols; users intend to deploy the SDN controller on a cloud data center that makes use of OpenStack’s orchestration system. Floodlight Controller: The Floodlight Open SDN Controller is a Java-based OpenFlow Controller supported by an open source community of developers that includes a number of engineers from Big Switch Networks. Floodlight is the core of Big Switch Networks commercial SDN products and is actively tested and improved by the industry and the developers community. Floodlight was created as a fork from the Beacon Java OpenFlow controller [Erickson 2013], the first Java-based controller to implement full multithread and runtime modularity features. Even though it has a quite extensive documentation and counts with official support from both the industry and open source community, Floodlight has a steep learning curve due to the large set of features implemented. Among those features, we can cite the ability to integrate with OpenStack orchestration system and the use of RESTful interfaces [Richardson and Ruby 2008] in the northbound API, enabling easy integration with external business applications. Floodlight controller is recommended when: users know the Java programming language; users need production level performance and would like to have industry support; applications should interact with the SDN controller through a RESTful API; users intend to deploy the SDN controller on a cloud data center that makes use of OpenStack’s orchestration system.

OpenDaylight Controller: OpenDaylight is a Java-based SDN controller built to provide a comprehensive network programmability platform for SDN. It was created as a Linux Foundation collaborative project in 2013 and intends to build a comprehensive framework for innovation in SDN environment. OpenDaylight project is supported by a consortium of network companies such as Cisco, Ericsson, IBM, Brocade and VMware, besides the open source community and industry that collaborate in the project. OpenDayligh is also based on the Beacon OpenFlow controller and provides production level performance with support for different southbound protocols, such as OpenFlow 1.0 and 1.3, OVSDB and NETCONF. It also provides integration with OpenStack’s cloud orchestration system. The OpenDaylight controller proposes an architectural framework by clearly defining the southbound and northbound APIs and how they interact with external business applications and internal network services. A a drawback, OpenDaylight has a steep learning curve due to its architectural complexity and the large set of services embedded in the controller. It is, nevertheless, recommended when: users know the Java programming language; users need production level performance and would like to have industry support; users intend to deploy the SDN controller on a cloud data center that makes use of OpenStack’s orchestration system; target applications require modularity through an architectural design; applications need to integrate with third party business applications, as well as with heterogeneous underlying network infrastructures. Table 1.1 presents a summary of the main characteristics of the described open source SDN controllers, based on the metrics hereby discussed. Table 1.1. Summary of the main characteristics of open source SDN controllers NOX

POX

Ryu

Floodlight

ODL

Language

C++

Python

Python

Java

Java

Performance

High

Low

Low

High

High

Distributed

No

No

Yes

Yes

Yes

OpenFlow

1.0

1.0

1.0, 1.2–1.4

1.0, 1.3

1.0, 1.3

Multi-tenant clouds

No

No

Yes

Yes

Yes

Moderate

Easy

Moderate

Steep

Steep

Learning curves

1.3.6. Network Virtualization using SDNs Even though network virtualization and SDN are independent concepts, the relationship between these two technologies has become much closer in recent years. Network virtualization creates the abstraction of a network that is decoupled from the underlying physical equipment, allowing multiple virtual networks to run over a shared infrastructure with a topology that differs from the actual underlying physical network. Even though network virtualization has gained prominence as a use case for SDN, the concept has in fact evolved in parallel with programmable networking. In especial, both technologies are tightly coupled by the programmable networks paradigm, which presumes mechanisms for sharing the infrastructure (across multiple tenants in a data center, administrative groups in a campus, or experiments in an experimental facility) and supporting logical network topologies that differ from the physical network. In what follows, we provide an overview of the state of the art on network virtualization technologies

before and after the advent of SDN. The creation of virtual networks in the form of VLANs and virtual private networks has been supported by multiple network equipment vendors for many years. These virtual networks could only be created by network administrators and were limited to run the existing protocols, delaying the deployment of new network technologies. As an alternate, researchers started building overlay networks by means of tunneling, forming their own topology on top of a legacy network to be able to run their own control-plane protocols. In addition to the significant success of peer-to-peer applications, built upon overlay networks, the networking community reignited research on overlay networks as a way of improving the network infrastructure. Consequently, virtualized experimental infrastructures such as PlanetLab [Chun et al. 2003] were built to allow multiple researchers to run their own overlay networks over a shared and distributed collection of hosts. The success of PlanetLab and other shared experimental network platforms motivated investigations on the creation of virtual topologies that could run custom protocols inside the underlying network [Bavier et al. 2006], thus enabling realistic experiments to run side by side with production traffic. As an evolution of these experimental infrastructures, the GENI project [Berman et al. 2014] took the idea of a virtualized and programmable network infrastructure to a much larger scale, building a national experimental infrastructure for research in networking and distributed systems. These technologies ended up by leading some to argue that network virtualization should be the basis of a future Internet, allowing multiple network architectures to coexist and evolve over time to meet needs in continuous evolution [Feamster et al. 2007, Anderson et al. 2005, Turner and Taylor 2005]. Researches on network virtualization evolved independently of the SDN concept. Indeed, the abstraction of the physical network in a logical network does not require any SDN technology, neither does the separation of a logically centralized control plane from the underlying data plane imply some kind of network virtualization. However, a symbiosis between both technologies has emerged, which has begun to catalyze several new research areas, since SDN can be seen as an enabling technology for network virtualization. Cloud computing, for example, introduced the need for allowing multiple customers (or tenants) to share a same network infrastructure, leading to the use of overlay networks implemented through software switches (e.g., Open vSwitch [Open vSwitch 2015, Pfaff et al. 2009]) that would encapsulate traffic destined for VMs running on other servers. It became natural, thus, to consider using logically centralized SDN controllers to configure these virtual switches with the rules required to control how packets are encapsulated, as well as to update these rules when VMs move to new physical locations. Network virtualization, on its turn, can be used for evaluating and testing SDN control applications. Mininet [Handigol et al. 2012a, Lantz et al. 2010], for example, uses process-based network virtualization to emulate a network with hundreds of hosts, virtual switches and SDN controllers on a single machine. This environment enables researchers and network operator to develop control logic applications and easily evaluate, test and debug them on a full-scale emulation of the production data plane, accelerating the deployment on the real production networks. Another contribution from network virtualization to the development of SDN technologies is the ability to slice the underlying network, allowing it to run simultaneous and isolated SDN experiments. This con-

cept of network slicing, originally introduced by the PlanetLab project [Chun et al. 2003], consists in separate the traffic-flow space into different slices, so each slice has a share of network resources and can be managed by a different SDN controller. FlowVisor [Sherwood et al. 2010], for example, provides a network slicing system that enables building testbeds on top of the same physical equipment that carries the production traffic. 1.3.7. SDN Applications in Network Virtualization SDN facilitates network virtualization and may, thus, makes it easier to implement features such as dynamic network reconfiguration (e.g., in multitenant environments). However, it is important to recognize that the basic capabilities of SDN technologies do not directly provide these benefits. Some SDN features and their main contributions to improve network virtualization are: • Control plane and data plane separation: The separation between control and data planes in SDN architectures, as well as the standardization of interfaces for the communication between those layers, allowed to conceptually unify different vendor network devices under the same control mechanisms. For network virtualization purposes, the abstraction provided by the control plane and data plane separation facilitates deploying, configuring, and updating devices across virtualized network infrastructures. The control plane separation also introduces the idea of network operating systems, which consists of a scalable and programmable platform for managing and orchestrating virtualized networks. • Network programmability: Programmability of network devices is one of the main contributions from SDN to network virtualization. Before the advent of SDN, network virtualization was limited to the static implementation of overlay technologies (such as VLAN), a task delegated to network administrators and logically distributed among the physical infrastructure. The programming capabilities introduced by SDN provide the dynamics necessary to rapidly scale, maintain and configure new virtual networks. Moreover, network programmability also allows the creation of custom network applications oriented to innovative network virtualization solutions. • Logically centralized control: The abstraction of data plane devices provided by SDN architecture gives the network operating system, also known as SDN orchestration system, a unified view of the network. Therefore, it allows custom control applications to access the entire network topology from a logically centralized control platform, enabling the centralization of configurations and policy management. This way, the deployment and management of network virtualization technologies becomes easier than in early distributed approaches. • Automated management: the SDN architecture enhances network virtualization platforms by providing support for automation of administrative tasks. The centralized control and the programming capabilities provided by SDN allow the development of customized network applications for virtual network creation and management. Autoscaling, traffic control and QoS are examples of automation tools that can be applied to virtual network environments.

Among the variety of scenarios where SDN can improve network virtualization implementations, we can mention campus network testbeds [Berman et al. 2014], enterprise networks [Casado et al. 2007], multitenant data centers [Koponen et al. 2014] and cloud networking [Jain and Paul 2013b]. Despite this successful application of SDN technologies in such network virtualization use cases scenarios, however, much work is needed both to improve the existing network infrastructure and to explore SDN’s potential for solving problems in network virtualization. Examples include SDN applications to scenarios such as home networks, enterprise networks, Internet exchange points, cellular networks, Wi-Fi radio access networks, and joint management of end-host applications. 1.3.8. Security, Scalability and Availability aspects Since the SDN concept became a prominent research topic in the area of computer networks, many studies have discussed fundamental aspects such as its scalability, availability and, in especial, security. Even though scalability issues apply both to the controller and to forwarding nodes, the latter are not specifically affected by the SDN technology and, thus, we only focus on the former. Specifically, there are three main challenges for attaining controller scalability [Yeganeh et al. 2013, Sezer et al. 2013], both of which originate in the fact the network’s intelligence is moved from the distributed forwarding nodes to the control plane: (1) the latency incurred by the communications between the forwarding nodes and the controller(s); (2) the size of the controller’s flow database, and (3) the communication between controllers in a physically distributed control plane architecture. As previously mentioned in Section 1.3.4, the first challenge may be tackled with proactive approach, i.e., by installing most flow rules on the SDN-enable switches so they do not need to contact the controllers too frequently. Even though this might sacrifice flexibility, this may be inevitable especially for large flows. Another strategy for tackling latency issues in the control plane, as well as the size of the flow databases, consists in using multiple controllers. As a result, they can share the communication burden, reducing delays potentially caused by queuing requests coming from switches, and also the storage of flow information, as each controller is responsible by a subset of forwarding elements. However, this also aggravates the third challenge, due to the need of further interactions between controllers to ensure a unified view of the network [Sezer et al. 2013]. Nonetheless, since a distributed controller architecture also improves availability by improving the system’s resiliency to failures, there have been many proposals focused on improving the scalability of this approach. One example is HyperFlow [Tootoonchian and Ganjali 2010], an NOX-oriented application that can installed on all network controllers to create a powerful event propagation system based on a publish/subscribe messaging paradigm: basically, each controller publishes events related to network changes to other controllers, which in turn replay those events to proactively propagate the information throughout the whole control plane. Another strategy, adopted in Onix [Koponen et al. 2010] and ONOS [Berde et al. 2014], consists in empowering control applications with general APIs that facilitate access to network state information. All things considered, ensuring scalability and state consistency among all controllers, as well as a reasonable level of flexibility, ends up being an important design trade-off in SDNs [Yeganeh et al. 2013].

Regarding security, the SDN technology brings both new opportunities and challenges (for a survey on both views, see [Scott-Hayward et al. 2013]). On the positive side, SDN can enhance network security when the control plane is seen as a tool for packet monitoring and analysis that is able to propagate security policies (e.g., access control [Nayak et al. 2009]) along the entire network in response to attacks [Scott-Hayward et al. 2013]. In addition, with the higher control over how the packets are routed provided SDN, one can install security appliances such as firewalls and IDS in any part of the network, not only on its edges [Gember et al. 2012]: as long as the controllers steer the corresponding traffic to those nodes, the packets can be analyzed and treated accordingly. This flexibility is, for example, at the core of the Software Defined Perimeter (SDP) concept [Bilger et al. 2013], by means of which all devices trying to access a given network infrastructure must be authenticated and authorized before the flow rules that allows its entrance are installed in the network’s forwarding elements. It is also crucial to thwart denial-of-service (DoS) attacks, since then the task of discarding malicious packets is not concentrated on one or a few security devices near the attack’s target, but distributed along the network [YuHunag et al. 2010]. Another interesting application of SDNs for thwarting DoS, as well as other threats targeting a same static IP (e.g. port scanning or worm propagation), is to create the illusion of a “moving target”, i.e., by having the SDN translate the host’s persistent address to different IPs over time [Jafarian et al. 2012]. Whereas the security enhancements resulting from the SDN approach is commonly recognized, it also brings security risks that need to be addressed. In [Kreutz et al. 2013], seven main threat vectors are identified, the first three being SDNspecific: (1) attacks on control plane communications, especially when they are made through insecure channels; (2) attacks on and vulnerabilities in controllers, (3) lack of mechanisms to ensure trust between the controller and management applications; (4) forged traffic flows; (5) attacks exploring vulnerabilities in switches; (6) attacks on and vulnerabilities in administrative stations that access the SDN controllers, and (7) the lack of trusted resources for forensics and remediation. Such threats usually require holistic solutions providing authentication and authorization mechanisms for handling the different entities configuring the network and detecting anomalies. This need is addressed, for example, by the FortNOX security kernel [Porras et al. 2012], as well as by its successor, Security-Enhanced Floodlight [Porras et al. 2015], which enable automated security services while enforcing consistency of flow policies and role-based authorization; it is also the focus of FRESCO [Shin et al. 2013], an application framework that facilitates the development and deployment of security applications in OpenFlow Networks. There are also solutions focused on specific issues, such as identifying conflicts and inconsistencies between the policies defined by multiple applications [Al-Shaer and Al-Haj 2010, Canini et al. 2012, Khurshid et al. 2013] or facilitating auditing and debugging [Handigol et al. 2012b, Khurshid et al. 2013]. Nevertheless, there is much place for innovation in the field, as the number of articles proposing solutions for SDN security issues are still considerably less prevalent in the literature than those focusing on using the SDN paradigm to provide security services [Scott-Hayward et al. 2013].

1.4. Cloud Network Virtualization using SDN This section discusses the connection between cloud computing and SDNs. We starting by analyzing the synergy between the concepts and technologies involving both paradigms. We then describe an integration architecture that takes advantage of this synergy for deploying new services (namely, Network- and Security-as-a-Service), which may be integrated with Network Function Virtualization (NFV) technologies. 1.4.1. Synergy between SDNs and clouds As previously discussed, networking plays a key role in clouds, both as a shared resource and as part of the infrastructure needed for sharing other computational resources. As any infrastructure, the network in a cloud environment should attend some critical requirements of modern networks, in especial [Cheng et al. 2014]: adaptability to new application needs, business policies, and traffic behavior, which new feature being integrated with minimal disruption of the network operation; automation of network changes propagation, reducing (error-prone) manual interventions; provision of high level abstractions for easier network management, so administrators do not need to configure each individual network element; capability of accommodating the nodes’ mobility and security features as a core service, rather than as add-on solutions; and on-demanding scaling. To fulfill these requirements, cutting-edge network equipments with advanced capabilities are likely to be needed. Nevertheless, the cloud cannot take full advantage of such resources without orchestration engines with deep knowledge of the available network capabilities. Unfortunately, such deep knowledge may require features that are very specific to a given proprietary network hardware and software, leading to vendor lockin issues and, consequently, limiting the creation of new network features or services. In addition, it is often the case that the services must be orchestrated over multi-carrier and multi-technology communication infrastructure (e.g., composed by packet switching, circuit switching and optical transport networks) [Autenrieth et al. 2013], or even among different cloud providers [Mechtri et al. 2013]. SDNs tackle this issue by placing an abstraction layer with standard APIs over the (proprietary) hardware, facilitating the access to the corresponding features and, thus, to innovations. In an environment as dynamic as the cloud, the flexibility and programmability provided by SDNs is essential to allow the network to evolve together with the services using it. 1.4.2. Integration Architectures SDNs and clouds display similar designs, with a 3-layer architecture composed by a Infrastructure Layer with computational resources controlled by a Control Layer, which in turn is controlled via APIs by applications in an Application Layer (see Figure 1.5). One simple form of integrating SDNs and clouds is to run their stacks in parallel, with both technologies being integrated by the applications themselves. Even though applications can benefit from both technologies with this strategy, it also brings a significant overhead to application developers. After all, applications would need to be SDN- and cloud-aware, assimilating and accessing APIs for both technologies in an effective manner, which is prone to complicate their design and implementation. To avoid these issues, an alternative approach would be to use a special cloud

Figure 1.5. SDN/Cloud Integration by applications. Adapted from [Autenrieth et al. 2013]

control/orchestration subsystem capable of controlling SDN devices directly, using SDN data plane control protocols (e.g. OpenFlow) instead of a separate SDN controller. This strategy, illustrated in Figure 1.6, brings some SDN benefits to the cloud infrastructure while hiding its complexity to applications: they would need to use only cloud APIs, remaining unaware of the SDN-enabled network infrastructure. Nevertheless, there are also some drawbacks: as it demands a specialized cloud orchestrator, development of new network control features is tied to the development of the orchestrator itself, or to the APIs it provides, possibly limiting innovation It may also restrict the deployment of proprietary SDN solutions.

Figure 1.6. SDN Functions incorporated in the Cloud Control/Orchestration subsystem

Finally, a third and probably preferable is to consider the cloud control/orchestra-

tion system as an SDN application to the SDN controller. In this scenario, depicted in Figure 1.7, the Cloud Control/Orchestration subsystem is augmented with modules that translate Cloud Operations to SDN operations, using existing SDN controllers APIs. This approach brings the benefits of the second approach while allowing greater flexibility: it is possible to evolve both the Cloud and SDN infrastructures separately, with minimal or no changes to their integration interface. It would also allow the use of existing SDN solutions without alterations, including proprietary SDN solutions or hardware-based controllers.

Figure 1.7. SDN Integration inside the Cloud

1.4.3. Network as a Service (NaaS) supported by SDN In a cloud computing environment, users (tenants) are provisioned with virtual computational resources, including virtual networks, by the cloud orchestrator. Usually, these virtual networks are also used as an infrastructure for the other computational resources. There are, however, some limitations with this approach [Costa et al. 2012]: little or no control over the network; indirect access or management to the network infrastructure (i.e., switches or routers); limited visibility over the network resources; inefficient overlay networks; no multicast support. To face these shortcomings, it is possible to share networking resources as services, similarly to that is done with computational resources. This model is called Network-as-a-Service (NaaS) [Costa et al. 2012]. In NaaS, networking resources are used and controlled through standard interfaces/APIs. In principle, a network service may represent any type of networking component at different levels, including a domain consisting of a set of networks, a single physical or virtual network, or an individual network node. Multiple network services can then be combined into a composite inter-network service by means of a service composition mechanism [Duan 2014]. NaaS and SDN can, thus, be combined: while the SDN technology provides dynamic and scalable network service management and facilitates the implementation of NaaS, the latter allows a control mechanism over a (possibly heterogeneous) underlying network infrastructure. This approach also enables the creation

of richer NaaS services: if a given feature is not fully implemented on the underlying network, it can be implemented as a SDN application inside the cloud control/orchestration layer and offered to users via cloud APIs. One possible solution for offering NaaS services through the combination of SDN and Cloud Computing is the OpenNaaS project1 . It offers an open source framework for helping creating different types of network services in an OpenStack cloud computing environment. The framework provides a virtual representation of physical resources (e.g., networks, routers, switches, optical devices or computing servers), which can be mapped inside OpenNaaS to SDN resources for the actual implementation of these networking elements [Aznar et al. 2013]. 1.4.4. Security as a Service (SecaaS) using SDN Security as a Service (SecaaS) refers to the provision of security applications and services via the cloud, either to cloud-based infrastructure and software, or to customers’ on-premise systems [CSA 2011]. To consume cloud security resources, end-users must be aware of the nature and limitations of this new computing paradigm, whereas cloud providers must take special care when offering security services. Aiming to provide guidance for interested users and providers, the Cloud Security Alliance (CSA) has published security guides that, based on academic results, industry needs and end-user surveys, discuss the main concerns that SecaaS applications must deal with [CSA 2011]: • Identity and Access Management (IAM): refers to controls for identity verification and access management. • Data Loss Prevention: related to monitoring, protecting and verifying the security of data at rest, in motion and in use. • Web Security: real-time protection offered either on-premise, through software/appliance installation, or via the cloud, by proxying or redirecting web traffic to the cloud provider. • Email Security: control over inbound and outbound email, protecting the organization from phishing or malicious attachments, as well as enforcing corporate polices (e.g., acceptable use and spam prevention), and providing business continuity options. • Security assessments: refers to third-party audits of cloud services or assessments of on-premises systems. • Intrusion Management: using pattern recognition to detect and react to statistically unusual events. This may include reconfiguring system components in real time to stop or prevent an intrusion. • Security Information and Event Management (SIEM): analysis of logs and event information analysis aiming to provide real-time reporting and alerting on incidents that may require intervention. The logs are likely to be kept in a manner that prevents tampering, thus enabling their use as evidence in any investigations. • Encryption: providing data confidentiality by means of encryption algorithms. 1 Projet

home page: http://opennaas.org

• Business Continuity and Disaster Recovery: refers to measures designed and implemented to ensure operational resiliency in the event of service interruptions. • Network Security: security services that allocate, access, distribute, monitor, and protect the underlying resource services. Security service solutions for the Internet can be commonly found nowadays, in what constitutes a segmentation of the Software as a Service (SaaS) market. This can be verified, for example, in sites that provide credit card payment services, that offer online security scanning (e.g. anti-malware/anti-spam) to a user’s personal computer, or even on Internet access providers that offer firewall services to its users. These solutions are closely related to the above-mentioned Web Security, Email Security and Intrusion Management categories, and have as main vendors Cisco, McAfee, Panda Software, Symantec, Trend Micro and VeriSign [Rouse 2010]. However, this kind of services has been deemed insufficient to attract the trust of many security-aware end-users, especially those that have knowledge of cloud inner workings or are in search of IaaS services. Aiming to attract this audience and, especially, to improve cloud internal security requirements, organizations have been investing in SDN solutions capable of improving security on (cloud) virtual networks. To cite a recent example of cloud-oriented SDN firewall, we can mention Flowguard [Hu et al. 2014b] Besides basic firewall features, Flowguard also provides a comprehensive framework for facilitating detection and resolution of firewall policy violations in dynamic OpenFlowbased networks: security policy violations can be detected in real time, when the network status is updated, allowing a (tenant or cloud) administrators to decide whether to adopt distinct security strategies for each network state [Hu et al. 2014b]. Another recent security solution is Ananta [Patel et al. 2013], an SDN-based load balancer for large scale cloud computing environments. In a nutshell, the solution consists of a layer-4 load balancer that, by placing one agent in every host, allows packet modification tasks to be distributed along the network, thus improving scalability Finally, for the purpose of detecting or preventing intrusions, one recent solution is the one introduced in [Xiong 2014], which can be seen as an SDN-based defensive system for detection, analysis, and mitigation of anomalies. Specifically, the proposed solution takes advantage of the flexibility, compatibility and programmability of SDN to propose a framework with a Customized Detection Engine, Network Topology Finder, Source Tracer and further user-developed security appliances, including protection against DDoS attacks. 1.4.5. Integration with Network Function Virtualization (NFV) Technologies The specifications of NFV are being developed by the European Telecommunications Standards Institute (ETSI). Their main goal is to transform the way that operators design their networks, by using standard IT virtualization technologies to consolidate network equipments onto industry-standard high volume servers, switches and storage devices, which could be located in data centers, network nodes or in the end-user premises [ETSI 2012]. The idea behind NFV is to allow the implementation of network functions (e.g., routing, firewalling, and load-balancing), normally deployed in proprietary boxes, in the form of software that can run on standard server hardware. As any software, the

network function could then be instantiated in (or moved to) any location in the network, as illustrated in Figure 1.8. The NFV Infrastructure (NFVI) could then provide computing capabilities comparable to an those of an IaaS cloud computing model, as well as dynamic network connectivity services similar to those provided by the NaaS concept discussed in Section 1.4.3.

Figure 1.8. Concept of NFV [Jammal et al. 2014]

It is interesting to notice that, although the NFV and SDN concepts are considered highly complementary, they do not dependent on each other [ETSI 2012]. Instead, both approaches can be combined to promote innovation in the context of network: the capability of SDN to abstract and programmatically control network resources are features that fit well to the need of NFV to create and manage a dynamic and on-demand network environment with performance. This synergy between these concepts has lead ONF and ETSI to work together with the common goal of evolving both approaches and provide a structured environment for their development. Table 1.2 provides a comparison between both SDN and NFV concepts. Table 1.2. Comparison between SDN and NFV (Adapted from [Jammal et al. 2014]).

Motivation

Network tion

Loca-

SDN

NFV

Decoupling of control and data planes; Providing centralized controller and network programmability

Abstraction of network functions from dedicated hardware appliances to Commercial offthe-shelf (COTS) servers

Data centers

Service provider networks

Network Devices

Servers and switches

Servers and switches

Protocols

OpenFlow

Not Applicable

Applications

Cloud orchestration and networking

Firewalls, gateways, content delivery networks

Standardization Committee

Open Networking Forum (ONF)

ETSI NFV group

1.5. Case Study with OpenDaylight and OpenStack This section presents a a case study involving the creation and management of virtual networks using the OpenDaylight SDN controller and the OpenStack cloud orchestration system, both open source projects widely adopted by academia and industry. After

presenting the main virtualization components provided by each system, as well as their functions and interfaces, we propose a practical analysis toward their integration for building a consistent and fully functional cloud virtual networking environment. 1.5.1. OpenStack Cloud Operating System OpenStack [OpenStack 2015] is a cloud computing project created in 2010, derived from NASA RackSpace initiative. The project is part of an effort to create an open-source, standards-based, highly-scalable, and advanced cloud computing platform that can be deployed on commodity server hardware [OpenStack 2015, Wen et al. 2012]. All OpenStack code is, thus, open for anyone wishing to build and provide cloud computing services, as well as to create applications on top of the platform. In its current version (named Juno), the OpenStack platform is composed of 11 core components, listed and briefly described in Table 1.3. Table 1.3. Components of OpenStack Juno and their functions. Service Cloud Management

Code-Name Description Nova

Controls the IaaS infrastructure, allocating or releasing computational resources, such as network, authentication, among others. Network Service Neutron Provisions network services for Nova-managed components. Specifically, allows users to create and attach virtual Network Interface Cards (vNICs) to these components. Object Storage Swift Allows the storage and retrieval of files not mounted on directories. It is a longterm storage system for static or permanent data. Block Storage Cinder Provides block storage for VMs. Identity Service Keystone Provides authentication and authorization to all OpenStack services. Image Service Glance Provides a catalog and repository for virtual disk images managed by Nova. Control Panel / Horizon Provides a web interface for all OpenStack components, allowing their manageDashboard ment straight from a common web browser. Accounting Ceilometer Monitors OpenStack components, providing metrics that can be used for generating usage statistics, billing or performance monitoring, among others. Orchestration Heat Orchestrates the cloud, allowing the management of the other components from a single interface Database Trove Provides relational and non-relational databases to the cloud. Elastic Map Reduce Sahara Creates and manages Hadoop clusters from a set of user-defined parameters, such as Hadoop version, cluster topology, hardware details, among others.

1.5.2. OpenStack Neutron Neutron is OpenStack’ component responsible for providing network services for tenant infrastructures, virtualizing and managing network resources operated by other OpenStack modules (e.g., Nova’s computing services). Neutron implements the virtualization layer over the network infrastructure in the OpenStack system, providing a pluggable, scalable and API-driven system for managing networks. As such, it must ensure that the network does not become the bottleneck in a cloud deployment and provision a selfservice environment for users. The main Neutron service capabilities are described as follows. • Provides flexible networking models, suiting the needs of different applications or user groups. Standard models include flat networks or VLANs for separation of networks and traffic flows.

• Manages IP addresses, allowing dedicated static IPs or DHCP service. Floating IPs allow traffic to be dynamically rerouted to any compute resource, so users can have their traffic redirected during maintenance procedures or in the case of failures. • Creates tenant networks, controls traffic and connects servers and devices to one or more networks. • Provides a pluggable back-end architecture that allow users to take advantage of commodity gear or advanced networking services from third party vendors. • Integrates with SDN technology such as OpenFlow, facilitating configuration and management of large-scale multitenant networks. • Provides an extension framework that allows the deployment of additional network services, such as intrusion detection systems (IDS), load balancing, firewalls and virtual private networks (VPN). 1.5.2.1. Components The Neutron service comprises several components. To explain how these components are deployed in an OpenStack environment, it is useful to consider a typical deployment scenario with dedicated nodes for network, compute and control services, as shown in Figure 1.9. The roles of each Neutron component illustrated in this figure are:

Figure 1.9. Neutron components in an OpenStack deployment with a dedicated network node [OpenStack 2015].

• neutron-server: The Neutron server component provides the APIs for all net-

work services implemented by Neutron. In the deployment shown in Figure 1.9, this component is located inside the cloud controller node, the host responsible to provide the APIs for all the OpenStack services running inside the cloud through the API network. The controller node can provide API access for both the other OpenStack and the end users of the cloud, respectively through the management network and the internet. • neutron-*-plugin-agent:Neutron plug-in agents implement the network services provided by the Neutron API, such as layer 2 connectivity and firewall. The plug-ins are distributed among network and compute nodes and provide different levels of network services for the OpenStack cloud infrastructure. • neutron-l3-agent: The Neutron L3 agent is the component that implements Neutron API’s layer 3 connectivity services. It connects tenant VMs via layer 3 networks, including internal and external networks. The Neutron L3 agent is located on the network node, which is connected to the Internet via the External network. • neutron-dhcp-agent: The Neutron DHCP agent provides dynamic IP distribution for tenant networks. It also implements the floating IP service, which provides external IP addresses for tenant VM, enabling Internet connectivity. It is also located on the network node and connected to the Internet via the External network. Figure 1.9 also depicts a standard deployment architecture for physical data center networks, which are: • Management Network: Provides internal communication between OpenStack components. IP addresses on this network should be reachable only within the data center. • Data Network: Provides VM data communication within the cloud deployment. The IP addressing requirements of this network depend on the Networking plug-in being used. • External Network: Provides VMs with Internet access in some deployment scenarios. Anyone on the Internet can reach IP addresses on this network. • API Network: Exposes all OpenStack APIs, including the Networking API, to tenants. IP addresses on this network should be reachable by anyone on the Internet. The API network might be the same as the External network, as it is possible to create an external-network subnet that has allocated IP ranges that use less than the full range of IP addresses in an IP block. Besides these components and physical networks, Neutron makes use of virtualized network elements such as virtual switches and virtual network interfaces to provide connectivity to tenant VMs. The concept of bridges is particularly important here: bridges are instances of virtual switches implemented by a software such as Open vSwitch [Open vSwitch 2015, Pfaff et al. 2009] and used to deploy network virtualization for tenant VMs. There are three types of bridges created and managed by Neutron in an OpenStack deployment: • br-int: Integration bridges provide connectivity among VMs running in the same compute node, connecting them via their virtual network interfaces.

• br-tun: Tunneling bridges provide connectivity among different compute nodes by using a layer 2 segmentation mechanism, such as VLAN [IEEE 2014], or a tunneling protocol such as GRE [Farinacci et al. 2000] or VXLAN [Mahalingam et al. 2014a]. • br-ex: External bridges provide tenant VMs with connectivity to the external network (Internet) by using Neutron routing services. 1.5.2.2. Communication Flows The connectivity architecture provided by Neutron, with the relationships between VMs, physical nodes and VMs, is illustrated in Figure 1.10. In this figure, it is assumed the same deployment scenario presented in Figure 1.9, focusing on the network and compute nodes of the OpenStack infrastructure.

Figure 1.10. Implementation of virtual networks using neutron and virtual switches.

To facilitate the discussion VMs’ communication flows, it is useful to separate the explanation in three different scenarios, Intra-node communication, Inter-VM communication and Internet communication, which are described in what follows. • Intra-node communication: The scenario where VMs communicate with each other inside the same compute node (e.g., VM1-to-VM2). In this case, the traffic sent by VM1 is forwarded to VM2 using the integration bridge (br-int). The forward rule in br-int is configured inside the the compute node virtual switch’s flow table, using OpenFlow. • Inter-VM communication: The scenario where VMs located different compute nodes communicate with each other (e.g., VM1-to-VM3). In this case, VM1’s traffic is sent to br-int, which forwards the traffic to the tunneling bridge (br-tun), where it is encapsulated using a tunneling protocol (e.g., GRE or VXLAN) and sent through the data

network to the other compute node. When the traffic arrives at the destination compute node, it is decapsulated by br-tun and forwarded to the br-int bridge, from which it is finally send to VM3. • Internet communication: The scenario where a VM communicates with the outside world (e.g., VM1 to the Internet). Similarly to the previous scenario, in this case the traffic originated in VM1 is sent to br-int, forwarded to br-tun for encapsulation with a tunneling protocol, and sent via the data network to the network node. There, the traffic is decapsulated on the network node’s br-tun and forwarded to the br-int bridge, which makes use of the routing services provided by Neutron and forward the traffic to the internet. This communication model involves at least one compute node and one network node. Other two important components of the Neutron networking are the network node’s router and dhcp components (see Figure 1.10). They are implemented by means of network namespace, which is a kernel facility that allows groups of processes to have a network stack (interfaces, routing tables, iptables rules) distinct from that of the underlying host. More precisely, a Neutron router is a network namespace with a set of routing tables and iptables rules that handle the routing between subnets, while the DHCP server is an instance of the dnsmasq software running inside a network namespace, providing dynamic IP distribution or floating IPs for tenant networks. 1.5.3. Neutron Plugin: Modular Layer 2 (ML2) Neutron plugins are fundamental components in the OpenStack network architecture, as they are responsible for implementing the entire set of network services provided by the Neutron API. The plugins are segmented according to the different network services provided, with each service being implemented by a Neutron plugin. In Neutron, they can be classified into two main categories: core plugins, which implement core layer 2 connectivity services; and service plugins, which provide additional network services such as load balancing, firewall and VPN. Figure 1.11 shows Neutron’s control flow when handling network service requests. After receiving a request from the Neutron API, a neutron plugin executes the requested operation over the cloud network by using neutron plugin agents, hardware and software appliances and external network controllers. Some of Neutron’s network plugins are able to implement API requests using different back-ends, delegating the actual execution to drivers, such as the Modular Layer 2 (ML2), Firewall, Load Balancing and VPN plugins. In particular, the ML2 plugin plays a key role in our the experimental scenario hereby discussed, so it is worthy detailing it further. The ML2 plugin is a framework that allows OpenStack Networking to simultaneously use the variety of layer 2 networking technologies found in real-world data centers. It currently works with openvswitch, linuxbridge, and hyperv L2 agents, and is intended to replace and deprecate the monolithic plugins associated with those L2 agents (more precisely, starting with OpenStack’s Havana release, openvswitch and linuxbridge monolithic plugins are being replaced by equivalent ML2 drivers). The ML2 framework is also intended to simplify the task of adding support for new L2 networking technologies,

Figure 1.11. Execution workflow for Neutron network services

requiring less initial and ongoing effort than adding a new monolithic core plugin.

Figure 1.12. Overall architecture of Neutron ML2 plugin.

Figure 1.12 presents the overall architecture of the Neutron ML2 plugin. As the name implies, the ML2 framework has a modular structure, composed of two different set of drivers: one for the different network types (TypeDrivers) and another for the different mechanisms for accessing each network type, as multiple mechanisms can be used simultaneously to access different ports of the same virtual network (MechanismDriver). Mechanisms can access L2 agents via remote procedure calls (RPC) and/or use mechanism drivers to interact with external devices or controllers. • Type drivers: Each available network type is managed by an ML2 TypeDriver. TypeDrivers maintain any needed type-specific network state, and perform provider network validation and tenant network allocation. ML2 plugin currently includes drivers for local, flat, VLAN, GRE and VXLAN network types. • Mechanism drivers: The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied, given

Figure 1.13. Overview of OpenDaylight functions and benefits

the specific networking mechanisms that have been enabled. The MechanismDriver interface currently supports the creation, update, and deletion of network resources. For every action taken on a resource, the mechanism driver exposes two methods: a precommit method (called within the database transaction context) and a postcommit method (called after the database transaction is complete). The precommit method is used by mechanism drivers to validate the action being taken and make any required changes to the mechanism driver’s private database, while the postcommit method is responsible for appropriately pushing the change to the resource or to the entity responsible for applying that change. The ML2 plugin architecture facilitates the type drivers to support multiple networking technologies, and mechanism drivers to facilitate the access to the networking configuration in a transactional model. 1.5.4. OpenDaylight SDN Controller Created in April 2013 as a Linux Foundation collaborative project, OpenDaylight is an open source OpenFlow controller and also a scalable SDN framework for the development of several network services, including data plane protocols. As such, OpenDaylight can be the core component of any SDN architecture. Figure 1.13 shows an overview of the main functions and benefits provided by the OpenDaylight framework. The OpenDaylight architecture follows the traditional SDN design, implementing the control layer as well as the northbound and southbound interfaces. However, differently from the majority of controllers, the OpenDaylight architecture clearly separates its design and implementation aspects. Figure 1.14 presents an overview of the OpenDaylight architecture on the Helium release. The OpenDaylight SDN controller is composed by the following architectural layers: • Network Applications, Orchestration and Services: Business applications that make use of the network services provided by the controller platform to implement control, orchestration and management applications. • Controller Platform: Control layer that provides interfaces for all the network services implemented by the platform via a REST northbound API. The controller plat-

Figure 1.14. OpenDaylight architecture overview [Linux Foundation 2015]

form also implements a service abstraction layer (SAL), which provides a high-level view of the data plane protocols to facilitate the development of control plane applications. • Southbound Interfaces and Protocol Plugins: Southbound interfaces contain the plugins that implement the protocols used for programming the data plane. • Data Plane Elements: Physical and virtual network devices that compose the data plane and are programmed via the southbound protocol plugins. The variety of southbound protocols supported by the OpenDaylight controller allows the deployment of network devices from different vendors in the underlying network infrastructure. The service abstraction layer (SAL) is one of the main innovations of the OpenDaylight architecture To enable communication between plugins, this message exchange mechanism ignores the role of southbound and northbound plugins and builds upon the definition of Consumer and Provider plugins (see Figure 1.15): providers are plugins that expose features to applications and other plugins through its northbound API, whereas consumers are components that make use of the features provided by one or more Providers. This change implies that every plugin inside OpenDaylight can be seen as both a provider and a consumer, depending only on the messaging flow between the plugins involved. In OpenDaylight, SAL is responsible for managing the messaging between all the applications and underlying plugins. Figure 1.16 shows the life of a package inside the OpenDaylight architecture, depicting the following steps: 1. A packet arriving at Switch1 is sent to the appropriate protocol plugin;

Figure 1.15. Communication between producer and consumer plugins using SAL.

Figure 1.16. Life of a package in OpenDaylight.

Figure 1.17. Execution flow for Neutron API request.

2. The plugin parses the packet and generates an event for SAL; 3. SAL dispatches the packet to the service plugins listening for DataPacket; 4. Module handles the packet and sends is out via the IDataPacketService; 5. SAL dispatches the packet to the southbound plugins listening for DataPacket; 6. OpenFlow message sent to appropriate switch 1.5.5. Integration Architecture The OpenStack cloud orchestration system and OpenDaylight SDN controller are integrated via a Neutron ML2 mechanism driver called ODL (acronym for OpenDayLight). This driver executes the layer 2 connectivity services provided by the Neutron API, as well as a Neutron service application running inside the OpenDaylight controller. The controller, in its turn, makes use of other OpenDaylight service applications to perform the requested actions over the layer 2 network. Figure 1.17 illustrates the execution flow of a request from the Neutron L2 API to the OpenDaylight controller. After receiving the request from Neutron L2 API, the ML2 plugin selects the ODL mechanism driver based on Neutron static configuration files and calls the driver’s methods with the parameters necessary for fulfilling the request. The ODL driver then implements the ML2 service methods by sending commands to the OpenDaylight controller via a Neutron REST API. Neutron REST API extends the OpenDaylight REST API to incorporate Neutron service features. Figure 1.18 depicts the execution flow inside the SDN controller. Inside OpenDaylight, the request is forwarded to the the Neutron service application, which in turn executes the necessary actions over the data plane using the southbound protocol plugins, communicating with them via SAL. The southbound protocol plugins implement the communication protocols necessary for data plane programming, such as OpenFlow and OVSDB; for example, in the case of OpenStack, this programming refers mainly to the Open vSwitch bridges detailed in Section 1.5.2.

Figure 1.18. Execution flow for ODL requests inside the OpenDaylight controller.

Table 1.4. Hardware and software requirements for the OpenStack nodes.

1.5.6. Deploying Cloud Networking with OpenStack and OpenDaylight In what follows, we conduct a simple and didactic experiment showing how to build an architecture that bring the benefits of SDN to cloud computing systems. For this, we use OpenStack and OpenDaylight, giving a practical example of the OpenStack networking in an SDN architecture, analyzing the interactions between Neutron and the OpenDaylight controller, as well as their specific roles in this deployment. To reproduce the experiment hereby presented, the reader needs at least two physical or virtual servers for installing separate nodes for the OpenStack’s computing and networking services. For convenience, the demo session already provides two VMs for those who want to execute the experiment in their own computer during the presentation or at a later date. The software and hardware requirements for each server node are presented in Table 1.4. It is important to notice that, in our deployment scenario, the OpenStack controller services run in the same node as the network services (i.e., in the Network Node) This is not strictly necessary, however, as it could run in any server connected to the same subnet as the Network Node.

Figure 1.19. Network topology used in the experiment.

1.5.6.1. Bootstrapping the Compute and Network Nodes As explained before, this experiment will be performed based on two VMs configured as described in Table 1.4. Any virtualization system can be adopted to perform the experiment in a virtualized environment, which means running the compute and network nodes as VMs. The only important restriction is that both nodes should be connected in the same local network. For the purpose of this demo, we assume the network configurations presented in Figure 1.19. Before proceeding with the experiment, it is important to execute ping requests between the nodes, with the corresponding IP addresses, to ensure there is connectivity between them.

1.5.6.2. Starting Open vSwitch To proceed with the setup of OpenStack, we should start the Open vSwitch software [Open vSwitch 2015], the switch virtualization software that is used to create the virtual bridges for both compute and network nodes, as discussed in Section 1.5.2. During the OpenStack startup process, Neutron makes use of the running Open vSwitch software to build the necessary bridges in the server nodes. To run the Open vSwitch software in the experiment’s servers, the following command should be executed on the terminal of both compute and network nodes. $ sudo /sbin/service openvswitch start

To verify that there is no existing bridges so far, the following command should be run on the terminal of both compute and network nodes: $ sudo ovs-vsctl show

The expected result is an empty list, indicating that the nodes have no bridge configured. If that is not the case, the existing bridges should be removed. This can be accomplished by running the following command, which deletes the bridge named br0 and all of its configurations: $ sudo ovs-vsctl del-br br0

1.5.6.3. Running the OpenDaylight Controller We are now ready to run the OpenDaylight SDN controller, which will be used by Neutron to dynamically program the created virtual network bridges. As explained in Section 1.5.5, the SDN controller receives REST requests from the ODL driver, which implements the methods called by ML2 plugin for implementing the layer 2 network services provided by Neutron API. To run OpenDayligh, the following commands should be executed on the Network and Controller node: $ cd odl/opendaylight/ $ ./run.sh -XX:MaxPermSize=384m -virt ovsdb -of13

This command starts OpenDaylight, limiting the memory consumption to 384MB (-XX:MaxPermSize=384m), and also allowing support for the version 1.3 of the OpenFlow protocol (-of13). While running the controller, the terminal presents a dynamic log register of all the controller operations, including bridge creation, flow configuration and new service applications deployments.

1.5.6.4. Running DevStack Now that we have the OpenvSwitch and the OpenDaylight SDN controller running, we can start the OpenStack services on the compute node and on the network and controller node. These nodes will then make use of these software resources to create the entire virtual network infrastructure. In this experiment, we run OpenStack through the Devstack project [Devstack 2015], which consists of an installation script to get the whole stack of OpenStack services up and running. Created for development purposes, Devstack provides a non-persistent cloud environment that supports the deployment, experimentation, debugging and test of cloud applications. To start the necessary OpenStack services in our deployment, the following commands should be executed on both compute and network nodes. $ cd devstack $ ./stack.sh

The initialization can take a few minutes. The reason is that the script “stack.sh” contains the setup commands to start all the specified OpenStack services in the local node. For the purpose of this demo, the Devstack scripts located inside the nodes are pre-configured to run network, controller and compute services inside the network node and only compute services inside the compute node. For didactic purposes, we also start compute services inside the network node. This allows us to have two compute nodes in our deployment infrastructure, so we can distribute the instantiated VMs in different servers over the layer 2 configuration. To verify that we have two compute nodes up and running, the following command should be run on the network node: $ . ./openrc admin demo $ nova hypervisor-list

As a result of this command, the ID and the hostname of two hypervisors running in both compute and network node should be shown. After executing the Devstack script, we should see new log messages in the OpenDaylight terminal, inside the network node. The messages correspond to the creation of two virtual networks by Neutron using the virtual bridges. The networks created correspond to the default private and public OpenStack networks, used to connect the VMs of the default tenant in a private virtual LAN and to provide connectivity to the Internet for those VMs. By running the following command inside each OpenStack node, we should able to visualize the virtual bridges created by Neutron during the setup process with the OpenDaylight controller. $ sudo ovs-vsctl show

The results obtained should be different for the compute and the network nodes. This happens because Neutron creates the external bridge (br-ex) only for the network node, enabling the network node to provide Internet connectivity for cloud VMs. 1.5.6.5. Instantiating Virtual Machines in OpenStack In this step, we instantiate two VMs for the default gateway of the OpenStack cloud. Then, we analyze their communication through the virtual network architecture created by Neutron. The following commands instantiate the VMs, named demo-vm1 and demovm2: $ nova boot --flavor m1.tiny --image cirros-0.3.1-x86_64-uec --nic net-id=$(neutron net-list | grep private | awk ’{print $2}’) demo-vm1 $ nova boot --flavor m1.tiny --image cirros-0.3.1-x86_64-uec --nic net-id=$(neutron net-list | grep private | awk ’{print $2}’) demo-vm2

To check the status of both VMs and verify if they were successfully created, the following command should be used. The command should show that both VMs are active. $ nova list

To verify the node where each VM is running, use the following command: $ sudo virsh list

Now, to make sure that the created VMs are reachable in the network created by Neutron, we can ping them from both the router and the dhcp servers created as components of the virtual network infrastructure deployed in the network node. To be able to do that, we should first run the following command on the network node: $ ip netns

From the output of the previous command, we are able to identify the namespaces of the qrouter and of the qdhcp services running inside the network node. The namespaces are necessary to ping the VMs from both qrouter and qdhcp using the VMs’ private network IPs, as illustrated by the following command.

qdhcp-3f0cfbd2-f23c-481a-8698-3b2dcb7c2657 qrouter-992e450a-875c-4721-9c82-606c283d4f92 $ sudo ip netns exec qdhcp-3f0cfbd2-f23c-481a-8698-3b2dcb7c2657

ping 10.0.0.2

If everything was correctly configured, we should be able to successfully ping the VM, since the dhcp instance is connected to them via the internal and the tunneling bridges (br-int and br-tun).

1.5.6.6. Accessing the OpenDaylight GUI Finally, we can visualize the network topology created by Neutron via the OpenDayligh GUI (Graphical User Interface). To do that, the following url should be accessed from any of the server nodes: http://192.168.56.20:8080

The Open Daylight GUI should show all the network bridges (represented by network devices) created and configured by Neutron using the controller. It is possible to visualize the flow tables of each virtual bridge, as well as to insert and delete flows. The OpenDaylight GUI also acts as a control point for the entire data plane elements supported by the southbound API protocols, enabling monitoring, management and operation functions over the network.

1.6. Final Considerations, Challenges and Perspectives In this course we introduced the role of network virtualization on implementing and delivering cloud computing services. We presented the available network virtualization mechanisms and describe how they can address the cloud networking requirements. The advent of SDN introduced new forms of approaching network virtualization and network control inside and outside the cloud. Together with the NFV approach, SDN improved cloud network control and accelerated the provision of innovative network services. To illustrate the feasibility of integrating both cloud computing and SDN paradigms, we presented a practical study of the deployment of OpenStack and OpenDaylight. Both OpenStack and OpenDaylight are wide open source projects supported by the user and the industry communities. It is important to analyze the different challenges experienced when virtualizing networks inside cloud data centers, as well to understand how SDN and available virtualization mechanisms can help to face these challenges. The first main challenge on deploying virtual network in cloud computing is to ensure predefined performance levels for different tenant applications running inside the cloud. Cloud providers should be able to provide specific network services for tenant applications such as bandwidth, predefined in a service level agreement (SLA). Insufficient bandwidth can cause significant latency on the interaction between users and the application, reducing the quality of the service (QoS) provided to and by cloud tenants. The emergence of control models such as SDN, together with hardware-assisted virtualization technologies such as SR-IOV, is expected

to improve the control capacity over the shared network resources. Efficient architectures to integrate both control and virtualization technologies in the same cloud platform should be developed aiming to address the control granularity necessary to provide different service levels for different tenants. Ensure the flexible deployment of security appliances on tenant network infrastructure is also a challenging task for cloud networking. Organizations usually deploy a variety of security appliances in their network, such as deep packet inspection (DPI), intrusion detection systems (IDSs) and firewalls to protect their valuable resources. These are often employed alongside other appliances that perform load balancing, caching, and application acceleration. The network virtualization infrastructure should provide for cloud tenants the flexibility to deploy security appliances inside their cloud infrastructure. The use of network programmability provided by SDN platforms and data plane protocols such as OpenFlow provides the capability to deliver network security solutions as a service inside the cloud. SDN-based solutions can also abstract the implementation of these security services, providing cloud agnostic solutions and furthering standardization efforts. SDN and NFV appliances can also address the challenges on policy enforcement complexity. These policies define the configuration of each virtual and physical resource in the cloud network. Traffic isolation and access control to end users are among the multiple forwarding policies that can be enforced by deploying SDN and NFV solutions in the cloud. SDN also provides the framework to implement support for vendor-specific protocols, addressing the challenge of building, operating, and interconnecting a cloud network at scale. The need for rewriting or reconfiguring applications to address network related constraints, such as the lack of a broadcast domain abstraction and the cloud-assigned IP addresses for virtual servers, also represents a barrier for the adoption of cloud computing. The separation between control plane and data plane provided by SDN enables the development of network services that abstracts the underlying network implementation for cloud applications. Tunneling protocols such as VXLAN enables the L2 overlay schemes over a L3 network, also providing transparency to build tenant networks inside the cloud. Dealing with different requirements for topology designs in cloud data centers increases the management complexity of network configurations. Network topologies optimized for communication among servers inside the data center are not the same topologies for communication among servers and the Internet. The topology design also depends on how L2 and/or L3 is utilizing the effective network capacity, and evolving the topology based on traffic pattern changes also requires complex and dynamic configuration of L2 and L3 forwarding rules. Traffic monitoring, network intelligence and dynamic data plane configuration are requirements that can be addressed by SDN control frameworks, integrated to the cloud data centers through the control and/or the application layers. Topology design and implementation are also key components to deal with the VM migration challenges. Network appliances are typically tied to statically configured physical networks, which implicitly creates a location dependence constraint VMs. Compute node IP address is usually based on the VLAN or the subnet to which it belongs, both configured in the physical switch ports. Therefore, a VM can not be easily migrated across the cloud network, decreasing the levels of flexibility and resource utilization. The centralization of the control over the logical abstraction of the underlying network, both enabled by the

SDN paradigm, facilitates the IP management tasks in VM migration. Technologies such as SR-IOV also provides the abstraction of shared virtualization mechanisms such as virtual switches, approaching VM migration with mechanisms similar to those applied to the migration of physical servers in a L3 network. As observed in the last paragraphs and also during this course, the network virtualization in cloud computing presents major challenges. Although, the new network organization paradigm introduced by SDN, added to virtualization technologies such as NFV and SR-IOV, provides not only an entire set of virtualization and control mechanisms to meet the challenges on cloud networking, but also a new model of thinking about networking. Therefore, exploring SDN in cloud computing environments can be seen as a promising path to the innovation in the network field.

References [Al-Shaer and Al-Haj 2010] Al-Shaer, E. and Al-Haj, S. (2010). FlowChecker: Configuration analysis and verification of federated Openflow infrastructures. In Proc. of the 3rd ACM Workshop on Assurable and Usable Security Configuration (SafeConfig’10), pages 37–44, New York, NY, USA. ACM. [Alkmim et al. 2011] Alkmim, G., Batista, D., and Fonseca, N. (2011). Mapeamento de redes virtuais em substratos de rede. In Anais do Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos – SBRC’2011, pages 45–58. Sociedade Brasileira de Computação – SBC. [Amazon 2014] Amazon (2014). Virtualization Types – Amazon Elastic Compute Cloud. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ virtualization_types.html. Accessed: 2015-03-13. [Anderson et al. 2005] Anderson, T., Peterson, L., Shenker, S., and Turner, J. (2005). Overcoming the Internet impasse through virtualization. Computer, 38(4):34–41. [Autenrieth et al. 2013] Autenrieth, A., Elbers, J.-P., Kaczmarek, P., and Kostecki, P. (2013). Cloud orchestration with SDN/OpenFlow in carrier transport networks. In 15th Int. Conf. on Transparent Optical Networks (ICTON), pages 1–4. [Aznar et al. 2013] Aznar, J., Jara, M., Rosello, A., Wilson, D., and Figuerola, S. (2013). OpenNaaS based management solution for inter-data centers connectivity. In IEEE 5th Int. Conf. on Cloud Computing Technology and Science (CloudCom), volume 2, pages 75–80. [Barros et al. 2015] Barros, B., Iwaya, L., Andrade, E., Leal, R., Simplicio, M., Carvalho, T., Mehes, A., and Näslund, M. (2015). Classifying security threats in cloud networking. In Proc. of the 5th Int. Conf. on Cloud Computing and Services Science (CLOSER’2015) (to appear). Springer. [Bavier et al. 2006] Bavier, A., Feamster, N., Huang, M., Peterson, L., and Rexford, J. (2006). In VINI veritas: Realistic and controlled network experimentation. In Proc. of the 2006 Conf. on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM’06), pages 3–14, New York, NY, USA. ACM.

[Berde et al. 2014] Berde, P., Gerola, M., Hart, J., Higuchi, Y., Kobayashi, M., Koide, T., Lantz, B., O’Connor, B., Radoslavov, P., Snow, W., and Parulkar, G. (2014). ONOS: towards an open, distributed SDN OS. In Proc. of the 3rd Workshop on Hot topics in software defined networking, pages 1–6. ACM. [Berman et al. 2014] Berman, M., Chase, J., Landweber, L., Nakao, A., Ott, M., Raychaudhuri, D., Ricci, R., and Seskar, I. (2014). GENI: A federated testbed for innovative network experiments. Computer Networks, 61:5–23. [Bilger et al. 2013] Bilger, B., Boehme, A., Flores, B., Schweitzer, J., and Islam, J. (2013). Software Defined Perimeter. Cloud Security Alliance – CSA. https: //cloudsecurityalliance.org/research/sdp/. Accessed: 2015-03-13. [Braun and Menth 2014] Braun, W. and Menth, M. (2014). Software-defined networking using OpenFlow: Protocols, applications and architectural design choices. Future Internet, 6(2):302–336. [Caesar et al. 2005] Caesar, M., Caldwell, D., Feamster, N., Rexford, J., Shaikh, A., and van der Merwe, J. (2005). Design and implementation of a routing control platform. In Proc. of the 2nd Symposium on Networked Systems Design & Implementation, volume 2, pages 15–28. USENIX Association. [Canini et al. 2012] Canini, M., Venzano, D., Perešíni, P., Kosti´c, D., and Rexford, J. (2012). A NICE way to test Openflow applications. In Proc. of the 9th USENIX Conf. on Networked Systems Design and Implementation (NSDI’12), pages 10–10. [Carapinha and Jiménez 2009] Carapinha, J. and Jiménez, J. (2009). Network virtualization: a view from the bottom. In Proc. of the 1st ACM workshop on Virtualized infrastructure systems and architectures, pages 73–80. ACM. [Casado 2015] Casado, M. (2015). List of OpenFlow software projects. http://yuba. stanford.edu/~casado/of-sw.html. Accessed: 2015-03-01. [Casado et al. 2007] Casado, M., Freedman, M., Pettit, J., Luo, J., McKeown, N., and Shenker, S. (2007). Ethane: Taking control of the enterprise. In Proc. of the 2007 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM’07), pages 1–12, New York, NY, USA. ACM. [Cheng et al. 2014] Cheng, Y., Ganti, V., Lubsey, V., Shekhar, M., and Swan, C. (2014). Software-Defined Networking Rev. 2.0. White paper, Open Data Center Alliance, Beaverton, OR, USA. http://www.opendatacenteralliance.org/ docs/software_defined_networking_master_usage_model_rev2.pdf. Accessed: 2015-03-13. [Chun et al. 2003] Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., and Bowman, M. (2003). PlanetLab: an overlay testbed for broad-coverage services. ACM SIGCOMM Computer Communication Review, 33(3):3–12.

[Costa et al. 2012] Costa, P., Migliavacca, M., Pietzuch, P., and Wolf, A. (2012). NaaS: Network-as-a-service in the cloud. In Proc. of the 2nd USENIX Conf. on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services (Hot-ICE’12), pages 1–1, Berkeley, CA, USA. USENIX Association. [CSA 2011] CSA (2011). SecaaS: Defined categories of service 2011. Technical report, Cloud Security Alliance. https://downloads.cloudsecurityalliance. org/initiatives/secaas/SecaaS_V1_0.pdf. [Devstack 2015] Devstack (2015). DevStack - an OpenStack Community Production. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ virtualization_types.html. Accessed: 2015-03-13. [Duan 2014] Duan, Q. (2014). Network-as-a-service in software-defined networks for end-to-end qos provisioning. In 23rd Wireless and Optical Communication Conference, (WOCC’2014), pages 1–5. [Enns et al. 2011] Enns, R., Bjorklund, M., Schoenwaelder, J., and Bierman, A. (2011). RFC 6241 – network configuration protocol (NETCONF). https://tools. ietf.org/html/rfc6241. [Erickson 2013] Erickson, D. (2013). The Beacon OpenFlow controller. In Proc. of the 2nd SIGCOMM Workshop on Hot topics in software defined networking, pages 13–18. ACM. [ETSI 2012] ETSI (2012). Network functions virtualisation: An introduction, benefits, enablers, challenges & call for action. White paper, European Telecommunications Standards Institute (ETSI). http://portal.etsi.org/NFV/NFV_White_Paper. pdf. [Farinacci et al. 2000] Farinacci, D., Li, T., Hanks, S., Meyer, D., and Traina, P. (2000). RFC 2784 – generic routing encapsulation (GRE). https://tools.ietf.org/ html/rfc2784. [Feamster et al. 2007] Feamster, N., Gao, L., and Rexford, J. (2007). How to lease the internet in your spare time. ACM SIGCOMM Computer Communication Review, 37(1):61–64. [Feamster et al. 2013] Feamster, N., Rexford, J., and Zegura, E. (2013). The road to SDN. Queue, 11(12):20–40. [Feamster et al. 2014] Feamster, N., Rexford, J., and Zegura, E. (2014). The road to SDN: An intellectual history of programmable networks. SIGCOMM Comput. Commun. Rev., 44(2):87–98. [Floodlight 2015] Floodlight (2015). Floodlight OpenFlow Controller. http://www. projectfloodlight.org/floodlight/. Accessed: 2015-03-01. [Gember et al. 2012] Gember, A., Prabhu, P., Ghadiyali, Z., and Akella, A. (2012). Toward software-defined middlebox networking. In Proc. of the 11th ACM Workshop on Hot Topics in Networks, HotNets-XI, pages 7–12, New York, NY, USA. ACM.

[Greenberg et al. 2005] Greenberg, A., Hjalmtysson, G., Maltz, D. A., Myers, A., Rexford, J., Xie, G., Yan, H., Zhan, J., and Zhang, H. (2005). A clean slate 4d approach to network control and management. ACM SIGCOMM Computer Communication Review, 35(5):41–54. [Greene 2009] Greene, K. (2009). TR10: Software-defined networking. Technology Review (MIT). [Gude et al. 2008] Gude, N., Koponen, T., Pettit, J., Pfaff, B., Casado, M., McKeown, N., and Shenker, S. (2008). NOX: towards an operating system for networks. ACM SIGCOMM Computer Communication Review, 38(3):105–110. [Handigol et al. 2012a] Handigol, N., Heller, B., Jeyakumar, V., Lantz, B., and McKeown, N. (2012a). Reproducible network experiments using container-based emulation. In Proc. of the 8th Int. Conf. on Emerging networking experiments and technologies, pages 253–264. ACM. [Handigol et al. 2012b] Handigol, N., Heller, B., Jeyakumar, V., Maziéres, D., and McKeown, N. (2012b). Where is the debugger for my software-defined network? In Proc. of the 1st Workshop on Hot Topics in Software Defined Networks (HotSDN’12), pages 55–60, New York, NY, USA. ACM. [Hu et al. 2014a] Hu, F., Hao, Q., and Bao, K. (2014a). A survey on software-defined network and OpenFlow: From concept to implementation. IEEE Communications Surveys & Tutorials, 16(4):2181–2206. [Hu et al. 2014b] Hu, H., Han, W., Ahn, G.-J., and Zhao, Z. (2014b). FlowGuard: Building robust firewalls for software-defined networks. In Proc. of the 3rd Workshop on Hot Topics in Software Defined Networking (HotSDN’14), pages 97–102, New York, NY, USA. ACM. [IEEE 2012a] IEEE (2012a). 802.1BR-2012 – IEEE standard for local and metropolitan area networks–virtual bridged local area networks–bridge port extension. Technical report, IEEE Computer Society. [IEEE 2012b] IEEE (2012b). IEEE standard for local and metropolitan area networks– media access control (MAC) bridges and virtual bridged local area networks– amendment 21: Edge virtual bridging. IEEE Std 802.1Qbg-2012, pages 1–191. [IEEE 2014] IEEE (2014). IEEE standard for local and metropolitan area networks– bridges and bridged networks. IEEE Std 802.1Q-2014, pages 1–1832. [Jafarian et al. 2012] Jafarian, J., Al-Shaer, E., and Duan, Q. (2012). Openflow random host mutation: Transparent moving target defense using software defined networking. In Proc. of the 1st Workshop on Hot Topics in Software Defined Networks (HotSDN’12), pages 127–132, New York, NY, USA. ACM. [Jain and Paul 2013a] Jain, R. and Paul, S. (2013a). Network virtualization and software defined networking for cloud computing: a survey. Communications Magazine, IEEE, 51(11):24–31.

[Jain and Paul 2013b] Jain, R. and Paul, S. (2013b). Network virtualization and software defined networking for cloud computing: a survey. Communications Magazine, IEEE, 51(11):24–31. [Jammal et al. 2014] Jammal, M., Singh, T., Shami, A., Asal, R., and Li, Y. (2014). Software-defined networking: State of the art and research challenges. CoRR, abs/1406.0124. [Khurshid et al. 2013] Khurshid, A., Zou, X., Zhou, W., Caesar, M., and Godfrey, P. (2013). VeriFlow: Verifying network-wide invariants in real time. In Proc. of the 10th USENIX Conference on Networked Systems Design and Implementation (NSDI’13), pages 15–28, Berkeley, CA, USA. USENIX Association. [Kim et al. 2013] Kim, D., Gil, J.-M., Wang, G., and Kim, S.-H. (2013). Integrated sdn and non-sdn network management approaches for future internet environment. In Multimedia and Ubiquitous Engineering, pages 529–536. Springer. [Kim and Feamster 2013] Kim, H. and Feamster, N. (2013). Improving network management with software defined networking. Communications Magazine, IEEE, 51(2):114– 119. [Koponen et al. 2014] Koponen, T., Amidon, K., Balland, P., Casado, M., Chanda, A., Fulton, B., Ganichev, I., Gross, J., Gude, N., Ingram, P., et al. (2014). Network virtualization in multi-tenant datacenters. In USENIX NSDI. [Koponen et al. 2010] Koponen, T., Casado, M., Gude, N., Stribling, J., Poutievski, L., Zhu, M., Ramanathan, R., Iwata, Y., Inoue, H., Hama, T., et al. (2010). Onix: A distributed control platform for large-scale production networks. In OSDI, volume 10, pages 1–6. [Kotsovinos 2010] Kotsovinos, E. (2010). Virtualization: Blessing or Curse? Queue, 8(11):40:40–40:46. [Kreutz et al. 2013] Kreutz, D., Ramos, F., and Verissimo, P. (2013). Towards secure and dependable software-defined networks. In Proc. of the 2nd ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN’13), pages 55–60, New York, NY, USA. ACM. [Kreutz et al. 2014] Kreutz, D., Ramos, F. M. V., Veríssimo, P., Rothenberg, C. E., Azodolmolky, S., and Uhlig, S. (2014). Software-defined networking: A comprehensive survey. CoRR, abs/1406.0440. [Lakshman et al. 2004] Lakshman, T., Nandagopal, T., Ramjee, R., Sabnani, K., and Woo, T. (2004). The softrouter architecture. In Proc. ACM SIGCOMM Workshop on Hot Topics in Networking, volume 2004. [Lantz et al. 2010] Lantz, B., Heller, B., and McKeown, N. (2010). A network in a laptop: rapid prototyping for software-defined networks. In Proc. of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks, page 19. ACM.

[Lin et al. 2014] Lin, Y., Pitt, D., Hausheer, D., Johnson, E., and Lin, Y. (2014). Software-defined networking: Standardization for cloud computing’s second wave. Computer, 47(11):19–21. [Linux Foundation 2015] Linux Foundation (2015). OpenDaylight, a Linux Foundation Collaborative Project. http://www.opendaylight.org/. Accessed: 2015-03-01. [Mahalingam et al. 2014a] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, L., Sridhar, T., Bursell, M., and Wright, C. (2014a). Virtual extensible local area network (VXLAN): A framework for overlaying virtualized layer 2 networks over layer 3 networks. Internet Req. Comments. [Mahalingam et al. 2014b] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, L., Sridhar, T., Bursell, M., and Wright, C. (2014b). Vxlan: A framework for overlaying virtualized layer 2 networks over layer 3 networks. draft-mahalingam-dutt-dcopsvxlan-08. [McKeown et al. 2008] McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., and Turner, J. (2008). OpenFlow: enabling innovation in campus networks. ACM SIGCOMM Computer Communication Review, 38(2):69–74. [Mechtri et al. 2013] Mechtri, M., Houidi, I., Louati, W., and Zeghlache, D. (2013). SDN for inter cloud networking. In IEEE SDN for Future Networks and Services (SDN4FNS), pages 1–7. [Medved et al. 2014] Medved, J., Varga, R., Tkacik, A., and Gray, K. (2014). OpenDaylight: Towards a Model-Driven SDN Controller architecture. In 2014 IEEE 15th International Symposium on, pages 1–6. IEEE. [Mell and Grance 2011] Mell, P. and Grance, T. (2011). The nist definition of cloud computing. Technical Report 800-145, National Institute of Standards and Technology (NIST). [Menascé 2005] Menascé, D. A. (2005). Virtualization: Concepts, applications, and performance modeling. In CMG Conference, pages 407–414. [Moreira et al. 2009] Moreira, M. D. D., Fernandes, N. C., Costa, L. H. M. K., and Duarte, O. C. M. B. (2009). Internet do futuro: Um novo horizonte. Minicursos do Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos – SBRC’2009, 2009:1–59. [Nayak et al. 2009] Nayak, A., Reimers, A., Feamster, N., and Clark, R. (2009). Resonance: Dynamic access control for enterprise networks. In Proc. of the 1st ACM Workshop on Research on Enterprise Networking (WREN’09), pages 11–18, New York, NY, USA. ACM. [NOXRepo.org 2015] NOXRepo.org (2015). About NOX. http://www.noxrepo. org/pox/about-nox/. Accessed: 2015-03-01.

[NOXRepo.org 2015] NOXRepo.org (2015). About pox. Accessed: 2015-03-01. [Open Networking Foundation 2012] Open Networking Foundation (2012). SoftwareDefined Networking: The New Norm for Networks. White paper, Open Networking Foundation, Palo Alto, CA, USA. [Open vSwitch 2015] Open vSwitch (2015). Production Quality, Multilayer Open Virtual Switch. http://openvswitch.org/. Accessed: 2015-02-28. [OpenContrail 2014] OpenContrail (2014). OpenContrail: An open-source network virtualization platform for the cloud. http://www.opencontrail.org/. Accessed: 2015-02-28. [OpenFlow 2009] OpenFlow (2009). Specification, openflow switch – v1.0.0. [OpenFlow 2012] OpenFlow (2012). Specification, openflow switch – v1.3.0. [OpenStack 2015] OpenStack (2015). OpenStack: Open source cloud computing software. https://www.openstack.org/. Accessed: 2015-02-28. [Pan et al. 2011] Pan, J., Paul, S., and Jain, R. (2011). A survey of the research on future internet architectures. Communications Magazine, IEEE, 49(7):26–36. [Pan and Wu 2009] Pan, L. and Wu, H. (2009). Smart trend-traversal: A low delay and energy tag arbitration protocol for large rfid systems. In INFOCOM 2009, IEEE, pages 2571–2575. IEEE. [Patel et al. 2013] Patel, P., Bansal, D., and Yuan, L. (2013). Ananta: Cloud scale load balancing. In Proc. of SIGCOMM 2013, pages 207–218, New York, NY, USA. ACM. [PCI-SIG 2010] PCI-SIG (2010). Single Root I/O Virtualization and Sharing 1.1 Specification. PCI-SIG. http://docs.aws.amazon.com/AWSEC2/latest/ UserGuide/virtualization_types.html. Accessed: 2015-03-13. [Pfaff and Davie 2013] Pfaff, B. and Davie, B. (2013). The Open vSwitch Database Management Protocol. RFC Editor. [Pfaff et al. 2009] Pfaff, B., Pettit, J., Amidon, K., Casado, M., Koponen, T., and Shenker, S. (2009). Extending networking into the virtualization layer. In Hotnets. [Porras et al. 2015] Porras, P., Cheung, S., Fong, M., Skinner, K., and Yegneswaran, V. (2015). Securing the Software-Defined Network Control Layer. In Proc. of the 2015 Network and Distributed System Security Symposium (NDSS). [Porras et al. 2012] Porras, P., Shin, S., Yegneswaran, V., Fong, M., Tyson, M., and Gu, G. (2012). A security enforcement kernel for openflow networks. In Proc. of the 2st Workshop on Hot Topics in Software Defined Networks (HotSDN’12), pages 121–126, New York, NY, USA. ACM. [Richardson and Ruby 2008] Richardson, L. and Ruby, S. (2008). RESTful web services. " O’Reilly Media, Inc.".

[Rouse 2010] Rouse, M. (2010).

Security as a Service (SaaS). http:// searchsecurity.techtarget.com/definition/Security-as-a-Service. Accessed: 2015-03-01.

[Ryu 2015] Ryu (2015). Component-based software-defined networking framework. http://osrg.github.io/ryu/. Accessed: 2015-03-01. [Scott-Hayward et al. 2013] Scott-Hayward, S., O’Callaghan, G., and Sezer, S. (2013). SDN security: A survey. In Future Networks and Services (SDN4FNS), 2013 IEEE SDN for, pages 1–7. [Sezer et al. 2013] Sezer, S., Scott-Hayward, S., Chouhan, P., Fraser, B., Lake, D., Finnegan, J., Viljoen, N., Miller, M., and Rao, N. (2013). Are we ready for SDN? Implementation challenges for software-defined networks. Communications Magazine, IEEE, 51(7):36–43. [Sherwood et al. 2010] Sherwood, R., Gibb, G., Yap, K.-K., Appenzeller, G., Casado, M., McKeown, N., and Parulkar, G. M. (2010). Can the production network be the testbed? In OSDI, volume 10, pages 1–6. [Shin et al. 2013] Shin, S., Porras, P., Yegneswaran, V., Fong, M., Gu, G., and Tyson, M. (2013). FRESCO: Modular composable security services for software-defined networks. In 20th Annual Network and Distributed System Security Symposium (NDSS). The Internet Society. [Sridharan et al. 2011] Sridharan, M., Greenberg, A., Venkataramiah, N., Wang, Y., Duda, K., Ganga, I., Lin, G., Pearson, M., Thaler, P., and Tumuluri, C. (2011). Nvgre: Network virtualization using generic routing encapsulation. IETF draft. [Tootoonchian and Ganjali 2010] Tootoonchian, A. and Ganjali, Y. (2010). HyperFlow: A distributed control plane for OpenFlow. In Proc. of the 2010 Internet Network Management Conference on Research on Enterprise Networking (INM/WREN’10), pages 1–6, Berkeley, CA, USA. USENIX Association. [Turner and Taylor 2005] Turner, J. and Taylor, D. (2005). Diversifying the Internet. In IEEE Global Telecommunications Conference (GLOBECOM’05), volume 2, pages 6– pp. [Wen et al. 2012] Wen, X., Gu, G., Li, Q., Gao, Y., and Zhang, X. (2012). Comparison of open-source cloud management platforms: OpenStack and OpenNebula. In 2012 9th Int. Conf. on Fuzzy Systems and Knowledge Discovery, pages 2457–2461. [Xiong 2014] Xiong, Z. (2014). An SDN-based IPS development framework in cloud networking environment. Master’s thesis, Arizona State University, Arizona, USA. [Yang et al. 2004] Yang, L., Dantu, R., Anderson, T., and Gopal, R. (2004). RFC 3746 – forwarding and control element separation (ForCES) framework. https://tools. ietf.org/html/rfc3746.

[Yeganeh et al. 2013] Yeganeh, S., Tootoonchian, A., and Ganjali, Y. (2013). On scalability of software-defined networking. Communications Magazine, IEEE, 51(2):136– 141. [YuHunag et al. 2010] YuHunag, C., MinChi, T., YaoTing, C., YuChieh, C., and YanRen, C. (2010). A novel design for future on-demand service and security. In 12th IEEE Int. Conf. on Communication Technology (ICCT), pages 385–388.