Software Defined Networking as a Mitigation Strategy ...

Software Defined Networking as a Mitigation Strategy for Data Communications in Power Systems Critical Infrastructure John O’Raw Department of Computing

David M. Laverty D. John Morrow

Letterkenny Institute of Technology Letterkenny, Ireland

School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Northern Ireland

Abstract—The frequency and cost of cyber-attacks continues to grow in commercial information systems. In power systems critical infrastructure, any level of breach may be unacceptable. Securing this infrastructure requires new approaches to communications networks which are not readily vulnerable to remote attack. Standard protocols like Ethernet and IP are difficult to secure. Within well-defined domains, SDN may provide mitigation for vulnerabilities which are otherwise intractable. Methodical approaches have been developed and implemented at a test site. Index Terms—Smart Grid, Critical Infrastructure, SDN.

I.

INTRODUCTION

As the volume of commodity computing and communications devices has grown, it has become more economical to use Commercial off the Shelf (COTS) technology than to develop bespoke solutions. This move to standard technologies has had many advantages; standardized equipment and protocols have more investment in development and maintenance resulting in better functionality and testing at lower cost. Standards like Ethernet and TCP/IP have become ubiquitous in information technology. Forty years of design knowledge and development of these standards have resulted in the “Internet of things”, where almost every device may eventually be interconnected. What is often ignored is that the equipment and underlying standards which are used were designed without security in mind. Protocols like Ethernet were developed for sharing resources like printers and file servers on coaxial cable based networks where every station could potentially read every frame of data. Although network’s security have developed with technologies like switching and VLANs, the underlying design, protocols and practice are based on historical requirements with little regard for what is now appropriate.

978-1-5090-4168-8/16/$31.00 ©2016 IEEE

Because this is how we have always designed networks, there exists an orthodoxy of practice. How many network designers stand back and question a lifetime of perceived wisdom in the practices applied to process control and automation [1]? The rest of this paper is organized as follows. Section 2 considers the underlying mechanisms to Ethernet and IP and examines why networks are vulnerable. A range of exploitable protocols are identified. Section 3 looks at the real underlying causes for this vulnerability and considers mitigation strategies using software defined networking. Section 4 draws conclusions from the work and introduces a test site where these principles are being tested. II.

WHY ARE OUR NETWORKS VULNERABLE?

In network design, engineers refer to the original OSI model [2] and tend to divide protocols into layers. Layer 1 refers to the physical standards used in a network, cabling encoding and signal levels. Layer 2 is the data link layer and (oversimplifying) the protocols at this level allow communications between nodes on the same network. Rationalization has occurred in what is normally referred to as layer 2 and Ethernet [3] has emerged as the dominant protocol allowing a device on a Local Area Network (LAN) to communicate with other devices on the same LAN. Every node has a unique hardware address referred to as a MAC address (Media Access Control). Ethernet supports unicast (node to node), multicast (node to group) and broadcast (node to all nodes) communications. The early iterations of Ethernet were based on a shared medium for communications. Coaxial cable was used to daisychain between nodes (10BASE2) or transceivers were attached to a cable (10BASE5). Access to the medium was contention based; a node listened until the medium was not in use and then transmitted a pre-amble to gain exclusive access.

Assuming no collision occurred between transmitting nodes during this preamble, the node egressed a frame of data with confidence that no other node would transmit until it was complete. The term collision domain is used to describe that part of the network where a collision can occur between nodes. The term broadcast domain is used to describe the part of the network to which an Ethernet broadcast frame will propagate. On the original shared media, the collision domain and broadcast domain was always the same; the coaxial cable segment. Security of any kind was difficult to implement and every frame of data was visible to every other node. As cabling styles changed, the hub was introduced. This device collapsed the shared media network into a single box, with nodes connecting in a star topology. Improving this original design, switching was introduced. In switched Ethernet, a packet egresses on a network segment only if the destination node can be reached on that segment. The connection between a node and a network switch becomes the collision domain and all the ports on the switch are part of the broadcast domain. Security is immediately improved; a frame of data only egresses the switch to its destination node, removing the casual ability of any other node to eavesdrop. A managed network switch can be programmatically broken up into logical segments and ports can be grouped into discrete broadcast domains. Switches learn where nodes connect to them by reading the source address in Ethernet frames which transit the switch. Every time a frame of data egresses from a node, the switch takes note of its unique source MAC address and retains this address in a MAC address table. To learn the address of a destination node, the switch floods the frame of data out every port. When the switch gets a response, it again records the source address of the response in its MAC address table. Now the connection ports of both devices are known and are recorded in the MAC address table with their corresponding MAC addresses. Frames of data between the two nodes can now be switched directly in hardware with low latency. There are weaknesses to this approach. Within the broadcast domain, the initial frames of data are flooded and every node can read them. In this case the switched network is no better than the original shared media. Any device can read these flooded frames and any device can respond as if it is the correct destination node (MAC Spoofing). There are a range of techniques to carry out these attacks and some vendor dependent mitigation strategies. The MAC address tables themselves are subject to resource exhaustion attacks. If a table is full and can add no more entries, the switch may revert back to flooding frames to every port. A simple attack mechanism is to generate random MAC addresses until the table fills. The switch now acts like a hub and ceases to be a switch and the network appears as if it is shared media [4]. An attacker can eavesdrop on data flows and potentially insert itself into a communications channel with a man in the middle attack or MitM. To scale networks, the concept of Virtual Local Area Networks or VLANs was introduced [5]. VLANs are created with a unique number or tag. This tag could be associated with port groups in a switch and this could be done for multiple

978-1-5090-4168-8/16/$31.00 ©2016 IEEE

switches across a large site or campus. The result is a scheme where nodes throughout a campus can be grouped together in a single broadcast domain regardless of their physical location. In an enterprise site, nodes are typically divided into VLANs based on function and location although formal rules for these groupings do not exist. As each VLAN is a broadcast domain, this limits the scope of layer 2 attacks. For data to pass between networks or VLANs, it needs to be passed to the next layer up of the ISO model. Layer 3 is the network layer and (oversimplified) allows for packets of data to be routed between layer 2 networks. The data payload of the Ethernet frames are re-encapsulated in Internet Protocol or IP packets; other layer 3 protocols exist but IP is the most common by orders of magnitude. To find the IP address of a node on the same network, Address Resolution Protocol (ARP) is used [6]. The source node sends a frame to the entire broadcast domain to enquire for the Ethernet address of destination device with a particular IP address. ARP can be used proactively by a node to update the address tables on switches; this is called gratuitous ARP. If a device responds on behalf of another device for which it will relay, this is called ARP proxy. Once again, a malicious actor can poison or spoof ARP or ARP responses and either disrupt communications and perform a denial of service attack, or insert itself into the communications channel with a MitM attack [7]. To egress a network, a packet has to be passed to a gateway or router port on the Ethernet network. This also offers opportunities for an attacker to insert themselves in a communications channel as a MitM. VLANs tagging may also be an attack vector. A malicious node may be able to insert traffic into a VLAN merely by tagging the traffic (VLAN Hopping). Alternatively, a tagged packet can be dispatched inside a tagged packet. The switch removes the external tagging but propagates the internal tagged packet, giving an attacker access to a VLAN to which it would not normally have (Double Tagging). A MAC address attack on a single VLAN can cause flooding on all VLANs [8]. There are very many attack vectors to penetrate a modern network. Regardless of vector, the attacker’s strategy is to find any individual weakness which allows the exploitation of a vulnerability on a target network to establish a beachhead. Once the attacker has control of a single node in a target network, this node is used for a horizontal attack; to reconnoiter, enumerate and penetrate other nodes in the same subnet. Eventually a node will be exploited which allows for a vertical attack on other subnets or perhaps to otherwise secure servers. The above discussion only scratches the surface. Data networks will commonly run protocols which like have the same weaknesses as those previously discussed. For example; STP/RSTP [9], CDP/LLDP [10], DHCP [11], HSRP/VRRP [12, 13], IEEE 802.1q [14]. For routing RIP, OSPF, BGP are used, each having rudimentary security only. These protocols were generally written before security was a primary concern and make little or no allowance for the presence of attackers. These protocols are at the core of modern local and wide area

network design and at the core of the Internet itself. The common wisdom is that as these protocols are ubiquitous and thus cannot be changed, there is no quick solution to existing security vulnerability. The situation may further complicated by vendor implementations which have bugs or limitations. Misconfiguration of systems are common and must also be considered. The security of Ethernet remains with limited academic treatment; it is the domain of the hacker community and equipment vendors [15]. III.

SDN AS A MITIGATION STRATEGY

The protocols discussed have some common characteristics which are at the root of their vulnerability. The first obvious characteristic is that the protocols used predate having security as a concern. The protocols are generally very simple but they are ubiquitous. To change the security posture of these protocols will involve changing the protocol stack in millions of devices. In many cases the compute requirements of the added security may exceed the processing or memory capabilities of the actual devices, requiring complete replacement. For example, BGP has been in use since the late 1980s and the current version 4 was initially released as RFC4271 in 2006. Despite the many security issues with this protocol [16] the challenge of changing the fundamentals on which the Internet is based is insurmountable in the short term. The next obvious characteristic is that these protocols were designed for building ad-hoc networks. With an unmanaged switch, any device can be plugged into any port and will immediately communicate with all other devices. On a functional network, any device can be connected to a VLAN and will expect a DHCP address. Any switch can listen, originate and respond to control frames from any other switch and any router can listen for and originate advertisements to and from other routers. These networks and protocols were designed to be convenient and ad-hoc; inadvertently, they were also designed to be insecure. Although many protocols have mitigation strategies, these were introduced retrospectively and are far from satisfactory. Perhaps the most fundamental characteristic is that the control paths of the network and the data paths of the network are one and the same. A host which can egress a data packet to a network can equally egress a control packet. As an analogy, imagine an airport where an airline ticket entitled you to sit at an air traffic controller’s station! It is interesting to note that in telephony, the problems associated with Channel-Associated Signaling (CAS) were known about and exploited throughout the 1950s and 1960s with an early form of hacking known as phone phreaking [17]. Phone systems evolved Common Channel Signaling (CCS) specifically removing the control signals from the data path and eliminating the vulnerability. Despite the lessons learned, the data networks which evolved afterwards did so sharing control and data planes For a range of reasons, many to do with data center scalability, a debate opened up in contemporary networking on the notion of separating the control and data planes. IEEE Forces [18] and Routing Control Platform [19] were two early

978-1-5090-4168-8/16/$31.00 ©2016 IEEE

attempts. A successful example of this progress has been the Open Flow protocol [20]. In traditional devices control packets were exchanged and tables of data were maintained to dictate the decision making for each protocol. With Open Flow, a network device ceases to process the packets associated with legacy protocols. Instead, a controller explicitly populates these tables of data with rules or flow table entries. The network becomes programmable. Consider the case of two nodes connected to a switch. In a traditional network, the source node will egress a frame for a destination node which the switch will flood to every available port. The switch will enter the MAC address and connection port of the source node into its MAC address table. Should the destination node reply, the frame of data will be switched back to the source node and the switch will record the destination node’s MAC address and connection port. Two way switched connections can now occur and the MAC address table will retain its entries as long as the nodes remain active. After a period of inactivity, the entry is cleared. In a switch configured for Open Flow and intended to be default-secure, no frames of data will be switched initially at all. Any frame of data which enters the switch and for which a flow table entry does not exist is considered unknown, potentially hostile and a security event. The network implementer will explicitly document where flows of data should exist and will provision these as flow table entries in the switch, using the controller. At layer 2, there is no flooding and there is no MAC address table; there is instead a flow table of allowable flows from node to node. Other layer 2/3 protocols can also be disabled, but again manual intervention is required. ARP is not required if the devices on a local network are defined using static ARP definitions at each node. DHCP may be eliminated or the allowable paths tightly defined. STP and routing protocols may be completely unnecessary and serve no function. The access control lists or ACLs which are used as forwarding rules in routers are no longer required. The current Open Flow standards allow for flows using a range of fields from Ethernet and IP packets and it is possible to precisely tailor which types of data can transit the network to and from precisely which nodes. There are caveats to the security of this design. It is assumed that the controller is on an isolated network and is not remotely accessible. If the controller is exposed and vulnerable, the entire network is vulnerable. It is also assumed that there is no traditional functionality in the data path; e.g. that all legacy protocols have been disabled. Care must be taken as this may be difficult to do on current switches. Conceptually the design of the network has changed completely, as has its security posture. Each group of nodes appear as if they are directly connected via links which only support specifically allowed protocols. From the perspective of security, many of these new concepts have yet to be explored in the literature.

In a VLAN based design, nodes were grouped together based on functional requirements and location. For example, all of the nodes in an equipment bay might be members of a VLAN; the controller for these nodes might be configured in the same VLAN. Routing would be allowed from the controller to a historian node.

Security analysis and planning also changes. In traditional network design nodes were grouped into VLANs and secured by ACLs at the gateway to the VLAN. In an SDN secured network, nodes can be grouped into sets where members of a set have at least one node which shares a communication path. Network based attacks are limited to members of the set. Care is taken to ensure that sets do not have overlapping members and that where such overlaps exist, the overlapping node is a priority for security logging, analysis and mitigation. One approach to preventing this vulnerability is to allow for one way flows of data where possible. For example, whereas it might be essential for a historian to receive flows of data from SCADA or Data Acquisition equipment, it may only be desirable for it to have a control path. That desirability may be outweighed by the benefit of additional security a one way flow may provide concept already exists in critical infrastructure as the notion of a data diode [21]. Many instrumentation protocols take advantage of multicast and these protocols may be default configurable as one way data flows.

Figure 1. Flows in a traditional network

In an SDN design, flows are configured from node to node so whereas each node can communicate with its controller, the nodes cannot communicate with each other. A compromised node cannot be used to launch a horizontal attack on the other nodes in its VLAN.

Figure 3. Mitigating set commanality with one way flows

Intrusion detect systems (IDS) are often retro-fitted to existing networks to identify anomalous communication patterns which may relate to an attack or an existing breach. In an SDN based design, the IDS is implicit; any data packet which is not part of a known and provisioned flow is by its very nature considered a security event. Intrusion protection systems (IPS) typically respond to an alert form an IDS and take proactive action to block anomalous traffic. Any packets outside the pre-provisioned flows are blocked by default. Figure 2. Flows in an SDN based network

To exploit a vulnerability on the controller, a compromised node can only use the legitimate channel for which it has configured with the controller. For example, if a PMU is authorized to communicate with a PDC via TCP port 4000 only, it cannot exploit a vulnerability in the PDC’s SSH application, or an inadvertently open telnet port. Unless there is a vulnerability in the PDC’s application on port 4000, there is no vector for attack.

978-1-5090-4168-8/16/$31.00 ©2016 IEEE

For completeness, it should be noted that there are a range of other ways to achieve node isolation and to enforce a hierarchical data path; these techniques are typically practiced in Metro Ethernet and in Data Center technologies.

IV.

CONCLUSION

In this paper, three characteristics of networks were identified which make vulnerability inevitable. It is impractical in the short term to mitigate the millions of devices using simple protocols which are weak from a security perspective. However we now do have a strategy which will allow us to separate the control and management from the data plane and thus remove the “unfixable” protocols as a vector for vulnerability. The greatest contribution of SDN to security though may be the introduction of formality in the planning, design and inter-connectivity of data networks. The current ad-hoc approach to network design leads to poorly thought-out and poorly understood networks where documentation rarely matches reality and network topography mapping and asset tracking may in themselves be an achievement [22]. The author’s approach is to perform a rigorous security analysis of critical infrastructure network documenting all data flows. Nodes are classified as members of sets, thereby defining domains where a compromised node can potentially compromise others. The number of nodes which are members of more than one set is minimized and mitigations are enforced to prevent attacks between domains. Simulated environments have been configured and evaluated using Mininet. During 2015, the techniques discussed in this paper were applied to physical hardware and were piloted in a data center. In this pilot, power supply equipment, power distribution and HVAC equipment have been configured and secured on the SDN based network. Additional work is on-going to provide secured power status information to the server fleet in the data center.

REFERENCES [1] B. Galloway and G. P. Hancke, Introduction to Industrial Control Networks, Communications Surveys & Tutorials, IEEE, vol. 15, no. 2, pp. 860–880, Second Quarter 2013. [2] The Basic Model, ISO/IEC 7498-1, 1984, 2nd Edition 1994, Corrections 1996. [3] Carrier sense multiple access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications, IEEE 802.3-2012, section 1-6 [4] [Online] http://www.monkey.org/~dugsong/dsniff/, accessed 8th November 2015

978-1-5090-4168-8/16/$31.00 ©2016 IEEE

[5] Virtual Bridged Local Area Networks, IEEE Std. 802.1Q, 2005. [6] Ethernet Address Resolution Protocol, RFC 826, IETF, Nov. 1982, updated by RFCs 5227, 5494. [7] C. L. Abad and R. I. Bonilla, Distributed Computing Systems Workshops, 2007. ICDCSW ’07. 27th International Conference on, pp. 60– 60, Jun. 2007. [8] E. Vyncke and C. Paggen, Lan switch security: what hackers know about your switches. Cisco Press, 2007. [9] 802.1d Spanning Tree (STP/RSTP) 2004, 802.1s Multiple Spanning Tree Protocol, Merged into IEEE 802.1Q 2005 [10] Station and Media Access Control Connectivity Discovery, LLDP, IEEE 802.1AB, 11th September 2009. [11] R. Droms, “Dynamic Host Configuration Protocol,” RFC 2131 (Draft Standard), IETF, Mar. 1997, updated by RFCs 3396, 4361, 5494. [12] S. Nadas, “Virtual Router Redundancy Protocol (VRRP) Version 3 for IPv4 and IPv6,” RFC 5798 (Proposed Standard), IEFT, Mar. 2010. [13] T. Li, B. Cole, P. Morton, and D. Li, “Cisco Hot Standby Router Protocol (HSRP),” RFC 2281 (Informational), IETF, Mar. 1998. [14] IEEE Standard for Local and metropolitan area networks--Media Access Control (MAC) Bridges and Virtual Bridged Local Area Networks, IEEE 802.1Q-2011, 31st August, 2011 [15] T. Kiravuo, M. Sarela, and J. Manner, Communications Surveys & Tutorials, IEEE, vol. 15, no. 3, pp. 1477–1491, Third Quarter 2013. [16] Butler, K.; Farley, T.R.; McDaniel, P.; Rexford, J., "A Survey of BGP Security Issues and Solutions," in Proceedings of the IEEE , vol.98, no.1, pp.100-122, Jan. 2010 [17] [Online] Secrets of the Little Blue Box, R. Rosenbaum, http://explodingthephone.com/hoppdocs/rosenbaum1971.pdf, accessed 8th November 2015 [18] Haleplidis, E.; Salim, J.H.; Halpern, J.M.; Hares, S.; Pentikousis, K.; Ogawa, K.; Wang Weiming; Denazis, S.; Koufopavlou, O., "Network Programmability With ForCES," in Communications Surveys & Tutorials, IEEE , vol.17, no.3, pp.1423-1440, third quarter 2015 [19] Caesar, Caldwell, Feamster, Rexford, Shaikh, Van der Merwe. 2005. Design and implementation of a routing control platform. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2 (NSDI'05), Vol. 2. USENIX Association, Berkeley, CA, USA, 15-28. [20] [Online] https://www.opennetworking.org/sdn-resources/openflow, accessed 8th November 2015 [21] H. Okhravi, F. Sheldon, “Data Diodes in Support of Trustworthy Cyber Infrastructure”, CSIIRW '10: Proceedings of the Sixth Annual Workshop on Cyber Security and Information Intelligence Research, April 2010. [22] O’Raw, D. M. Laverty, and D. J. Morrow, Environment and Electrical Engineering (EEEIC), 2015 IEEE 15th International Conference on, pp. 1816–1820, Jun. 2015.