cloud hypervisor and security 1

CLOUD HYPERVISOR AND SECURITY Enterprise Security Architecture: Selected cloud Hypervisor Security Issues Jean-Servais Dakaud December 15, 2017 EN.695.791.81: Spring 2017: Information Assurance Architectures and Technologies Instructor: Dr. Harold J. Podell Engineering and Applied Science for Professionals Whiting School of Engineering Johns Hopkins University



1

CLOUD HYPERVISOR AND SECURITY Executive Summary Purpose The purpose of this research paper is to present within a layered security architecture1 selected mobile cloud and supporting hypervisor security architecture issues. Cloud computing security for the enterprise may also include 1) selected levels of risk for FedRAMP certification; and 2) continuous monitoring. We are focusing on risk management architecture for cloud computing and hypervisor security architecture issues. The goal of this document is to provide within a layered security architecture perspective an understanding of cloud computing and full virtualization technologies, specifically mobile cloud computing. Background: Cloud Computing Mobile devices or Internet of Things (IoT) or cyber physical systems (CPS)2 are widely used and their implementation is constantly growing. Initially, there were mobile phones and laptops. Now there are tablets, smart-watches, smart-phones, and other mobile devices that have the ability to connect to the Internet to share data. Cloud computing is a model used for accessing shared resources. There are many variations to the model.3 Mobile cloud computing is using the cloud computing framework in the context of mobile devices. These Internet connected devices serve limited purpose if not to store and share data. The networks that this data can be shared or stored on may be unsecure. In this case, cybersecurity threats in mobile cloud computing by way of cyberattacks or other potential threats expose user’s personal data and enterprise intellectual property. Mobile cloud computing requires built-in cybersecurity mechanisms and frameworks that secure data in unsecure environments for example, systems security engineering guidance from the National Institute of Standards and Technology (NIST) provides suggestions for designing in security. These mechanisms should be maintained and updated frequently to stay current with cybersecurity threats. Context We introduce two conceptual aspects of context or authoritative documents for cloud computing and hypervisor security: 1) cloud computing: candidate layered security architecture; and 2) cloud computing: continuous monitoring. For the executive summary, we introduce cloud computing risk management: candidate layered security architecture,

1 Harold J Podell, Three Key Issues for Layered Architecture, v1.21, March 11, 2015, Section 3: Five Candidate Enterprise Cybersecurity Architecture Layers presents NIST, NIST security maps, and layered security architecture. This document provides resources and references to standards and guidelines. 2 Harold J. Podell, Enterprise Security Architecture: Selected Standards-Based Security Issues (Research Paper and Systems Risk Analysis Guidance, v3.3, August 17, 2017). NIST Tier 1: Organization: Selected Updates for Big Data, Cloud computing, and Internet of Things (IoT)/Cyber-Physical Systems (CPS) or Smart Systems [Update: July 22, 2017]. 3 NIST Special Program 500-322, Evaluation of Cloud Computing Services Based on NIST 800-145, Draft 20170417. Figure 1: On-site Private Cloud; Figure 2: Outsourced Private Cloud: Figure 3: On-site Community Cloud; Figure 4: Outsourced Community Cloud; Figure 5: Public Cloud; and Figure 6: Hybrid Cloud.



2

CLOUD HYPERVISOR AND SECURITY enterprise context in the Executive Summary for selected cloud hypervisor security issues. Cloud computing: continuous monitoring is introduced in the paper after the Introduction in a Context section, it may be considered necessary to supplement effective layered security architecture. For example, for effective maintenance of cloud hypervisor security issues, we need continuous monitoring to maintain our target level of cyber risk management. Cloud Computing: Candidate Layered Security Architecture We analyze within three-candidate architectural layers selected NIST guidance, NCCoE use cases and professional publications to further understand selected mobile enterprise cybersecurity issues. Mobile enterprise cybersecurity issues depend in part on effective cloud hypervisor security capabilities. We interpret NIST’s cybersecurity guidance as providing a three-level model for enterprise architectural layers. Our interpretation is to assign NIST and related authoritative cybersecurity guidance to each of the three NIST levels 1) organization; 2) mission/business processes; and 3) system. These NIST levels are introduced in Figure 1. We illustrate some of the layered security architecture concepts that are included in Figure 1. Our illustration focuses on NIST offering for two of its frameworks a mapping of selected guidance to its three-level architecture. The two frameworks for Figure 1 are the NIST Cybersecurity Framework (CSF).4 NIST integrates cybersecurity guidance for the enterprise. For example, security framework integration may involve a mapping to a layered security architecture. The first two frameworks for enterprise layered architecture integration are NIST CSF and RMF. We interpret the primary NIST cybersecurity framework to be CSF and RMF as well as a secondary or supporting framework. The three-level architectural guidance in Figure 1 is provided in DHS C5, which supports practitioners’ implementation of the CSF by providing greater detail into how the components of the framework can be integrated into an organization’s risk management program. It specifically identifies in eight rows for common cybersecurity risk management goals and cross-references them with both the CSF components (i.e., Framework Core, Framework Profile and Framework Implementation Tiers) and NIST’s multitiered risk management approach. This mapping is introduced in Figure 1 for CSF (levels 1-3); and RMF (levels 2 and 3). We provide examples of the NIST CSF and RMF mapping in Figure 1: 1. The NIST CSF “uses risk management processes to enable organizations to inform and prioritize decisions regarding cybersecurity.”6 a. Figure 1 presents risk management issues for level 1: organization: b. NIST Level 1: Organization: “Integrate enterprise and cybersecurity risk

4 Framework for Improving Critical Infrastructure Cybersecurity, Version 1.1 (Draft), NIST, January 10, 2017. 5 NIST IR 8170 (Draft), “The Cybersecurity Framework: Implementation Guidance for Federal Agencies” May 2017. 6 Framework for Improving Critical Infrastructure Cybersecurity, Version 1.1 (Draft), NIST, January 10, 2017. Section 1.2: Risk Management and the Cybersecurity Framework.



3

CLOUD HYPERVISOR AND SECURITY management link with universally understood risk terms.” CSF Core. 2. The NIST RMF NIST Risk Management Framework offers guidance on protecting tier three information systems against threats. NIST RMF contains six steps in a cycle: categorization of information systems, selection, implementation, assessment, authorization of security controls and continuous monitoring. a. Figure 1 presents NIST RMF issues for levels 2: mission/business processes; and 3: system. For example: b. NIST Level 3: System “Inform the tailoring process using a comprehensive reconciliation of all cybersecurity requirements (supports RMF implementation).” Profiles.









7

Figure 1: Federal Cybersecurity Risk Management Needs Mapped to NIST Documentation

The relationship between the components, architectural layers and the risk management needs again reflects the layered approach to risk management, as well as the need for linkage between higher-level policies and lower-level implementation practices. NIST also provides for Internet of Things (IoT) or cyber-physical systems (CPS) risk management a conceptual extension of a three-level architecture for IoT/CPS. For example, we interpret: 1. NIST Special Publication 1500-201 Framework for Cyber-Physical Systems: Volume 1, Overview Version 1.0, June 2017. Figure 1: CPS Conceptual Model a. Level 1: Systems-of-systems b. Level 2: System c. Level 3: Device A systems security engineering aspect of IoT or CPS could include the concept of a “system risk budget.”8

7 NIST IR 8170 (Draft), “The Cybersecurity Framework: Implementation Guidance for Federal Agencies” May 2017. 8 a. NIST Special Publication 1500-201, Framework for Cyber-Physical Systems: Volume 1, Overview,

Version 1.0, June 2017.



4

CLOUD HYPERVISOR AND SECURITY 1. For example: NIST Special Publication 1500-202 Framework for Cyber-Physical Systems: Volume 2, Working Group Reports Version 1.0, June 2017. This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.1500-202 a. 2.3.3 need for cross-property risk analysis for CPS: Consider “system risk budget” and Figure 3: Physical, Analog, Cyber Components of CPS. Layer 1: Organization: Big Data and Internet of Things (IoT) or CyberPhysicalSystems (CPS) Security This layer focuses on big data security and Internet of Things (IoT). The following documents introduce security concepts in cyber-physical systems (CPS or IoT): • NIST Framework for Improving Critical Infrastructure Cybersecurity, Draft Version 1.1, January 10, 2017 • Draft NIST Special Publication 800-53, Rev 5: Draft Security and Privacy Controls for Federal information systems and Organizations, August 2017; • NIST Special Publication 1500-5: NIST Big Data Interoperability Framework: Volume 5, Architecture White Paper Survey, Final Version 1, September 2015; http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1500-5.pdf. Layer 2: Mission/Business Processes: Cloud Computing Architecture • NIST SP 800-37, Rev 2: Draft: Risk Management (RMF) for Information Systems and Organizations: System Life Cycle Approach Security and Privacy, Sep 28, 2017 • NIST Special Publication 1500-201, Framework for Cyber-Physical Systems: Volume 1, Overview, Version 1.0, June 2017. • NIST Special Publication 1500-202, Framework for Cyber-Physical Systems: Volume 2, Working Group Reports, Version 1.0, June 2017. Section 2.3.3: The need for cross-property risk analysis for CPS; “system risk budget”; and Figure 3. • NIST Special Publication 800-125: Guide to Security for Full Virtualization Technologies, January 28, 2011. • NIST Special Publication 800-125A (Draft): Security Recommendation for Hypervisor Deployment (2nd Draft), September 14, 2017. • NIST Special Publication 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Definition, March 7, 2016. • NIST Special Publication 1800-11 (Draft), Draft Data Integrity: Recovering from Ransomware and Other Destructive Events, September 2017. • Harold J. Podell, Johns Hopkins University, IoT: Network and Security Architecture: Introduction, v1.5.6, August 27, 2017. Layer 3: System: Cloud Computing Security Layer 3 emphasizes cloud-computing security in areas such as mobile cybersecurity. We consider NIST guidance and cloud service FedRAMP (Federal Risk and Authorization Management Program) certifications and standards for guidance concerning cloud computing systems security architecture.

b. NIST Special Publication 1500-202, Framework for Cyber-Physical Systems: Volume 2, Working Group Reports, Version 1.0, June 2017. Section 2.3.3: The need for cross-property risk analysis for CPS; “system risk budget”; and Figure 3. c. Harold J. Podell, Johns Hopkins University, IoT: Network and Security Architecture: Introduction, v1.5.6, August 27, 2017.



5

CLOUD HYPERVISOR AND SECURITY • The NIST Definition of Cloud Computing: Recommendations of the National Institute of Standards and Technology, SP 800-145 (Gaithersburg, MD September 2011) defines cloud computing. In particular, cloud computing is defined as a model for enabling convenient, on-demand network access to a shared pool of networks, services, servers, storage or applications. The document defines some characteristics of cloud computer such as on demand self-service, broad network access, resource pooling, rapid elasticity and measured services. NIST defines the service models used in cloud computing as Software as a Service (SaaS) and Platform as a service (PaaS). Lastly, SP defines deployment models as private/public cloud, community cloud, and hybrid cloud. • NIST SP 500-322, Evaluation of Cloud Computing Services Based on NIST 800145, Draft 20170427 (Figure 1: On-site Private Cloud; Figure 2: Outsourced Private Cloud: Figure 3: On-site Community Cloud; Figure 4: Outsourced Community Cloud; Figure 5: Public Cloud; and Figure 6: Hybrid Cloud). • NISTIR 7628: Guidelines for Smart Grid Cybersecurity V.1, September 2014. Section 2.3.6: Logical Interface Category 10: Interface between control systems and non-control/corporate systems. http://nvlpubs.nist.gov/nistpubs/ir/2014/NIST.IR.7628r1.pdf • NIST SP 800-184: Guide for Cybersecurity Event Recovery, December 22, 2016. Figure 3-1 (pp 17 of 53): NIST SP 800-184 Guide for Cybersecurity Event Recovery Relationship with the NIST CSF [Cybersecurity Framework, e.g., Level 1: Organization]—Detect, Respond, Recover. • CSA: Cloud Security Alliance: Cloud Security Alliance release Software Defined Perimeter (SDP), framework detail; https://downloads.cloudsecurityalliance.org/initiatives/sdp/SDP_Specification_1. 0.pdf • FedRAMP: Security Assessment Framework V 2.1, December 2015; https://www.fedramp.gov/files/2015/01/FedRAMP-Security-Assessment-Framework-v2. Draft): Announcement and Draft Publication https://nccoe.nist.gov/sites/default/files/library/sp1800/fs-arm-nist-sp1800-9draft.pdf. • Splunk: Splunk for Security; http://www.splunk.com/content/dam/splunk2/pdfs/solution- guides/splunkfor-security.pdf. Splunk Webinar: Leveraging Splunk to Support the NIST Cybersecurity Framework (CSF). • NIST Special Publication 500-299: NIST Cloud Computing Reference Architecture, May 15, 2013; http://collaborate.nist.gov/twiki-cloudcomputing/pub/CloudComputing/CloudSecurity/ NIST_Security_Reference_Architecture_2013.05.15_v1.0.pdf. Layer 3.1: System Technology: Mobile Enterprise Cybersecurity Layer 3 focuses on mobile enterprise cybersecurity. For example, mobile technology becomes integrated into the critical infrastructure. We consider NIST guidance on vetting security for mobile applications and case study on mobile security architecture. • NIST Special Publication 800-160: System Security Engineering, An Integrated Approach to Building Trustworthy Secure Systems, November 2016. • NIST Special Publication 800-163: Vetting the Security of Mobile Application, January 2015; http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800163.pdf.



6

CLOUD HYPERVISOR AND SECURITY • NIST, Mobile Device Security: Cloud and Hybrid Builds, NIST Special Publication 1800-4a (Rockville, MD: November 2015) details a security standards example mobile device and enterprise mobility management solution designed to protect organization against potential a data breach. The guide, developed by the National Cybersecurity Center of Excellence (NCCoE), defines security characteristics that reduce and mitigate risks caused b y mobile devices storing or accessing sensitive enterprise data. The guide maps security characteristics t o NIST standard and best practices, details example solutions and instructions, identifies possible enterprise mobility management systems, and provides solutions, which may be suitable to organizations of any size. • NIST Special Publication 1800-4b: Mobile Device Security: Cloud and Hybrid Builds: Approach, Architecture, and Security Characteristics: Draft, November 2015. • NIST Special Publication 800-63-3 Digital Identity Guidelines, June 2017.

CTIA, Today’s Mobile Cybersecurity: Protected, Secured, Unified describes the wireless communication and mobile cloud industry. It details the challenges of constantly changing threats, up-to-date cybersecurity solutions, and advanced mobile device protections. Scope and Limitations Our focus is within a layered security architecture on the effectiveness of cloud computing with hypervisor support. We consider selected enterprise security and cybersecurity risk management issues. Specifically, this document will address selected cloud models and hypervisor security issues for virtualization technologies. From understanding what mobile cloud computing and virtualization technologies are and how data is stored and accessed, we will uncover the threats faced in the mobile cloud computing industry. These threats to mobile computing include the following [1]: We also plan to address applicable mitigation measures. Mobile Malware • NIST Special Publication 800-160: System Security Engineering, An Integrated Approach to Building Trustworthy Secure Systems, November 2016. • Social Engineers • Stolen data due to loss, theft or disposal • Unauthorized access • Electronic eavesdropping • Electronic trafficking • Access to data by legitimate third party applications We will discover how these cloud security architecture IoT or CPS are built into mobile devices. This includes consideration of an architectural shift to continuous monitoring due to the growing need for on-demand access to data and services. We will discuss selected frameworks, guidelines, and best practices developed by NIST in conjunction with industry leaders in the mobile cloud-computing world. This paper will outline how these guidelines and specifications can help limit and mitigate risks associated with mobile cloud computing and cloud hypervisor security issues. We will map selected mobile cloud computing security characteristics to NIST standards and best practices. We will understand the enterprise mobility management (EMM) solutions developed by NIST to secure data storage and data



7

CLOUD HYPERVISOR AND SECURITY transmission. Particularly, the baseline for incorporating mobile security was developed from three main principles [2]: device integrity, isolation and protected storage. These three main principles are related in part to EMM policy described in: NIST SP 800-124r EMM/MDM Policy. We are introducing within a layered security architecture an introduction to selected cloud computing and hypervisor cloud security issues. Summary of Conclusions In this paper, we performed an overview of a layered security architecture related to cloud hypervisors security and their enterprise security approaches for the protection of virtual machines (VMs) as a virtual machine monitor (VMM) in virtualized infrastructures. Associating the standards-based of NIST tiered risk management approach for the security of the cloud hypervisor allows us to find out how the pieces of the puzzle, such as people, technology and processes, should be connected and guided using information security policies. A virtualization environment presents several virtual machines (VMs) that may have independent security zones9, which are not accessible from other VMs that have their own zones. Comparatively, a hypervisor has its own security zone, and it is the controlling agent for everything within the virtualization host, therefore, hypervisor can touch and affect all acts of the virtual machines running within the virtualization host [3]. There are multiple security zones, but these security zones exist within the same physical infrastructure, which in a more traditional sense only exists within a single security zone. This can cause a security issue when an attacker takes control over the hypervisor or the VMM. Then if the attacker has full control over the hypervisor, will do for all data within the hypervisor’s territory. This paper will discuss the various approaches of cloud and its effects the security of the hypervisor from architectural perspective. Keep in mind, Cloud Security Alliance’s (CSA) scopes observes up to 14 domains of cloud security issues. We are limiting the research to the governance and enterprise risk management of the hypervisor. Knowing that, hypervisors can run one or more complete virtual systems on a physical machine. Each of those systems, such as VMware’s vSphere or Citrix XenServer both leaders of virtualized management platform optimized for application, desktop and server virtualization infrastructures. Finally, this paper explores the safety and analysis of a Cloud Hypervisor security and provides standards-based security guidance based on the recommendation of the authoritative from Cloud Security Alliance (CSA). In addition, the transportation of management traffic will apply to every logical network segmentations using overlays based network segmentation, which extends the limitation of VLAN IDs configuration known as Virtual Extended Local Area Network (VXLAN). Objectively, Security on the virtual layer can be established securing the manner in which the VMs and the hypervisors communicate with the rest of the virtual network.

9 Security zones are either based on networks employed (Management, Storage, DMZ, etc.) or it can be

based on functionality such as hypervisor, management tool, backup systems, or virtual machine. IT Knowledge Exchange. December 2, 2008.



8

CLOUD HYPERVISOR AND SECURITY Contents

Jean-Servais Dakaud .................................................................................................... 1 December 15, 2017 ...................................................................................................... 1 Instructor: Dr. Harold J. Podell ..................................................................................... 1 Executive Summary ..................................................................................................... 2 Purpose .......................................................................................................................................................................... 2 Background: Cloud Computing ........................................................................................................................... 2 Context ........................................................................................................................................................................... 2 Cloud Computing: Candidate Layered Security Architecture ................................................................ 3 Scope and Limitations .................................................................................................. 7 Summary of Conclusions .............................................................................................. 8 LIST OF FIGURES ........................................................................................................ 11 LIST OF TABLES .......................................................................................................... 11 Introduction .............................................................................................................. 12 a. Purpose ................................................................................................................................................................... 12 Layered Security Architecture: Introduction .............................................................................................. 12 b. Context .................................................................................................................................................................... 14 NIST’s Role as a Standards Based Organization: Selected Security Perspectives ......................... 15 NIST’s Tiered Risk Management: An Independent Interpretation ..................................................... 16 INTEGRATED APPROACH to STANDARDS-BASED SECURITY ........................................ 16 Context: Cloud Computing Hypervisor Architecture ....................................................................... 17 Context: Cloud Computing Continuous Monitoring .......................................................................... 27 Layers 1-3: Organizations; Mission/Business Processes; System Multilevel Architectural View of Cloud Computing: Continuous Monitoring .................................................................................. 28 Cloud Computing: Selected Commercial Issues .................................................................................. 29 Virtualization Technologies: Selected Industry Views ........................................................................... 30

CASES ........................................................................................................................ 33 Intel Type1 Hypervisor ......................................................................................................................................... 33 Architectural Design of VMware ....................................................................................................................... 36 VMware vSphere Data Center Design ............................................................................................................. 37 Citrix Virtualization: ............................................................................................................................................... 38 The Citrix XenServer System Architecture ................................................................................................... 38 ANALYSIS ................................................................................................................... 41 Mapping of Federal Cybersecurity Risk Management (RMF) .............................................................. 42 Data threats and their root causes ................................................................................................................... 44 Security related concern about the security of cloud hypervisor ...................................................... 45 Mitigation measures to the security related concern .............................................................................. 46 Rollout planning and managerial issues ....................................................................................................... 46 Intrusion detection and prevention measures ........................................................................................... 46 Securing virtualization programs .................................................................................................................... 47 Security of the hypervisor ................................................................................................................................... 47 VLAN configuration ................................................................................................................................................ 48



9

CLOUD HYPERVISOR AND SECURITY Security consideration when carrying out Network Segmentation .................................................. 48

CONCLUSIONS ........................................................................................................... 50 MATTERS FOR CONSIDERATION ................................................................................. 51 Segmentation using Virtual Switches and FW ............................................................................................ 52 Security on virtual layer ....................................................................................................................................... 52 Security on physical layer .................................................................................................................................... 53 REFERENCE CITED ...................................................................................................... 54 ANNOTED GLOSSARY ................................................................................................. 58 Appendix: CONTEXT: Selected NIST Issues ................................................................. 60 Candidate Layered Security Architecture ..................................................................................................... 62 NIST Framework for Improving Critical Infrastructure ......................................................................... 68 Safety and Analysis of Cloud Hypervisor security and concept of System Risk ........................... 69 NIST: Cybersecurity Framework Feedback ................................................................................................. 69 Users of NIST Cybersecurity Framework ..................................................................................................... 70 Updates to the NIST Cybersecurity Framework ........................................................................................ 70 CISQ’s Statements .................................................................................................................................................... 70







10

CLOUD HYPERVISOR AND SECURITY LIST OF FIGURES

FIGURE 1: FEDERAL CYBERSECURITY RISK MANAGEMENT NEEDS MAPPED TO NIST DOCUMENTATION ................................ 4 FIGURE 2: FEDERAL CYBERSECURITY RISK MANAGEMENT NEEDS MAPPED TO NIST DOCUMENTATION .............................. 14 FIGURE 3: THREE-TIERED RISK MANAGEMENT APPROACH ......................................................................................... 15 FIGURE 4: NIST CYBERSECURITY FRAMEWORK ........................................................................................................ 17 FIGURE 5: FULL VIRTUALIZATION ARCHITECTURE ..................................................................................................... 18 FIGURE 6: SEGMENTATION USING VIRTUAL SWITCHES AND VIRTUAL FIREWALLS ............................................................ 23 FIGURE 7: AN EXAMPLE VLAN CONFIGURATION ...................................................................................................... 25 FIGURE 8: VIRTUAL NETWORK SEGMENTATION USING OVERLAYS (VXLAN) .................................................................. 26 FIGURE 9: ORGANIZATION-WIDE RISK MANAGEMENT APPROACH .............................................................................. 28 FIGURE 10: HYPERVISOR VIRTUALIZATION: HOSTED AND BARE-METAL ARCHITECTURES. ................................................. 29 FIGURE 11: VIRTUAL MACHINE (VMS) AND PHYSICAL MACHINES IN PRACTICAL TERMS ................................................... 32 FIGURE 12: INTEL TYPE1 HYPERVISOR ................................................................................................................... 35 FIGURE 13: VMWARE ESX BINARY TRANSLATION .................................................................................................... 37 FIGURE 14: VMWARE ESXI VERSION ..................................................................................................................... 38



LIST OF TABLES

TABLE 1: CANDIDATE ENTERPRISE LAYERED SECURITY ARCHITECTURE .......................................................................... 16 TABLE 2: MAPPING THE HYPERVISOR FUNCTIONS ..................................................................................................... 20 TABLE 3: TRACEABILITY OF SECURITY RECOMMENDATION TO HYPERVISOR BASELINE FUNCTIONS ...................................... 22 TABLE 4: MANAGEMENT ORGANIZATION RISK FOR RMF .......................................................................................... 62 TABLE 5: CANDIDATE ENTERPRISE LAYERED SECURITY ARCHITECTURE .......................................................................... 63





11

CLOUD HYPERVISOR AND SECURITY Introduction a. Purpose The purpose of this paper is therefore to analyze the advantages and the disadvantages of the virtual machines from a security point of view and come up with the suggestions that will guide an organization in using a configuration option10. This paper will explore the network protection of the virtual machines. The organization also needs to ensure the safety of the virtual machines and the application hosted on them. This paper will guide us in identifying and managing specific security risks in certain computer virtualization technologies that operate server hardware different from the storage virtualization, network and desktop. The hypervisor also known as virtual machine monitors (VMM), which runs on either 1) a bare machine or 2) the host operating system (OS) and allocates imitating resources to each guest operating system. Indeed, the way we assess security is based on a layered architecture with components connected in such a way that everything is part of a puzzle that should be well connected and understood. Layered security architecture consists of key security guidance from standards developing organizations (SDOs)11, such as the National Institute of Standards and Technology (NIST)12, Federal Risk and Authorization and Management Program (FedRAMP)13, and Cloud Security Alliance (CSA)14. To demonstrate each layer in a topdown description15, it is important to show its corresponding subtopics to the puzzle. Layered Security Architecture: Introduction We introduce within a layered security architecture risk management16 for an enterprise selected cloud computing and hypervisor security issues. A layered security architecture may be related in part to other dimensions of enterprise risk management. Other dimensions could include 1) enterprise architecture (EA)17; 2) systems security engineering during the life cycle as depicted in Figure 2, as well as in Baldrigde

10 Sailer, Reiner, et al. "Building a MAC-based security architecture for the Xen open-source hypervisor." Computer security applications conference, 21st Annual. IEEE, 2005. 11 NIST SP 800-39, Managing Information Security Risk: Organization, Mission and Information System View. March 2011. 12 NIST is housed within the U.S> Department of Commerce and promote its Framework for improving

Critical Infrastructure Cybersecurity against other jurisdictions using common language. August 19, 2016.

13 FedRAMP is a government-wide program that provides a standardized approach to security

assessment, authorization, and continuous monitoring for cloud products and services. September 26, 2017. 14 CSA Focus in Cloud Computing seeks to establish a stable, secure baseline for cloud operations. 2011: https://downloads.cloudsecurityalliance.org/assets/research/security-guidance/csaguide.v3.0.pdf 15 A top-down approach describes each layer in a top-down fashion and its corresponding subtopics. December 1, 2014: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4299037/ 16 The NIST CSF often refers to it as the “Framework” provides guidance to an organization on mapping cybersecurity risk (Section 1.0: Framework Introduction). May 12, 2017. 17 Using EA to Design Future –Ready Agencies and Implement Shared Services. May 2, 2012: https://bigdatawg.nist.gov/_uploadfiles/M0197_v1_3201181507.pdf



12

CLOUD HYPERVISOR AND SECURITY Cybersecurity Excellence Builder.18 The NIST defined levels in figure 2 are 1) organization; 2) mission/business processes; and 3) system. Finally, it helps those with level 3 system level security responsibilities understand how system-level issues impact the organization at level 1) organization; and 2) mission/business processes. Additionally, an organizational or standards-based risk analysis can include the review within three-layered security architecture for an enterprise as presented in figure 2: Cloud computing and hypervisor security may be analyzed within the three organizational layers. Data center infrastructures are increasingly becoming virtualized as a result or rapid deployment of the virtualized hosts also known as hypervisor hosts.19 Virtual machines may be considered as important resources that need to be protected in any virtualized infrastructure because they are the computer engines that hosts mission critical applications of the enterprise. Virtual machines are the products of the virtual networks; therefore, the network virtual configuration is a significant element in the entire security strategy of the virtual machines. The importance of NIST Special Publications is to provide authoritative metrics to assist in the analysis of different virtual network configuration options of protecting the virtual machines and present the suggestions. An example of an article that may be analyzed using NIST hypervisor security metrics is a technical report for adaptive reservations for hypervisor-based virtualization by Groesbrink et al20. The following are the configured areas discussed in the Special Publication 800-125A/B; both include guide to security for full virtualization technologies and deployment, such as network segmentation, traffic control through firewalls and the virtual machine monitoring.21 We consider the communication network of the virtual machines the virtual network and their associated configuration parameter are important for ensuring the security of the virtual machines and the mission application running inside them. This paper will include consideration of several cases. For example, we will analyze four main virtual network configurations of different interest in terms of network path redundancy, virtual machine traffic, traffic control using the firewalls and the security network segmentation. Different

18 Baldridge Cybersecurity Excellence Builder: Key questions for improving your organization’s

cybersecurity performance (Draft) September 2016. NIST: https://www.nist.gov/sites/default/files/documents/2016/09/15/baldrige-cybersecurity-excellencebuilder-draft-09.2016.pdf 19 An example of VMware Education is Cisco VDC Nexus 7000, understanding and techniques of Data Center Virtualization Fundamentals [V6.5]. 2009 Cisco Expo. 20 Groesbrink, S., Almeida, L., de Sousa, M., & Petters, S. M. (2014, April). Towards certifiable adaptive reservations for hypervisor-based virtualization. In Real-Time and Embedded Technology and Applications Symposium (RTAS), 2014 IEEE 20th (pp. 13-24). IEEE. 21 nd a. NIST Special Publication (SP) 800 – 125A. Security recommendations for Hypervisor deployment (2 draft). Publication date: September 2017. b. NIST Special Publication (SP) 800 – 125B. Secure Virtual Network Configuration for Virtual Machine (VM) Protection. Publication date: March 2016.



13

CLOUD HYPERVISOR AND SECURITY configuration options have different advantages and disadvantages22.







23

Figure 2: Federal Cybersecurity Risk Management Needs Mapped to NIST Documentation

b. Context The context of considering a three-layered security architecture for enterprise risk management is to bring focus to the key areas of concern for the enterprise security, such as cloud hypervisors, then highlighting decision criteria and context for each domain [4]. Since security may be considered as system property24, it can be difficult for a system administrator to separate the disparate concerns that exist at different system layers and to understand their role in the system as a whole. NIST SP 800-160 and SP 1500-202 provide frameworks for understanding disparate design and process considerations and organizing architecture and actions toward improving enterprise security25. An example of NIST Cybersecurity Framework draft metrics and measures includes 1) practices, 2) process; 3) management; and 4) technical26. An objective of security metrics is to enable decision support regarding risk management for the business.

22 Sailer, Reiner, et al. "Building a MAC-based security architecture for the Xen open-source hypervisor." Computer security applications conference, 21st Annual. IEEE, 2005. 23 NIST IR 8170 (Draft), “The Cybersecurity Framework: Implementation Guidance for Federal Agencies”

May 2017.

24 NIST SP 1500-202: Framework for Cyber-Physical System (Volume 2). The five “system risk budget”

properties are security, safety, reliability, resilience, and privacy. Version 1.0 June 2017. 25 NIST Special Publication 1500-202 Framework for Cyber-Physical Systems: Volume 2, Working Group

Reports Version 1.0, June 2017. This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.1500-202 26 Metrics are tools designed to facilitate decision-making and improve performance and accountability through collection, analysis, and reporting of relevant performance-related data. IT Security Metrics are metrics based on IT security performance goals and objectives: July 2008 http://csrc.nist.gov/publications/nistpubs/800-55-Rev1/SP800-55-rev1.pdf a. 2.3.3 The need for cross-property risk analysis for CPS: Consider “system risk budget” and figure 3: Physical, Analog, and Cyber Components of CPS b. An update for security metrics is provided in NIST: Framework for Improving Critical Infrastructure Cybersecurity, Draft Version1.1 January 10, 2017. Section 4.0: Measuring and Demonstrating Cybersecurity; and Table 1: Types of Framework Measurement: https://www.nist.gov/sites/default/files/documents////draft-cybersecurity-framework-v1.11.pdf.



14

CLOUD HYPERVISOR AND SECURITY The following metrics are examples that may be useful for cloud computing and hypervisor security27: a. Risk metrics: Measure the overall assets, and their attendant countermeasures, threats, and vulnerabilities b. Enterprise reporting metrics: Reports show the states and rates of security, they can show which areas deserve additional focus and where the security services are increasing or decreasing the overall risk exposure. c. Domain specific metrics: Provides granular view of security in a system, which can be aggregated into risk metrics and enterprise reporting formats. Run time metrics, such as alerts and warnings can be used to understand the security events that are visible across a number of systems NIST’s Role as a Standards Based Organization: Selected Security Perspectives As a SDO, the National Institute Standards and Technology (NIST) drives the processes and procedures of cryptography standards and guidelines development process in the US [5]. The complex relationships among organization, mission/business processes, and information system as shown in Figure 3, help to understand the security issues and isolate their vulnerabilities respectively. Notably, risk management requires that organizations operate in highly complex, interconnected environments using state-of-the-art and systems that organizations depend on to accomplish their missions and to conduct important business- related functions [6]. Leaders should recognize that well-informed risk-based decisions are necessary in order to balance the benefits gained from the operations and use them to avoid purposeful attacks28, environmental disruptions, or human errors that could cause business failure. Risk management brings together the best collective judgments of the administrator responsible for strategic planning, oversight, and day-to-day operations. Therefore, providing both the necessary and sufficient architecture are metrics to protect the missions and business functions of those organizations.

29

Figure 3: Three-Tiered Risk Management Approach



27 IT Security Metrics are tools based on IT security performance goals and objectives: July 2008. http://csrc.nist.gov/publications/nistpubs/800-55-Rev1/SP800-55-rev1.pdf 28 NIST SP 800-160: Systems Security Engineering. Targeted Cyber-threats attacks may include those are deliberate and persistent. There are additional targeted cyber-threats, such as those caused by human errors. November 2016. 29 The risk management hierarchy is defined in NIST SP 800-39, and provides multiple risk perspectives from a strategic to tactical level. March 2011: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-39.pdf



15

CLOUD HYPERVISOR AND SECURITY NIST’s Tiered Risk Management: An Independent Interpretation NIST tiered risk management is a structured process that monitored security programs to ensure operations of an organization remain within an acceptable level of risk, despite any changes that occur. The components of risk management are framing risk, assessing risk, responding to risk and monitoring risk. The multi-tiered risk management approach is shown in Table 1 below.





30

Table 1: Candidate Enterprise Layered Security Architecture

INTEGRATED APPROACH to STANDARDS-BASED SECURITY31 This paper addresses the layered security perspective and actions to develop more defensible and survivable systems, including the machine, and human components that compose the capabilities of the systems and services delivered by those systems. Systematically, the integrated approach to standards-based security is essentially related to the issues of the security of the national financial institutions of the United States. It depends on the reliable functioning of critical infrastructure, as a result, NIST provides a Cybersecurity Framework to improve the security of the sixteen critical infrastructures

30 Harold Podell, Johns Hopkins University: Enterprise Security Architecture: Selected: Standards-Based Security Issues: Research Paper and Systems Risk Analysis Guidance, v3.4, September 16, 2017. 31 NIST SP 800-160: System Security Engineering. November 2016. http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-160.pdf

16

CLOUD HYPERVISOR AND SECURITY sectors32. The approach of the standards-based risk analysis views the enterprise security and cybersecurity risk management as one, which is depicted in Table 1. Effectively, cybersecurity risk affects all sector of the American industry, and can drive up cost and impact revenue if the proper measures are not implemented, which in turn, it can harm the functionality of US organizations including their ability to innovate and to compete in the global market. In order to strengthen the resilience of this infrastructure, President Obama issued an Executive Order (EO) 13636 on February 12 2013 to improve Critical Infrastructure Cybersecurity. This EO called upon for the development of a voluntary risk-based Cybersecurity Framework, which elaborates a set of industry standards and best practices to help organizations manage cybersecurity risks. This Framework consists of three parts as shown Figure 4; the Framework Core, the Framework Profile, and the Framework Implementation Tiers, builds upon a set of wellestablished standards from SDOs. The objective is to address security issues, concerns, and requirements perspective and to use established engineering processes to ensure that such needs, concerns, and requirements are addressed appropriately. It is known by now that security cannot be achieved by technology alone, because there exist other factors that are equally important, such as behavioral, cultural, procedural etc. Consequently, technologies called biometrics can automate the identification of people by one or more of their distinct physical or behavioral characteristics. The term biometrics covers a wide range of technologies that can be used to verify identity by measuring and analyzing human characteristics relying on attributes of the individual instead of things the individual may have or know. [7].



33

Figure 4: NIST Cybersecurity Framework

Context: Cloud Computing Hypervisor Architecture The creation of NIST Special Publication (SP) that addresses the reference architecture of cloud computing was aimed to accelerate the adoption of the cloud computing technology by Federal agencies. These Special Publications (SPs) were developed to meet the collective effort by NIST Cloud Computing Public Security Working Group (NCC-SWG), particularly NIST Cloud Security Reference Architecture.34

32 NIST: Framework for Improving Critical Infrastructure Cybersecurity, Draft Version 1.1, January 10, 2017. Executive Summary: https://www.nist.gov/sites/default/files/documents////draft-cybersecurityframework-v1.11.pdf 33 NIST Framework for Improving Critical Infrastructure Cybersecurity, Version 1.1 Draft 2. Revised December 5, 2017. 34 a. NIST SP 500-299: Cloud Security Reference Architecture, draft version 2, July 2011:



17

CLOUD HYPERVISOR AND SECURITY It is important to note that full virtualization technologies for system applications such as server and client service provides an interface to software that maps to some underlying system, which run simultaneously more than one operating system (OS) on top of virtual hardware. As can be seen, full virtualization enables the process of multiple operating systems on a single computer. Furthermore, full virtualization architecture as depicted in figure 5 shows a difference system virtualization, where one emulates a hosted virtualization and other a classical (bare metal) virtualization and does not modify guest operating system [8]. On both models, the system virtualization interface with operating system (OS), which sees virtual machine (VM) as an actual machine. The difference lies where the bare metal (classical) virtualization layer runs on top of the hardware, usually found on servers (Xen, VMware ESX), whereas, the hosted virtualization runs on an operating system, popular for desktops (VMware Workstations, Virtual PC). As can be seen, the Hypervisor as the underlying virtualization system seats between the guest OS and the hardware, also know as a Virtual Machine Monitor (VMM).

35

Figure 5: Full Virtualization Architecture



True that the above functions are carried out by different hypervisor components or software module. Notably, the guest OSs are managed by the hypervisor, which controls the flow of instructions between the guest Oss and the physical hardware, comprising the central processing unit (CPU), hard disk, memory, and network interface cards (NICs). As we can see on both models, the hypervisor provides most the of the same hardware interfaces as those provided by the bare metal model, such that OSs and applications provisioned as full virtualization do not need to be modified for virtualization to work. Consequently, the bare and hosted platforms as shown in Figure 5 represent the two forms of full virtualization. To differentiate the twos platforms, the hosted architecture is design to run applications such as file sharing (NFS), Web browsers, email clients and hosted virtualization applications [9]. As a result, all can be compromise by a single security glitch, which could affect the hypervisor, then the host.

https://bigdatawg.nist.gov/_uploadfiles/M0007_v1_3376532289.pdf b. NIST SP 500-292: NIST Cloud Computing Reference Architecture, September 2011. 35 NIST, Special Publication 800-125, Guide to Security for Full Virtualization Technologies, January 2011.



18

CLOUD HYPERVISOR AND SECURITY Mapping the hypervisor functions to mediate access to physical resources across multiple VMs require a set of security suggestions based the hypervisor baseline functions, which consist of:36 • VM Process Isolation • Devices Emulations & Access Control • Execution of Privileged operations by Hypervisor for Guest VMs • MV Lifecycle management • Management of hypervisor These above security suggestions should be executed in order to ensure an overall integrity of all component of a full virtualize hypervisor platform. The above functions go inline with the basic function of a hypervisor that is to virtualize the hardware, as a physical host, enables VMs. The mapping of the hardware virtualization together with the hypervisor and the location of the component are present as the baseline features or functions as shown in Table 2, described below [10]: • HY-BF1: VM Process Isolation – Scheduling of VMs for execution, management of the application processes running in VMs such as CPU and Memory Management, context switching between various processor states during the running of applications in MVs etc. • HY-BF2: Device Emulation & Access Control – Emulating all Network and Storage (block) devices that different native drivers in VMs are expecting, mediating access to physical devices by different VMs. • HY-BF3: Execution of Privileged Operations for Guest VMs – Certain operation s invoked by Guest OSs, instead of being executed directly by the host hardware, may have to be executed on its behalf by the hypervisor, because of the privileged nature. • HY-BF4: VM Lifecycle management – This involves all functions from creating and management of VM images, control of VM states (Start, Pause, Stop, et cetera), VM migration, and VM monitoring and policy enforcement. • HY-BF5: Management of hypervisor – This involves defining some artifacts and setting values for various configuration parameters in hypervisor software modules including those for configuration of a Virtual Network inside the hypervisor. The hypervisor baseline functions may depend on each other. For instance, in order to protect the translation functions from over-utilization, the isolation function should process the control traffic. It means that, the hypervisor needs to provide mechanisms for orchestrating and managing the functions, so as to guarantee the valid operation of dependent functions. These orchestration and management mechanisms could be implemented and run in a centralized or distributed manner. In fact, the five baseline functions HY-BF1 through HY-BF5 do not run collectively on the same kernel, some of them could be assigned to dedicated privileged VM, or other in the hypervisor kernel itself, like HY-BF4 and HY-BF5.

36 NIST, Draft (2nd) NIST Special Publication 800-125A, Security Recommendations for Hypervisor Deployment, September 2017.

19

CLOUD HYPERVISOR AND SECURITY The five baseline as described above layout in a mapping fashion shown in Table 2.



37

Table 2: Mapping the hypervisor functions

The security of a full virtualization solutions is heavily dependent on the individual security of each of its components, including the hypervisor, host OS, guest OSs, applications, networks interface cards, and storage. Therefore, the traceability of security suggestions to hypervisor baseline function is very important to maintain the security practices describe by NIST, which enables organizations to detect and stop attacks. Since the hypervisor software resides on a physical host, therefore it manageable remotely and accessible as well. Consequently, the hypervisor could a source of threat, and one can identify three basic sources of threats as follows: • HY-TS1: Threats from and through the enterprise network in which the hypervisor host (virtualized host) resides. • HY-TS2: Threats emanating from rogue or compromised VMs through channels such as shared hypervisor memory and virtual network inside the hypervisor host. • HY-TS3: Threats from web interfaces to VM management daemon and hypervisor management console. The above threats sources are well known; therefore their traceability is periodically monitored pulled to prevent potential threats to hypervisor baseline functions. Below, Appendix B (Table 3) lists the traceability of Security Recommendation (a total of 20 SR) to hypervisor baseline function as describe in the table below:38

37 NIST, Draft (2nd) NIST Special Publication 800-125A, Security Recommendations for Hypervisor Deployment, September 2017. 38 Draft (2nd) NIST Special Publication 800-125A, Security Recommendations for Hypervisor Deployment, September 2017.

20

CLOUD HYPERVISOR AND SECURITY







21

CLOUD HYPERVISOR AND SECURITY



39

Table 3: Traceability of Security Recommendation to Hypervisor Baseline Functions



39 Draft (2nd) NIST Special Publication 800-125A, Security Recommendations for Hypervisor Deployment, September 2017.

22



CLOUD HYPERVISOR AND SECURITY With respect to architecting and configuring for VM protection, the security suggestions are the solution for the protection of the hypervisor, and HY-SR-14 through HY-SR-16 are dedicated of it. Segmentation allows VM images to be created quickly and easily. This can generate many unnecessary distributions of the same VM, and this vulnerability is generally called VM sprawl [11]. Consequently, the main reason for segmentation is to separate logically VM images with different sensitive levels or association, which increase the order of their scalability accordingly as shown in Figure 6.



40

Figure 6: Segmentation Using Virtual Switches and Virtual Firewalls

Obviously, network segmentation goes behind the purpose of visibility and network management, for the simple reason that, it enhances the security of security as an integral part of a defense-in-depth strategy. For example, the commercial industry use it entirely to facilitate and protect their credit card payment system called the Payment Card Industry Data Security Standard (PCI DSS). The method of virtual separation of VMs has five distinct scalable approaches to network segmentation, and one of these levels of sensitivity is comparable to the system of a virtual local area network (VLAN). The logical segmentation as describe is driven by levels of sensitivity in different virtualized hosts, which permits the regulation of traffic of data between different racks of physical switches using firewall rules, defined as top-of-rack (ToR) switches. The architectural design of the logical segmentation is to simplify network configuration and monitoring, but on the other side of the coin, this design will negatively impact workload balancing of the data center.

40 NIST Special Publication 800-125B, Secure Virtual Network Configuration for Virtual Machine (VM) Protection, March 2016.

23



CLOUD HYPERVISOR AND SECURITY Next the logical and physical structures of the virtual switches, and the security of the separated subnets comes in mind are defined as Demilitarized Zone (DMZ) The security structure of this topology is enforced as a security zones for both virtual and physical realms. The secured separated subnets called DMZ, which resides between secured VLANs and the outside (unsecured) network usually hosts services like webserver, ftp server, mail server, DNS server or any other application server that external user need access to from the Internet [12]. A configuration for a DMZ inside a virtualized host is depicted in Figure 6, where three virtual switches VS-1, VS-2, and VS-3 hosted inside the secured subnet show physical connection to their respective virtual NIC (vNIC), directly protected by the firewall appliances, designated as VM1 and VM4 respectively. These acting virtual bridges VM1 (outbound interface) and VM4 (inbound interface) control traffics that are going between VS-1, VS-2 and VS3. Interesting to note that, none of VM2 and VM3 is connected to the external virtual switches, therefore, not connected to any physical NIC. Such a virtual switch is called an internal-only switch.41 The design of virtual firewalls is purposely made to be easily manageable, configurable and deployable but the constraint is their similarity in their virtual network configuration. Yes, the technique of VLAN ID remains as a valuable tool for the virtual segmentation of many data center management, because VLAN technique provides tagging to all outgoing traffic from VMs to their respective media access control (MAC) address. The Figure 7 demonstrates how VLAN IDs are configured inside the virtualized host, which in turn are linked to their trunk ports, capable to send and receive traffic of multiple VLANs. In order to aggregate the traffic on the uplink, one should configure the link from the pNICs of the virtualized hosts and the physical switches, this technique enable traffic to be carried to all corresponding VLAN IDs, assuming all uplinks are also configured as trunking ports, for physical network to be carried into a virtualized host, which isolates traffic emanating from VMs for security purpose. Indeed, the implementation of VLANs in the network segmentation is more scalable than virtual firewalls because of the broadcast separation at the port level, which makes it more practical. Whereas, the firewall could limit broadcast span of VLAN IDs, which is 12 bits long On the contrary, it should exist an exact match between the virtualized host and the physical mesh of the network, which is tight. Obviously, the MAC addresses should be associated to their respective VLAN IDs; therefore, the hypervisor should be highly efficient. Also I mentioned earlier, one should take in consideration the vulnerability of the VM sprawl.42

41 NIST Special Publication 800-125B, Secure Virtual Network Configuration for Virtual Machine (VM) Protection, March 2016. 42 Virtualization sprawl occurs when the number of VMs reaches a point they can no longer manage them effectively. November 2012. https://www.vmware.com/techpapers/2012/controlling-virtual-machinesprawl-10339.html



24

CLOUD HYPERVISOR AND SECURITY





43

Figure 7: An example VLAN Configuration

Another disadvantage of using VLANs to segment a virtualized network is the limitation of the VLAN ID set to 12-bit long, which is not scalable for larger data center. Now let’s talk about network segmentation using Overlays Based Virtual Networking that is depicted in Figure 8. The meaning of an overlay network is when a computer network is built on top of another network. The sole objective to overlay a virtual network is dramatically increase the number of virtual subnets that can be created on a physical network, which systematically support VM mobility and can speed up configuration of a deployment of a data center [13]. Another key point as indicated in NIST SP 800-12B, the configuration is achieved by encapsulating an Ethernet frame received from a VM, which of course is not the only encapsulation scheme known in the network industry, but this time, the extension includes Virtual Extended Local Area Network (VXLAN), Generic Routing Encapsulation (GRE), and Stateless Transport Tunneling (STT) as depicted in Figure 8, which shows the encapsulation process using VXLAN that take place in two stages associated the VXLAN ID with the source/destination IP addresses of the tunnel endpoints (VTEP) [14]. These applications reside in the kernel module (called the overlay module) of the hypervisors. For one thing, the encapsulated VXLAN works on layer-2 and layer-3. The two stages encapsulation is performed by module of the hypervisor kernel, which maps (either by flooding or by the use of SDN controller) the layer-2 address (MAC address) with its corresponding layer-3 address (VTEP’s IP address).

43 NIST Special Publication 800-125B, Secure Virtual Network Configuration for Virtual Machine (VM) Protection, March 2016.

25

CLOUD HYPERVISOR AND SECURITY







44

Figure 8: Virtual Network Segmentation using Overlays (VXLAN)

Overlay tunneling techniques alleviate and provides Layer-2 connectivity independent of physical locality or underlying network design. By encapsulating traffic inside IP packets, that traffic can cross Layer-3 boundaries, removing the need for preconfigured VLANs and VLAN trunking. This technique provides massively scalable virtual network overlays on top of existing IP infrastructure and enables multiple VXLAN IDs assignment, which could be distributed among multiple virtual applications, where make possible to dissociate different VXLAN segments, such that belonging to different tenants can be prohibited. On the positive side, overlay-based network segmentation include the following:45 • Scalable compared to the VLAN-based approach due to the following: (a) A VXLAN network identifier (VNID) is a 24-bit field compared to the 12bit VLAN ID. Hence the namespace for VXLANs (and the number of network segments that can be created) is about 16 million as opposed to 4096 for VLANs (b) The encapsulating packet for overlay-based network segmentation is an IP/User Datagram Protocol (UDP) packet. So the number of network segments that can be defined is limited only by the number of IP subnets in the data center and not by the number of ports of virtual switches, as is the case for VLAN-based network segmentation. • In a data center offering Infrastructure as a Service (IaaS) cloud services, isolation between the tenants can be achieved by assigning each of them at least one VXLAN segment (denoted by a unique VXLAN ID). Since VXLAN is a logical L2

44 NIST Special Publication 800-125B, Secure Virtual Network Configuration for Virtual Machine (VM) Protection, March 2016. 45 NIST Special Publication 800-125B, Secure Virtual Network Configuration for Virtual Machine (VM) Protection, March 2016.

26

CLOUD HYPERVISOR AND SECURITY layer network running on top of a physical L3 layer (IP) network inside the data center, the latter is independent of the former. In other words, no device of the physical network has its configuration dependent on the configuration of any part of the virtual network. This provides the freedom to locate the computing and/or storage nodes belonging to a particular client in any physical segment of the data center network. In turn, this helps to locate those computing/storage resources based on performance and load balancing considerations. This results in greater VM mobility and availability. • It eliminates the need to configure the trunking links going into every virtualized host with many VLANs (even though VMs belonging to some of the VLANs may not exist in the host at that time), and thus avoids an increase in traffic due to overprovisioning. • Any overlay-based deployment in a production environment needs to have a control plane (and hence a controller) that facilitates automation of the provisioning functions, eliminating the chance of errors due to manual provisioning and enabling easier troubleshooting. • It is easier to configure and manage the physical firewalls since only the VXLAN (or any other overlay scheme) port needs to be allowed for all VM traffic. On the negative side, the database of the combined addressing (layer-2 and layer-3) will be significantly larger than a mapping of typical routing table. We could associate it to a SDN controller, but it becomes an additional overhead to the frame size. Context: Cloud Computing Continuous Monitoring The NIST Cybersecurity Framework (CSF)46 and the NIST Risk Management Framework47 support continuous monitoring strategy to better address the dynamic nature of computing environments and related security risks. NIST Risk Management Framework step 6 continuous monitoring may be considered as supporting the NIST Cybersecurity Framework interactive security function Detect (DE.CM). We review an aspect of DE.CM pertaining to the current version NIST Special Publication 800-37, Rev 2: Draft; Special Publication 800-37, Rev 2: Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy, September 28, 2017. According to a NIST press release for SP 800-37, Rev2 (September 28, 2017): There are four major objectives for this update • To provide closer linkage and communication between the risk management processes and activities at the C [Corporate] – suite level of the organization and the processes and activities at the system and operational level of the organization. • To institutionalize critical enterprise-wide risk management preparatory activities to facilitate a more efficient and cost-effective execution of the Risk Management Framework at the system and operational levels.

46 Framework for Improving Critical Infrastructure Cybersecurity”, Version 1.1 DRAFT, NIST Jan 2017. 47 NIST SP 800-37 Rev. 2, Draft: Risk Management Framework for Information Systems and Organizations: A Security Life Cycle Approach for Security and Privacy (Discussion Draft), September 28, 2017.

27

CLOUD HYPERVISOR AND SECURITY • To demonstrate how the Cybersecurity Framework can be implemented using

the established NIST risk management processes (i.e., developing a Federal use case). • To provide an integration of privacy concepts into the Risk Management Framework and support the use of the consolidated security and privacy control catalog in NIST Special Publication 800-53, Revision 5. The NIST risk management approach for continuous monitoring is consistent with the NIST risk management approach for risk management. In brief, NIST recommends a three-level approach to risk management 1) organization; 2) mission/business; and 3) system. This is presented in Figure 1 and 2. For example, we may view continuous monitoring from a layered security architecture viewpoint: Level 1: Organization: Big Data and Internet of Things (IoT) or Cyber Physical Systems (CPS) Security NIST Cybersecurity Framework, Draft Version 1.1, January 10, 2017. Detect. Continuous Monitoring (DE.CM) Level 2: Mission/Business Processes: Cloud Computing Architecture NIST SP 800-37, Rev 2 Draft: Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy, September 28, 2017. Figure 3: Risk Management Framework, Step 6: Monitor. Level 3: System: Cloud Computing Security NIST SP 800-184: Guide for Cybersecurity Event Recovery, December 22, 2016. Figure 3-1: NIST SP 800-184 Guide for Cybersecurity Event Recovery Relationship with the NIST CSF [Cybersecurity Framework, e.g., Level 1: Organization]—Detect, Respond, Recover Layers 1-3: Organizations; Mission/Business Processes; System Multilevel Architectural View of Cloud Computing: Continuous Monitoring We provide a multi-layered security architecture view of cloud computing continuous monitoring relating the NIST Cybersecurity Framework to the NIST Risk Management Framework.







48

Figure 9: Organization-Wide Risk Management Approach



48 NIST SP 800-37 Rev. 2, Draft: Risk Management Framework for Information Systems and Organizations: A Security Life Cycle Approach for Security and Privacy, September 28, 2017:



28

CLOUD HYPERVISOR AND SECURITY NIST, Draft NISTIR 8170, The Cybersecurity Framework: Implementation Guidance for Federal Agencies, May 2017.49 • Key Issues: Relationships of Key NIST Risk Management Guidance - an interpretation of Figure 1: Federal Cybersecurity Uses • Organization Levels 1-3 represent guidance levels for layered security architecture. For example, the NIST Cybersecurity Framework may be viewed applying top down from Levels 1 and 2. • NIST suggests that we use the five iterative security controls defined in the Cybersecurity Framework as a common set of terminology for Levels 1-3. • NIST identifies the NIST Risk Management Framework (RMF) as applying to levels 3 and 2 • Further, we may view the RMF step 6: Monitor Security Controls as supporting continuous monitoring, which is defined in the Cybersecurity Framework as Detect. Continuous Monitoring (DE.CM). Cloud Computing: Selected Commercial Issues Commercial Cloud computing is today’s most exiting field of cybersecurity, because it reduces the cost and complexity of applications, also, it is flexible and scalable.50 These parameters make commercial cloud computing one of the fastest growing technologies. This paper focuses on the security of hypervisor hosts, like many virtualization techniques, it allows multiple operating systems (OS), terms guests, to run concurrently on a host computer, commonly called hypervisor virtualization as shown in Figure 10. The term hypervisor because it seats virtually one level higher than a virtually supervisor, and presents to the guest OS a virtual operating platform and monitors the execution of the guest OS (guest operating systems). Multiple instances of a variety of OS may share the virtualized hardware resources as depicted below. Hypervisor is installed on server hardware whose only task is to turn, runs guest OS.





51

Figure 10: Hypervisor Virtualization: Hosted and Bare-Metal Architectures.



https://csrc.nist.gov/CSRC/media/Publications/sp/800-37/rev-2/draft/documents/sp800-37r2-discussiondraft.pdf 49 NIST, NISTIR 8170 (Draft), The Cybersecurity Framework: Implementation Guidance for Federal Agencies: May 12, 2017: https://csrc.nist.gov/publications/detail/nistir/8170/draft 50 Use of Commercial Cloud Computing and Services. November 2015: www.dtic.mil/get-tr-

doc/pdf?AD=AD1002758 Real-Time Hypervisor Architecture and Performance details. September 28, 2016. http://www.ni.com/white-paper/9629/en/

51



29

CLOUD HYPERVISOR AND SECURITY The hypervisor-based virtualization approach as shown above establish a controllable environment and can utilize additional security tools recommended by guidance from SDOs, including NIST, VMware FedRAMP compliance resources and most importantly from Cloud Security Alliance (CSA). However, the hypervisor-based virtualization is vulnerable because of its single point of failure. If the hypervisor crashes or the attacker gains control over it, then all VMs are under the attacker’s control. However, taking control over the hypervisor from the virtual machine level is difficult, though not impossible. According to this characteristic, this layer chose for implementing proposed security architecture. A virtualization environment presents several virtual machines (VMs) that may have independent security zones52, which are not accessible from other VMs that have their own zones. Comparatively, a hypervisor has its own security zone, and it is the controlling agent for everything within the virtualization host, therefore, hypervisor can touch and affect all acts of the virtual machines running within the virtualization host [15]. There are multiple security zones, but these security zones exist within the same physical infrastructure that, in a more traditional sense, only exists within a single security zone. This can cause a security issue when an attacker takes control over the hypervisor. Then the attacker has full control over all data within the hypervisor’s territory This paper will discuss the various approaches of cloud and its effects the security of the hypervisor from architectural perspective. Keep in mind, Cloud Security Alliance’s (CSA) scopes observes up to 14 domains of cloud security issues. We are limiting the research to the governance and enterprise risk management of the hypervisors types 1 (a bare metal, which runs directly on top of hardware); and type 2 (operates as an application on top of existing OS). Virtualization Technologies: Selected Industry Views Virtualization technologies and cloud computing have made significant changes to the way computer resources are managed and administered. Virtualized hosts (also called hypervisor hosts) are increasingly deployed in data centers53 because of efficiency, scalability and cost considerations. The virtualized infrastructure resulting from the deployment of virtualized hosts has three main categories of components, Hypervisor Software, Virtual Machines (VMs) and Virtual Networking components such as Virtual Network Interface Cards (vNICs), Virtual Switches and Virtual Firewalls [16]. A virtualized host is a physical host that runs or operates server virtualization products such as hypervisors and enables it to capture various computing stacks, which have different configuration platforms such as the middleware and the operating system. An individual computing stack in a virtualized host is summarized in virtual machines. Since virtual machines are compute engines, they have various resources assigned to them and these resources are known as virtual resources. Data center with virtualized hosts are believed to have virtualized infrastructure and the hypervisor that are inside the

52 CSF: Framework for Improving Critical Infrastructure Cybersecurity. January 2016: https://www.nist.gov/sites/default/files/documents/cyberframework/CybersecurityFramework-for-FCSM-Jan-2016.pdf 53 Cisco System definition of Data Center. Technology Design Guide. August 2014:

https://www.cisco.com/c/dam/en/us/td/docs/solutions/CVD/Aug2014/CVD-DataCenterDesignGuideAUG14.pdf



30

CLOUD HYPERVISOR AND SECURITY virtualized host usually defines a network that interlinks all the virtual machines and the physical enterprise network.54 The network is known as the virtual network because it is software defined. The most important components of these virtual networks are the sequence of network interface card inside every virtual machine and the virtual switches that are designed to function inside hypervisor kernel. Hypervisors are available at boot time of machines to control the sharing system of various resources across all virtual machines. Some of the virtual machines are partitioned in a manner that allows them to manage the virtualization platform and host the VMs. The partitioning of the Virtual machines establishes a more controllable environment, which uses additional security tools such as the IDS. Virtual machines are however vulnerable because they are hosted by hypervisor and when the hypervisors are hacked, they can be controlled by a hacker or unauthorized person. Out of the three categories of components above, hypervisors constitute the fundamental resource to be protected in a virtualized infrastructure, since they are the compute engines on which business/mission critical applications of the enterprise run. These VMs are virtual counterparts of physical servers as shown in Figure 11 and hence just like their physical counterparts, security for these VMs has to be provided through host-level and network-level measures. In reality, the mechanisms or approaches required for providing protections to hypervisors are different because the VMs are end nodes of a virtual network as opposed to being end nodes of a physical network [17]. For this reason, this paper gives guidance on the identification and the management of specific security risks related to hypervisors as opposed to storage virtualization, network and desktop. The hypervisor runs on the host operating system (OS) and allocate imitating resources to each guest operating system [18]. Hypervisors are available at boot time of machines to control the sharing system of various resources across all virtual machines. Some of the virtual machines are partitioned in a manner that allows them to manage the virtualization platform and host the VMs. The partitioning of the Virtual machines establishes a more controllable environment, which uses additional security tools such as the IDS55. Virtual machines are however vulnerable because they are hosted by hypervisor and when the hypervisors are hacked, they can be controlled by a hacker or unauthorized person. Under those circumstances, virtualized environments are subject to different risks than traditional network environments. One of the risks with virtualized platforms is that the number of layers through which VM infrastructure is implemented is enormous. Troubleshooting events, activity logs, and crashes can be quite difficult. It is vital to setup software tools properly to ensure that all information necessary for monitoring can be captured accurately. Just like physical machines, virtual machines also contain a lot of critical, sensitive, user profiles, passwords, license keys, and history. While the risk of data loss is immense with both physical and virtual machines, the risk is much greater with virtual machines as it is much easier to move files and images from virtual machines than it

54 Sabahi, F. (2012). Secure virtualization for cloud environment using hypervisor-based technology. International Journal of Machine Learning and Computing, 2(1), 39. 55 SANS Institute InfoSec Reading Room: An Intrusion Detection System (IDS) is a network security

technology originally built for detecting vulnerability exploits against a target application or computer. 2001



31

CLOUD HYPERVISOR AND SECURITY is to hack into physical machines via network links. Many images and snapshots are captured by virtual machines in order to deploy or restore system restores and they can be prone to data theft.





56

Figure 11: Virtual Machine (VMs) and Physical machines in practical terms

In addition, virtual machines, the risk is much greater with virtual machines as it is much easier to move files and images from virtual machines than it is to hack into physical machines via network links. Many images and snapshots are captured by virtual machines in order to deploy or restore system restores, and they can be prone to data theft. It is critical for organizations that are implementing virtualization technologies to understand the kind of risks that such systems face, and put in mitigating measures. Again, this paper presents and integrated view of how to analyze a selected security architecture issues for a tiered risk management approach that pertain to standards-based security. We consider standards-based security as a multidisciplinary approach to develop and measure the success of an enterprise security program [19]. Our focus is on the effectiveness of enterprise security and cybersecurity risk management efforts. The selected architecture issue would be the security of cloud hypervisors. Hence, in providing security suggestions for the hypervisor, two different approaches have been adopted in this paper. One approach based on architectural options that provide ease of security assurance and the second approach based on configuration choices that form part of its core administrative functions such as management of VMs, hypervisor host, and virtual networks. This approach creates communication and improves key security requirements, as well as, principles and models that describe the enterprise’s future security state to enable its evolution. Enterprise Security Architecture is not about developing for a prediction, it is about ensuring that we develop in a way that allows us to maintain and sustain our agility to change. We do not know where we are going or how we are going to get there but we need to be ready.

56 Virtual Machine (VMs) and Physical machines in practical terms:. November 17, 2016. Sourced from: https://www.hostingaustralia.com.au/hosting/physical-vs-virtual-server/

32

CLOUD HYPERVISOR AND SECURITY CASES The growth of virtualization in data centers has introduced vulnerabilities, such as basic forms of attack include attack on the hypervisor through either the guest OS or host OS. Other forms of attacks that can potentially compromise a virtualized system are virtual library checkout, migration attack, and encryption attack. These types of attacks, unlike the basic forms, do not attack the architecture of virtualization directly. Also, some new challenges in security are faced with the implementation of virtualization. Monitoring is harder due to lower visibility in a virtualized environment. This is also caused by the abstraction that virtualization brings. In addition, the infrastructure of virtualization is an on-going challenge for modern data centers and clouds With the vulnerabilities in virtualization, many solutions have been developed to combat them. The most basic forms of security involve implementing traditional security mechanisms such as intrusion detection software and firewall on components of virtualization such as the hypervisor and the guest OS. Also, security on how images of VMs are transported, stored and managed is very important due to mobility of VMs. To add additional layer of security, infrastructure security of virtualization is used. This form of security usually involves securing the virtual infrastructure, the physical infrastructure or bot. Two examples that implement these generic solutions to combat vulnerabilities in virtualization are Trend Micro's solution and vBlock (for enterprises) developed by EMC. Intel Type1 Hypervisor57 One of the biggest challenges with virtualization is the lack of visibility into virtual networks used for communications between virtual machines. This poses problems when enforcing security policies since traffic flowing via virtual networks may not be visible to devices such as intrusion-detection systems (IDS) installed on a physical network. This is due to the nature of virtualized systems. Network traffic flowing between virtual machines does not originate at a particular host and the hypervisor is generally not able to monitor all communications happening between virtual machines. The hypervisor is a software layer between the underlying hardware platform and the virtual machines. It provides one more possible attack point for hackers to gain access to VMs. This is a potentially serious vulnerability as the hypervisor is the program that controls the operation of the VMs. There can even be entry points via the VMs themselves whereby malware that has infected one particular VM is able to penetrate the hypervisor and by doing so, also compromise other VMs that the hypervisor controls. To mitigate against this lack of visibility into VMs, we need to understand the five basic functions of a hypervisor, particularly when it has to virtualize a physical host to enable running of multiple VMs.

57 Xen* Hypervisor Case Study - Designing Embedded Virtualized Intel® Architecture Platforms. Aneja, A. (2011, March). Sourced: https://pdfs.semanticscholar.org/bf0f/6a586858e730f5873edd1f2b8006dfd093ba.pdf



33

CLOUD HYPERVISOR AND SECURITY The required functionality of the hypervisor can be interpreted as VM manager (VMM) with the following features [20]: • HY-BF1: Execution Isolation for VMs: It schedules VMs for execution, management of the application processes running in VMs such as CPU and Memory Management, context switching between various processor states during the running of applications in VMs etc. • HY-BF2: Devices Emulation & Access Control: It emulates all Network and Storage (block) devices, so that different native drivers in VMs are expecting, mediating access to physical devices by different VMs. • HY-BF3: Execution of Privileged Operations for Guest VMs: Certain operations invoked by Guest O/Ss, instead of being executed directly by the host hardware, may have to be executed on its behalf by the hypervisor, because of their privileged nature. • HY-BF4: Management of VMs: Setting configuration parameters for VMs (VM Images) and control of VM states (Start, Pause, Stop etc.). • HY-BF5: Administration of Hypervisor Platform and Hypervisor Software: This involves setting of parameters for user interactions with the hypervisor host as well as hypervisor software and configuration of Virtual Network inside the hypervisor. In general, functions HY-BF1 and HY-BF 3 run in a kernel of the hypervisor, while the function HY-BF2 is assigned to the module called QEMU that runs outside the hypervisor, usually in a dedicated privileged VM. The QEMU module spawns one process for each running VM. Whereas HY-BF4 and HY-BF5 are assigned to module management or service console, and module QEMU, generally runs on top of the hypervisor as a privilege VM and could be built either with a full-fledged O/S installed inside in it or with a ultra-light O/S used to present an API (shell and network access) with utility functions that facilitates performing just the hypervisor-specific configuration and administrative tasks. Based on this baseline architecture, HY-BF1 is provided by the Virtual Machine Manager (VMM), while HY-BF2 and HY-BF3 are encapsulated within a privileged VM called the “Management VM”, with the former function provided by a process called “QEMU” while the latter function provided by a “Management Daemon”. Coming from the description of the baseline functions of the hypervisor, we can explore couple of cases. Starting with the open source hypervisor, Xen for an embedded application to be virtualized, such as Intel Type1 Hypervisor as depicted in Figure 12. The open source hypervisor type1 Xen has evolved in the past few years, keep up with the stringent realtime and mission-critical demands of customers for virtualized x86 CPU in the embedded systems industry [21]. The goal of this case study is to get a better understanding of the architectural design of the hypervisor software implementation. The hypervisor itself is a complete piece of software since it has all the resource allocation, separation and isolation functions, which allows it to perform some of the I/O data traffic flow to/from between VMs.



34

CLOUD HYPERVISOR AND SECURITY













58

Figure 12: Intel Type1 Hypervisor

Xen as shown in Figure 12 is a Type1 Hypervisor, and contains three main components: • Xen Hypervisor: The Xen hypervisor is the basic abstraction layer of software that sits directly on the hardware below the operating systems. A small hypervisor kernel deals with virtualizing the CPU, memory and critical I/O resources such as the Interrupt controller. • Domain0, the Privileged Domain (Dom0): Dom0 is the Privileged guest (Paravirtualized Linux) running on the hypervisor with direct hardware access and guest management responsibilities. Dom0 is an integral part of the Xen-based virtualized system and is basically Linux OS modified to support virtualization. Dom0 also has a control panel application that controls and manages the sharing of the processor, memory, network and disk devices. • Multiple DomainU, Unprivileged Domain Guests (DomU) –DomU‟s are the unprivileged guests running on the hypervisor; they typically do not have direct access to hardware (for example, memory, disk, et cetera). The DomU can be of two types – PV Guest (Para-virtualized Guest OS) and HVM (Hardware Virtualized Machine) Guest. Dom0 and DomU operating systems use hypercall‟s to request services from the Xen hypervisor layer underneath. The HVM type DomU guests however use hardware based mechanism like VM Exit/VM Entry on Intel® VT enabled platforms to switch control between the hypervisor and HVM guest OS. The HVM guests are also known as fullyvirtualized guests as the OS and device drivers run unmodified in their native configuration versus paravirtualized DomU guests where the guest OS or device driver code is typically modified to support virtualization. The hypervisor Xen manages the processor/CPU utilization in a virtualized environment to be associated with other host CPUs. Important point, For the HVM domain, the Virtual CPU module in the hypervisor provides the abstraction of a processor to the guest. In order to translate the guest physical address, the Virtual Memory Management Unit (MMU) module in the hypervisor provides an abstraction of the hardware MMU to the guest operating systems. While the main key design ingredients are processor, memory and I/O device management, there are other essential key features that need to be evolves, such as; inter-process communication; PCI

58 Bare-metal hypervisors (Type-1) are booted as a machine’s operating system and  sometimes through a primary privileged virtual machine (VM). XenServer and VMware ESXi are prominent modern examples of Type-1: April 21 2011: https://hackernoon.com/linux-server-virtualization-the-basics-32079b0e7d6e



35

CLOUD HYPERVISOR AND SECURITY device pass through, USB (HVM and PV guests) pass through and graphic pass through. Xen is a good example to understand what hardware and OS support may come into play. Understanding how the hypervisor layer controls resource allocation such as processor, memory and device I/O is very important to find the right embedded virtualization software solution. The Type1 Hypervisor Xen is an evolving virtualization solution that implements many known resource control and optimization techniques and can be a good case study for understanding how hypervisor layer is architected in a virtualized solution. Architectural Design of VMware59 This is a server virtualization platform that comes with VMware ESX hypervisor along with relevant management tools. vSphere can be found in a variety of editions namely, Standard, Advanced and Enterprise Plus. As a hardware virtualization, VMware makes it possible for organizations to leverage server infrastructure in a manner that is more efficient. Either, such organizations are able to simplify their administration costs for it allows multiple virtual OS machines to be created on just a single hardware platform. Notably, VMware guest machines are able to run virtually any type of Operating system. They are also presented with standardized sets of emulated hardware so as to do away with the sophistication that comes along with varied hardware. In light of management tools, it has the ability of migrating Physical hosts to the Virtual machines (P2V) in addition to rapid backup and at time migration of virtual machines to the other virtual hosts. Another management tool is the ability to allow fast prototyping and finally providing for the use of shares SAN storage. Any vSphere environment is made up of the following products; • ESXi: This is regarded as the base mainstay of virtualization. The VM lives and runs on it • vCenter: The product is responsible for the management of the multiple ESXi servers, creates cluster, runs HA, the DRS and features such as vMotion stem from it. • Single Sign-On (SSO): the most recent development in VSphere saw the development of vSphere 5.1 that has SSO. The SSO manages identity. But, it should be noted that currently, there is no integration that provides for View into SSO. • Inventory Service: This is the store, keeping an inventory of vSphere objects. Through it, the response time for each request made to the inventory is fastened and in so doing, there is less load created into the inventory center. Within the View environment are various products some of which are not available thus should be installed. To begin with, there is a View Connection Server which is the major component. In the View connection Server is the HTTPS-instituted View Administration Interface. The other components of the View Connection Server include the standard, the replica and the security. Whereas the replica is used to balance loads besides failure capacity, the security server forwards incoming View Client connections to the new standard server. The security server is deployed in what is called the DMZ. DMZ is a form of virtual firewall mainly used for the internet facing applications are running nonvirtualized hosts. The use of DMZ helps in separating the virtualized hosts in a bid to

59 NIST SP 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Protection. March 2016

36

CLOUD HYPERVISOR AND SECURITY protect network60. This is so for enterprise applications that are at different sensitivity levels at the VMs hosting points. The ESX has taken towards virtualization is best described through binary virtualization. This is to means that any time the Operating system makes a request to the processor, the request is intercepted after which it is translated into an instruction that is virtualization friendly. For instance, if a halt request is made the operating system, it is only the specific VM which is suspended so as to release resources to be used by the other VMs. In so doing, the guest OS is tricked by the ESX into thinking that it is actually running on some physical hardware. Since there is need for a lot of such work to be executed, the ESX system is generally very sophisticated.





61

Figure 13: VMware ESX binary translation

VMware vSphere Data Center Design62 As described previously, the Xen hypervisor is a layer software running directly on computer hardware replacing the operating system (OS) thereby allowing the computer hardware to run multiple guest OSs concurrently as shown on Figure 13. Support for x86, x86-64, Itanium, Power PC, and ARM processor allow the Xen hypervisor to run on a wide variety of computer devices that we could find in data center, which concurrently support Linux, NetBSD, FreeBSD, Solaris, Windows, and other common OSs as guests running on the hypervisor. We are assuming this datacenter doesn’t have any redundant site, and the architecture in the vSphere datacenter is viewed as the highest level of logical boundary that, its main

60 NIST SP 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Protection. DMZ is usually created to separate subnets under physical firewalls. March 2016. 61 Full Virtualization using Binary Translation: An example of server virtualization platform comes with VMware ESX hypervisor along with relevant management tools. Source https://www.unf.edu/~sahuja/cloudcourse/Fullandparavirtualization.pdf pp 3-5 of 47. November 24, 2017. 62 VMware virtual infrastructure and Cloud Management. September 2010: https://scap.nist.gov/events/2010/itsac/presentations/day2/network_automation-extendingscap_into_the_vmware_virtual_infrastructure.pdf



37

CLOUD HYPERVISOR AND SECURITY purpose is to delineate different physical sites location and potential additional vSphere infrastructure with an independent function. Within the datacenter, EXSi hosts are mostly grouped in clusters, which are later used in various groups of virtual machines that require different storage requirements and the networks as shown in Figure 14. Categorizing EXSI into clusters facilitates the use of technologies such as vSphere, MVware and vMotion. vSphere Distributed Scheduler (DRS), vSphere Distributed Power Management (MVware DPM), vSphere Higher Availability (HA) and vSphere Fault tolerance. It is therefore recommendable for one to create a single vSphere cluster with 10 hosts because several clusters would results into high overheads from an AH perspective. Citrix XenServer system is a bare metal or simply the Type 1 kind of hypervisor technologies. Type 1 hypervisor are generally installed directly onto a variety of physical servers without necessarily having to use the host operating system (OS). Most IT experts have recommended that the bare-metal approach can offer better performances as well as manageability, when compared to the hypervisors that depend on host OS, such as VMware GSX Server (GSX) in addition to Microsoft Virtual Server.











63

Figure 14: VMware ESXi version

Citrix Virtualization64: The Citrix XenServer System Architecture The development of XenServer started once the virtualization processors and operating systems had hit the market. XenServer is established on an open-source environment, known as Xen Hypervisor. It does not make use of binary translation, such as VMware but rather uses paravirtualization, besides hardware-support virtualization. For Xen, paravirtualizations started with Linux, which is also one of the main OS.

63 VMware ESXi is an upgrade from ESX without the Linux Kernel and uses Direct Console User Interface (DCUI) for management. It is a software-driven solution and provides some of the best backwardscompatibility in the Intel world. May 23, 2011: http://virtualization.info/en/news/2011/05/paper-vmware-esxi-4-1-operations-migration-guide.html 64 Citrix virtualization optimized server and boost data center efficiency. July 12, 2017: https://virtualizationreview.com/articles/2017/07/12/citrix-past-present-and-future.aspx



38

CLOUD HYPERVISOR AND SECURITY







65

Figure 15: XenServer Hypervisor Architecture

There are multiple components involved in a Citrix virtualization infrastructure, available in two main editions; XenApp (suitable for businesses that have fewer application and configuration requirements) and XenDesktop (suitable for organizations that require fully capable desktop environments for client devices). XenApp is the central product that is used for seamless application delivery, while XenDesktop extends the capabilities to publish VDI (Virtual desktop Infrastructure) desktops Both of these products are capable of securing data and applications by delivering virtual resources to users based on location and device-specific security configurations.

66 Figure 16: Citrix Systems XenServer Architecture In a Citrix application delivery setup, applications and resources are hosted on central servers. XenApp isolates these applications from the underlying OS and other applications,

65 Using NIST 800-125-A to understand hypervisor security threats Paul Henry. SANS Institute, 2017 http://searchcloudsecurity.techtarget.com/tip/Using-NIST-800-125-A-to-understand-hypervisorsecurity-threats 66 The Citrix XenServer is a “type 1” hypervisor that runs directly on the hardware, on which runs a Linux virtual machine, called “domain 0” and based on CentOS, that runs the management toolstack and API, also contains drivers for network and storage I/O. June 30, 2016: https://www.citrix.com/blogs/2016/06/30/xenserver-7-building-the-foundations-of-a-great-future/



39

CLOUD HYPERVISOR AND SECURITY and streams them into an isolated environment on the target device where they are executed. You don’t have to install applications or software on the client device; only the configuration settings, data, and application files are copied to the client device. The user sends keystrokes and mouse clicks to the server and receives screenshot updates. The client device should have the XenApp client software Receiver installed on it. As XenApp provides access to hosted resources from a Windows server, users share the server’s physical resources. Citrix XenDesktop has an extended scope. It is used to publish complete virtual desktops from a hypervisor to remote client devices. This means every user gets his/her own instance of the OS and desktop; resources are not shared between users. When comparing Citrix XenServer with VMware vSphere, it should be noted that these two software systems are developed and supported by different companies. The VMware vSphere is by VMware Inc., whereas XenServer is developed by Citrix. Their difference lays in the intended usage of the software. The Citrix XenServer is developed for small to medium businesses, including for personal use. Whereas, VMware vSphere is only intended for businesses, therefore no structure for personal use. Both of these system programs run the bare-metal (type1) hypervisor type and support the x86/x64 architecture. Though they support various types of virtualization such as hardware assisted virtualization and paravirtualization, only VMware vSphere does support full virtualization. Neither of the two software programs supports operating system virtualization. After all, of the main problem that has come along with the advent of virtualization to the data centers is that the introduction of vulnerabilities to such virtualized environments. The primary attacks have been effected in such environments have come in the form of the hypervisor being attacked by the guest OS or the host OS. Apart from that, it should be noted that some of such attacks have the potency of compromising the whole virtualized system, for instance, library checkout, migration attacks and encryption attacks. Unlike other forms of attack, the above namely never attack the architecture in a direct manner. There are also those challenges that are have resulted from the implementation of virtualization since it is overtly hard to monitor, it is hard to lower the visibility of any virtualized environment due to abstraction brought about by virtualization. Notably, virtualization infrastructure is one of the challenges that keep affecting modern data centers and clouds. A number of solutions have been developed to curb some of the problems that virtualized environments have been experiencing. Some of such mitigations include the implementation of traditional security measures like intrusion detection, use of firewall also software components that come with virtualization, for instance, the hypervisor together with the guest OS. Another area of concern is the manner in which the images are transported, stored and taken through various form of management since there is increased mobility within the VMs. Using infrastructure security follows addition of any extra layer of security. Such a kind of security infrastructure if achieved through virtual environment or at times both the physical and virtual infrastructure. Two of the basic examples entail the implementation of generic solution that have the capability of combating many of the vulnerabilities within the virtualized environment, for example, using the Trend Micro solutions or the vBlock, which are developed by EMC.



40

CLOUD HYPERVISOR AND SECURITY ANALYSIS Virtualization has hugely benefited the information technology and networking as it has delivered tremendous cost saving and returns on investments to enterprise data centers and cloud service providers. Generally, drivers involved in machine virtualization include multi tenancy machine, which provide the best server utilization, data center consolidation and relative ease and speed provisioning. Cloud Service Providers (CSPs)67 in most cases achieve high density that translated to better margins. Most business enterprises use virtualization to minimize capital expenditures on server hardware and at the same time increase operational efficiencies. The following are some of the reason that may make one think that virtualized environments are more secure than the traditional ones is because of: • The isolation of virtual machines provided by the hypervisor • The ability to deliver the most important infrastructure and security technologies in virtual appliances such as the firewall and the network switches. • There has been no known successful attack on hypervisors except for the theoretical ones, which requires one to have an access to the hypervisors source codes and the ability to implement them. • An ability to recover quickly and quarantine from incidents. Some people think that the new designed virtualized environment requires a security similar to the rational physical environments and as a results it is not always the case to see the processes, legal security solutions and strategies applied to virtual environments. The bottom line is that the new environment is the one that is more convoluted thus it requires a new approach to security. Once services have been moved from the physical virtual realms, the enterprises by all means increase their exposure to threats. While operating under the physical realm, majority of the threats seem to be observed from the external networks also internal networks. On the contrary, the virtual realms increased the attack surfaces. This implies that there should be increased attention directed towards the hypervisor. Thus, several security considerations should be put into play if at all there is need to curb the risks associated with such threats. As enterprises get on their virtualization agendas, it is important to consider the existing processes and come with strategies that address security risks in both physical and virtual environments. This approach will enhance compliance and the security visibility in data centers [22]. • According to the reports by the cloud security alliance (CSA) experts identified the following as the most critical threats to cloud security • Data breaches • Data loss • Service traffic hijacking • Malicious insiders

67 Real-World Business Technology: A Cloud Service Provider or CSP is a company that

offers a variety of cloud computing platforms: IaaS, SaaS or PaaS to other businesses or individuals; e.g.: The IBM Cloud, The Dell EMC Cloud, Amazon Web Services, Microsoft Azure, Google Cloud etc. Source: Toms IT Pro. March 7, 2014



41

CLOUD HYPERVISOR AND SECURITY • Share technologies vulnerabilities • Insecure APIs and interfaces • Abuses of cloud services

Considering the number of notable breaches report in the year 2014, virtual security is therefore given due to consideration in creation, management and planning of an enterprise and the provider environment [23]. This paper thus explores and proposes the most important security frameworks that can help and secure any virtual environment and prevents it from threats that include the aforementioned, from exploiting the vulnerabilities. This paper tackles virtualization security from the perspectives of the hypervisors and it briefly elaborate on the specifics security concerns. Mapping of Federal Cybersecurity Risk Management (RMF) Cloud computing is defined by the National Institute of Standards and Technology (NIST) as a model for enabling convenient, on-demand network access to a share pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”.68 The evolution of information processing that moved from mainframes to the Web has come alone with a major milestone in technology and business collaboration. It quickly turned out that the version of delivering hosted services over the Internet required an implementation of a layered security architecture with many dimensions of enterprise risk management, depicted in Figure 2. One of these dimensions include the three level of the Risk Management Framework, shown as follow: 1. Organization as a core, which integrates enterprise and cybersecurity risk management. 2. Mission / Business processes as a profiles and implementation tiers, which enable integration and prioritization of all requirements, such as: 1) cybersecurity requirements; 2) alignment and acquisition of all processes; 3) evaluation of organizational cybersecurity based on NIST Standards; 4) management of the cybersecurity programs (supports RMF Implement & Monitor); 5) maintain a comprehensive understanding of cybersecurity risk based on NIST Standards (support RMS Authorize); and 6) report of a structured cybersecurity risks 3. System as a profile(s), which dictates the tailoring process of all cybersecurity requirements (supports RMF Implement). Effectively, the adoption of cybersecurity framework in the cloud-computing environment has increase significantly as other recent studies show that over 70 percent of companies are starting to use the three levels of the Risk Management Framework. The NIST Federal Cybersecurity RISK Management Framework is especially helpful in the management of cybersecurity programs as it is based on a continuous improvement cycle and it is the most comprehensive frame to date. As defined, Risk is the possibility that something will happen and adversely affect the business’ goals and those types of Risks are security, integrity, availability, and

68 Peter Mell and Tim Grance. The NIST Definition of Cloud Computing, Version 15, October 7, 2009: https://www.nist.gov/sites/default/files/documents/itl/cloud/cloud-def-v15.pdf

42

CLOUD HYPERVISOR AND SECURITY performance.69 In fact, the types of Risks are the same with any environment where business is performed, including cloud technology solutions and the level of an organization, Risk profile will in most cases change if cloud solutions are adopted. The second line of defense would the various risk control and compliance oversight functions, such as requirements, alignment and acquisition, evaluation, management and maintenance. Finally the third would be methodologies to process all cybersecurity requirements; therefore, each of these three levels of Risk Management plays a distinct role within the organization’s wider governance framework. In order to fully understand and then implement the three levels of NIST Federal Cybersecurity Risk Management, one should have a complete appreciation of the structure of the organizational multi-tiered risk management. Important to know that risk management process is specifically detailed by NIST SP 800-3070, which is a Guide for Conducting Risk Assessments by providing an overview of how risk management fits into the system development life cycle (SDLC). This approach is consistent with the process for managing information security elaborated in NIST SP 800-39, which includes framing risk, assessing risk, responding to risk and monitoring, known as the flexible approach as shown in Figure 17.





71

Whereas, NIST SP 800-37 discusses the Risk Management Framework that is the subject to Federal Information Systems, which is developed to ensures that managing information system security risks in consistent with the organization’s objectives and overall risk strategy, as well as, information security requirements that are integrated into a layered security architecture and SDLC as depicted in Figure 18. Members of NIST and the Joint Task Force Transformation Initiative determined that the best approach to risk management is to view risks at not only the system level, but also at the business unit level and the organization level [24].

Figure 17: Multi-Tiered Process Applied to Managing Information Security Risk

69 Guide for Conducting Risk Assessments. NIST SP 800-30 Revision 1. September 2012: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-30r1.pdf 70 NIST SP 800-30, Rev 1. Guide for Conducting Risk Assessments and uses the key factors of threats, vulnerabilities, impact to missions and business operations. September 18, 2012. 71 NIST SP 800-39, Revision 4. Security and Privacy Controls for Federal Information Systems and Organizations. Updated as of January 22, 2015.

43

CLOUD HYPERVISOR AND SECURITY

Figure 18: Risk, Management Framework to Federal Information Systems72 Data threats and their root causes When data has been keyed into a computer, the expectation is that it be stored until it is processed to information. There are several threats related to data insecurity because of numerous factors. The biggest threat to any data is virus infection. It makes data inaccessible despite being physically seen on the screens of the computers. These viruses can be transmitted from one computer to another by the use of flash discs as well as USB cables. Another way of virus transmission is downloading of documents via the email that contains unsecured sites. Another threat is the corruption of data to available different information. In case a hacker would like to pass wrong information to the public from an organization, he only needs to access the data and makes some unnecessary changes73. The changed information will reach the public so fast considering the rate at which information flows in the current world. The hackers mostly use the social media platform in conducting their illegal activities. When wrong information about a company is sent, their image will be negatively branded causing the customers to pool out of its operations. This results from changing the original data to a misleading one. Sometimes it becomes difficult to convince the public of the happenings since at the back of their minds they are aware that company was responsible. Therefore, company data should be well protected to avoid such occurrences from happening.

72 NIST SP 800-37, Rev 1 (February 2010). Guide for Applying the Risk Management Framework to Federal Information Systems, which includes updates as of June 5, 2016: http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-37r1.pdf 73 Mitchem, T., Lu, R., & O'Brien, R. (2016, December). Using kernel hypervisors to secure applications. In Computer Security Applications Conference, 1997. Proceedings., 13th Annual (pp. 175181). IEEE.



44

CLOUD HYPERVISOR AND SECURITY Another data threat is crashing of computers. Some computers are of low standards and can crash at any given time74. When this happens, it becomes difficult to retrieve such data unless there was a backup set up. When purchasing a computer in a company, one should go for the quality brands to avoid such mischief. All the saved documents should also have back up for easy retrieval of the lost data. The root cause of the data threats results from the nature of the exact threat. For instance, virus attacks occur when the antivirus software is not installed to a computer or else they are not updated. This makes it simpler for unwanted programs to install themselves in the computer hence affecting the normal functioning of such computers. Unsecured Internet sites are also a leading cause of virus transmission. Some criminals send viruses deliberately to emails to affect other computers. One should therefore be so keen while downloading files from unfamiliar sites. Unhealthy competition among companies is also a cause of data threat. Some companies would prefer to use the hackers to pass wrong information to the public about their competitors. The aim is to create a negative image of the company for its customers to shift their consumption. An unskilled worker who cannot secure data properly is also another cause of data threats. A company should employ experts who will be able to secure the data available from being accessible by the unauthorized persons. This will ensure proper storage of information for the good of the company. Lastly, purchasing computers of low quality is also a threat since they may crash leading to the loss of important information. Companies should use reliable computers that will not develop unnecessary problems. Security related concern about the security of cloud hypervisor Any data access within a virtual machine can generate a series of other data access operation in the hypervisor on a remote server, which can easily change the subject and the type of access. The interaction between the client and the server programs in a particular network can result in a similar situation. During the process of interaction over the network, the subject on the server’s side and the subject on the client’s side belong to the same user as the authentication and identification mechanisms are used. In case vulnerabilities exists within the hypervisors, attackers can easily exploit it and gain an access to the host, from which he will have an access to virtually all the guest virtual machines running on the host machine. Since hypervisors are rarely updated, any kind of vulnerability can interfere with the security of the entire system leading to loss of important information. Suppose a weakness is identified, it is important to scrap it up as soon as possible to prevent it from compromising the system. Data leakage sometimes poses security threat in a system especially when the data storage is used by one virtual machine and relocated to another. In most cases, a leakage does occur when a virtual machine, which is no longer, required in the system, is deleted and the freed resources are relocated to other virtual machines. When a new virtual machine

74 Szefer, J., & Lee, R. B. (2012, March). Architectural support for hypervisor-secure

virtualization. In ACM SIGPLAN Notices (Vol. 47, No. 4, pp. 437-450). ACM.



45

CLOUD HYPERVISOR AND SECURITY receives other additional resources, they can use forensic investigations techniques to acquire the image of the entire physical memory and data storage [25]. The images can be used in forensic analysis, which can be used to reveal important information that has been left from the previous virtual machines. Another security concern is the virtual machine attacks. If a person successfully hacks in to the systems or compromises one virtual machine, he or she can hack the other virtual machines belonging on the same host on a network for a very long period. This is the most common method of cross virtual machine attacks because standard IPS and IDS software programs cannot monitor the traffics between the virtual machines. Cloud hypervisors are also prone to migration attack. When necessary, the migration of a virtual machine can be efficiently done in most virtualization interfaces. The virtual machines send over the networks to another corresponding virtualization server where a similar virtual machine is set up. When the process is not properly conducted and managed, the virtual machine can be send over an unencrypted channel, which can be hijacked by a hacker, or an attacker who is conducting a man-in-the-middle attack in the system. In most cases, this usually happens when an attacker accesses another virtual machine when the network is compromised. Mitigation measures to the security related concern Hardening is the process of improving the security of the virtual machines by reducing their exposure to vulnerabilities and threats. Generally a single function system is considered to be more secure than the multifunction systems thus the larger the vulnerability surface the more the virtual machine are exposed to threats. Hardening thus involves removing of unimportant username and disenabling unnecessary services and login details. The following are method that can be sued to mitigate potential threats associated with virtual environment. Rollout planning and managerial issues Establishment of a comprehensive security policy with regard to virtualization follows that the same guidelines are applicable to any IT system development. Therefore, the fundamental framework defined by the Sarbanes Oxley is applied to almost all machines environment [26]. Intrusion detection and prevention measures Intrusion detection and prevention systems are of great importance in all networks. Detection techniques is involved detecting the various logging and intrusion s and categorizes them and finally provide feedback to security devices with an aim of changing the set rules thereby generating an IPs architecture. Intrusion detections and preventions measures can be used for the purposes of identifying the various problems with security policies and preventing users from going against the security policies. IDs and IPs can also be used in the documentation of security threats in a system or network [27].



46

CLOUD HYPERVISOR AND SECURITY Securing virtualization programs When an organization acquires a server virtualization program, it should ensure that its entire information security governance framework is applicable to its virtualized information technology systems and services. An organization considering virtualization should identify and explore the possible security risks and come up with measure of addressing them before implementation [28]. In the initiation phase, the organization is supposed to identify the virtualization requirements by providing an overall vision on how the virtualization solution will provide solutions to support the vision and the mission of the organization and at the same time, creating strategies that is at forefront of implementing the virtualization solutions. An organization should also develop a virtualization policy by coming up with platforms and applications that virtualizes and specify the function of a business requirement. During the design/planning phase, an organization is supposed to provide all the required guidance that specifies and evaluate the technical features of the virtualization solutions, including other related components, which include the authentication mechanisms and the cryptographic method that are aimed at protection important information. Some of the most important consideration in this phase includes selection of virtualization, network topology, and availability of bandwidth, storage systems and business continuity [29]. Design should also consider the use of appropriate logical segregation of instances, which contains very important information. A separate authentication form application, hypervisor, and guest operating system and host operating system with an aim of providing a different layer of security and protection. Organization should define processes that are core to handling and incidents that involves virtualization of solutions. During implementation, an organization or an enterprise should ensure that all the security practices according to NIST guidance are established through an extensive assessment of the vulnerability to the virtualization components. The primary virtualization platform should be hardened using a vendor-provided guideline or the party tool. In a virtualized environment, keys are supposed to be enforced to allow segregation of duties thus facilitating proof of governance. Appropriate and proper data governance is required in the implementation stage to keep track, control and identify data that contain sensitive information at any given time. A proper virtual machine encryption is required to minimize the risks that are associated with accessing the physical servers and the storage that contains the most important information. During deposition, all the tasks are defined. At this stage, all the virtual machines retirement processes are supposed to meet regulatory and legal requirement in order to minimize the cases of data breaches and leakages that may include shredding of keys that are associated with encrypted virtual machines. Constant internal and external audit of the virtualized environment will help in identifying and mitigating the weaknesses and the vulnerabilities and at the same time maintaining the required legal standards. Security of the hypervisor In so far as the security of the hypervisor is concerned, it is paramount for the VMs to be protected. This is so since most of the main threats in many virtualization platforms take place by malicious VM gaining access to an area of the memory of another VMs. The above condition is referred to as VM Escape attack or at times jailbreak attacks as an attacker basically “Escapes” a VM’s confinement into other layers, which are to a greater extent unknown to a given VM. But, it should be noted that most of the VMs do share physical



47

CLOUD HYPERVISOR AND SECURITY resources therefore, many of the attackers can easily establish the manner in which the VM virtual resources have been mapped to such physical resources making is possible for them to carry on with attacks directly onto the physical resources. Either, any virtualized system has the hypervisor as the sole management layer meaning that in case the hypervisor is compromised, then all the VMs that have been created and are under the control of the hypervisor are also compromised [30]. The aforementioned as it relates to security is a similar scenario for image management form of security because securing of the manner in which the VM images are stored, undergo transportation and management for a given data center. A good illustration for the above is in VLAN configuration. VLAN configuration Traffic among various members, which, become isolated from the traffic that belongs to another group create a logical group of VMs. It is possible to related logical separation in the network traffic that is offered through the VLAN’s configuration to a range of arbitrary criteria. In so doing, it is possible to achieve the following. Managing the VLAN used in the transportation of management traffic. The management traffic is utilized when one needs to send either management or configuration commands towards the hypervisor [31]: • To log the VLAN carrying the traffic utilized in fault tolerant logging. • To mitigate the VM VLAN used to carry the traffic that is generated at the time of migrating the VM. This is like migrating the VMs from some virtualized host onto another one such that it is available or for purposes of load balancing. • The storage VLAN used in carrying the traffic relating to the Network File System (NFS). • The desktop VLAN used to carry traffic from the VMs that runs the Virtual Desktop Infrastructure (VDI) software. • To create a group of production VLANs to be used in carrying traffic between among the VMs, that is, the VM groups that host a variety of business applications. Nowadays, virtually very enterprise business architecture has three tiers, that is, the webserver, the application and the database. The creation of separate VLANs is possible among the three tiers and traffic between them regulated by the use of firewall rules. On top of that, in cloud data centers, the VMs can belong to varied clients by the use of VLAN configuration. Consequently, there is creation of one or more logical and at times virtual network segment for every tenant by either assigning or connecting the VMs that belong to the each onto different VLAN fragments. Apart from confidentiality assurance, there is integrity assurance that is provided through the logical subdivision of the network traffic not forgetting the quality of service (QoS) rules, which can be applied to the different VLANs in line with the type of traffic in transit. Security consideration when carrying out Network Segmentation The protection of VMs is one of the potential security vulnerabilities for hypervisor, because the common threats in virtualization platforms is a malicious VM accessing area of memory of other VMs. Also known as VM Escape attack or jailbreak attacks as the attacker essentially “escapes” the confinement of the VM into layers that are otherwise unknown to the VM, and does it through guest OS to gain unauthorized access to other MVs, or the hypervisor [32]. However, many VMs share the same physical resources, then many attackers could find how the VM virtual resources is map to the physical resources, so that



48

CLOUD HYPERVISOR AND SECURITY he can conduct attacks directly on the physical resources. We know that the hypervisor is the entire management layer for a virtualized system, thus if the hypervisor is compromised, so all the VMs created and controlled by the hypervisor. This goes for image management security, since securing of how VM images are stored, transported, and managed in a virtualized data center is an important aspect of security In line with the various approaches applied for VM protection, there a number of suggestions that have been put forth in so far as security is concerned. Every recommendation is linked to some unique identifier with a format VM-NS-Rx. In the identifier, VM represents Virtual machine, NS stands for network segmentation while Rx stands for recommendation sequence [33]. VM-NS-R1: If for a particular environment uses virtual switches for purposes of network segmentation, it is good to use distributed virtual switches. Standalone switches should be avoided due to a number of reasons. First, there is need to make sure that there is consistency in the configuration all over the virtualized hosts at the same time reduce the possibility of any configuration error from occurring. Second, there is need to do away with the VM migration constraints because a given distributed virtual switch can serve many virtualized hosts. VM-NS-R2: Using virtual switches to isolate the network used in managing the hypervisor but with special configuration. For instance, apart from dedicated virtual switches being used in providing security having the traffic encrypted, it will be wise to have separate set of pNICs along the management traffic path. Furthermore, the dedicated virtual switches should be of a type known as stand- alone virtual switch to make it possible for them to be configured at virtualized host levels. The recommendation is based on the tenet that there is a close dependency existing between the virtualized switches and the various centralized management switches that have been virtualized. The only way to configure the distributed virtual switches is through the use of virtualized management servers. This implies that such servers ought to be available. However, it may be necessary to have the distributed virtual switches modified if at all it is anticipated to that virtualized management servers are to be used. VM-NS-R3: During the deployment of VLANs it is advised that the switch port configuration ought to be VLAN aware. This is to mean that the configuration should replicate the VLAN profile, which the virtualized host has. VM-NS-R4: If a data center is large, that is, has a myriad of virtualized hosts and VMs thus required many subdivisions ought to deploy overlay-based form of networking. The reason behind this is that the overlay virtual network allows both scalability and provides both virtual and physical network independence. But, it is advised that the entire network traffic that is generated through such an overlay kind of network segmenting techniques like the VXLAN network traffic should be sequestered on the logical network using techniques such as VLAN indoor in the process of maintaining segmentation guarantees. VM-NS-R5: Use either centralized SDN controller or a federated SDN controller that employs standardized protocols in configuring the overlay modules when used across a variety of hypervisor platforms just in case you are using the large overlay based virtual networking.



49

CLOUD HYPERVISOR AND SECURITY CONCLUSIONS In this paper, we adopt virtualization architecture to secure cloud by considering enterprise security architecture approach and the cloud hypervisor security for the protection of MVs in virtualized infrastructures. Adopting this proposed architecture, we are able to reduce the workload, decentralize security-related tasks between hypervisor and MVs, and convert the centralized security system to a distributed one. The distributed security system is very good way to reduce the workload from hypervisor-based virtualization, but this distribution may inject vulnerabilities to cloud. In addition, distributed security systems have more complexity than centralized ones. Consequently, we introduced layered security architecture and its connected elements, which are useful to manage risk management at different levels regarding information in an organization. In the layered architecture, every sub-item is apiece of the puzzle, so information security can be analyzed as a whole. In this paper, we performed a detailed analysis of the enterprise security architecture approach and the cloud hypervisor security for the protection of VMs in virtualized infrastructures. Associating the standards-based of NIST tiered risk management approach for the security of the cloud hypervisor allows us to find out how the pieces of the puzzle, such as people, technology and processes, should be connected and guided using information security policies. Furthermore, the enterprise security issues related to the cloud hypervisor for a tiered risk management approach establish standard requirements for all cryptographic-based security systems used by government organization and commercial entities to protect sensitive or valuable data. Conformance testing can then be performed against the standard to provide assurance to users that standards-based security modules are built to requirements. For such, authoritative documents may be used as metrics for standards-based risk analysis, which can be analyzed by mapping the security characteristics of a case to the NIST cyber security framework (CSF) security control function. CPS cybersecurity requirements should be provided in connection with the safety, resilience, reliability and the privacy requirements. The resilience of the CPS should state the ways and mechanisms that perpetuates in not only the Information Technology systems but also the critical CPS operations in case of failures in the cyber-attack about the CPS recovery. This can be achieved through co-designs of the CPS cybersecurity that involves the privacy, the security, resilience and reliability. As a result, some factors need to be put into consideration of the previous tenants and of privacy and confidentiality, the cybersecurity of the CPS and the integrity of the system. CPS cybersecurity literally involves prioritization and application of the CPS. NIST Special Publications help establish common security requirements and the capabilities needed for secure solutions. Furthermore, NIST establishes standard requirements for all cryptographic-based security systems used by government and businesses to protect sensitive or valuable data. Conformance testing can then be performed against the standard to provide assurance to users that cryptographic modules are built to requirements. It consistently addresses security in every layer, reducing unmanaged risks and improving operational security efficiency. From such a perspective, the proposed layered security architecture can be used as a guide to helps achieve better results regarding cloud hypervisor security.



50

CLOUD HYPERVISOR AND SECURITY Correspondingly, we are observing an increasing adoption of cloud services by large enterprises that have to host multi-tier applications, so that the network engineers in charge of the data center need a flexible virtual networking topology with capability to provide the necessary isolation through the technique of network segmentation. In addition, it is essential for those virtual network segments to span scalable IP subnets of the data center, including remote hypervisor clusters. Today, the only virtual networking technology that can provide these capabilities without a great deal of physical network reconfiguration or addition of networking resources is the overlay-based virtual networking. The separation between the virtual networks and the physical networks provided by overlay-based techniques allows to configuration without difficulty to maintain the logical network segmentation within large data centers. As a result, the overlay-based network segmentation has been adopted economically and operationally to support multi-tenant workloads for IaaS cloud providers. Using the NIST 800-125-A publications, all forms of firms can understand the threats that exist in the virtual real in a better way. The NIST 800-125-A draft clarifies why it is necessary for the organizations to provide for some mitigations that can thwart away security threats such as segregation, isolation in addition to providing the rule of last privilege75. The three recommendations as provided by NIST 800-125-A are applicable across any form of platform. As such, any environment where the highest level of risk mitigation is required should consider putting them in place. In particular, NIST 800-125A provides extra value by addressing the extended threat envelope while operating on top of the hypervisor apart from prevention of threats that are within the hypervisor itself. MATTERS FOR CONSIDERATION It is important to decentralize security tasks and reduce the workload between the virtual machines and the hypervisors. Distribution of security system based on the layered security architecture approach is an ideal method of reducing the workload of the hypervisor but it also comes with vulnerabilities that may jeopardize the operation of the system or network. If it does not use utilize a context of authoritative guidance, such as the National Institute of Standards and Technology (NIST) security guidance. In most cases, distributed securities appear to be more complex than the centralized ones. Security is the most important thing in cloud computing, the consideration would be to consider the features of the set of security recommendations provided in the context of NIST Guidance documents as given below: • Consider set of security recommendation for the deployment of the foundational component of the Server Virtualization technology – i.e., the Hypervisor • Security suggestions for all components in a real-world virtualization infrastructure built around the Hypervisor deployment such as VM Images and Virtual Network.

75 Using NIST 800-125-A to understand hypervisor security threats Paul Henry. SANS Institute, 2017 http://searchcloudsecurity.techtarget.com/tip/Using-NIST-800-125-A-to-understand-hypervisorsecurity-threats



51

CLOUD HYPERVISOR AND SECURITY • Recognizing that Hypervisor being a piece of server software, all the generic security considerations have been included as part of the suggestions. These include: (a) Keeping Patches up to date, protection of the execution instance using host-based Firewalls and IDPS, Analysis of Logs etc. • Recognizing that the Hypervisor is nothing but an OS kernel and that the security of a server system configured with any OS distribution depends upon some weakest links such as the device driver software, security suggestions relating to these components have been provided as well. • Recognizing that the hypervisor has to perform certain privileged operations without interference from any other entity in the virtualized host and that leveraging hardware support for these operations will make a big difference to the overall security of hypervisor deployment. • The ability to deliver the most important infrastructure and security technologies in virtual appliances such as the firewall and the network switches. • There has been no known successful attack on hypervisors except for the theoretical ones, which requires one to have an access to the hypervisors source codes and the ability to implement them. • Last but not the least, all security suggestions are traceable to one or more the hypervisor’s baseline functionality, the secure execution of whose constituent tasks, the security suggestions in this document are intended to provide protection. Applying various measures, which have been discussed in this paper, can curb many of the security threats that affect virtualized environments. But, due to increased use of virtualization in many areas such as the clouds, data centers and other areas that continue emerging with increased use of technology presents new kinds of vulnerabilities and more measures need to continue evolving. Segmentation using Virtual Switches and FW Any solution built on virtualization infrastructure is aimed at providing a solution to various security vulnerabilities through the creation of secure gateways for any virtualization infrastructure. The solutions described below are relevant more so when protecting data centers and cloud infrastructure throughout the construction process. Security on virtual layer Security on the virtual layer can be established securing the manner in which the VMs and the hypervisors communicate with the rest of the virtual network. So as to take a complete advantage this kind of infrastructure, it is advised that virtual private network (VPNs) should be established to enable management of different levels of authority between the VMs. Due to the virtual nature of these kinds of networks, some of the features like monitoring, gaining access to the controls, integrity, transportability, encryption as well as authenticity in the VMs can be executed directly into this network. In so doing, most of the vulnerabilities within the virtualized environment pertaining security of the virtual layers isolate the different virtualized management networks. Additionally, they bring about easiness in deploying and in the operations of the VMs across most of the data centers.



52

CLOUD HYPERVISOR AND SECURITY Security on physical layer The security of the physical layer concerns all the activities related to the design of substructure of various physical systems that result into beefing up security in the virtualized environment. Generally, the security in this area is achieved by putting features such as host-instituted intrusion detection as well as prevention. The features ensure that the physical layer is never compromised by any other means. In spite of that, the structure of the data center or the cloud also plays vital roles. The manner in which the machines that run the VMs have been interconnected physically determines the kind of security measures to be put in place. Other factors that influence the manner in which the security of a virtualized environment is include the routine inspection of the hardware failures of any outdated system.



53

CLOUD HYPERVISOR AND SECURITY REFERENCE CITED [1] Souppaya, M., Scarfone, K. (2013, June). NIST SP 800-124, Revision 1: Guidelines for Managing the Security of Mobile Devices in the Enterprise. Available from: http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-124r1.pdf [2] Franklin, J. et al. (2015, November). NIST SP 1800-4b (DRAFT), Mobile Device Security – Cloud and Hybrid Builds: Approach, Architecture, and Security Characteristics. Available from: https://nccoe.nist.gov/sites/default/files/library/sp1800/mds-nist-sp1800-4b-draft.pdf [3] Bhatele K. R., Bhatele M. Dr., Arjaria A., & Pahade A. (2013, October 3). IJMEMR Publication: A survey over Virtualization and Hypervisor. Available from: http://ijmemr.org/Publication/V1I3/IJMEMR-V1I3-008.pdf [4] Peterson, G., (n.d.). ArtecGroup: Security Architecture Blueprint. Available from: http://arctecgroup.net/pdf/ArctecSecurityArchitectureBlueprint.pdf [5] NISTIR 7977 (2016, March). NIST Interagency Report: NIST Cryptographic Standards and Guidelines Development Process. Available from: http://nvlpubs.nist.gov/nistpubs/ir/2016/NIST.IR.7977.pdf [6] Joint Task Force Transformation Initiative (2011, March). NIST: Managing Information Security Risk: Organization, Mission, and Information System View. Available from: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-39.pdf [7] Rhodes, K. A., (2003, September 9). US Government Accountability Office (GAO): Information Security: Challenges in Using Biometrics. Available from: http://www.gao.gov/assets/120/110297.pdf [8] Radack, S. (2011, April). National Institute of Standards and Technology; ITL Bulletin: Full Virtualization Technologies: Guidelines Implementation and Management. Available from: https://csrc.nist.gov/csrc/media/publications/shared/documents/itl-bulletin/itlbul2011-04.pdf [9] Scarfone, K., Souppaya M., Hoffman, P. (2011, January). NIST Special Publication 800-125: Guide to Security For full Virtualization Technologies. Available from: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-125.pdf [10] Chandramouli, R. (2017, September). IEEE PES PSCC: NIST SP 800-125A Draft on Hypervisor Security Open for Comments. Available from: http://sites.ieee.org/pes-pscc/nist-sp-800-125adraft-on-hypervisor-security-open-for-comments/ [11] VMware Corporation. (2012, November). WHITE PAPER - Controlling Virtual Machine Sprawl: How to better Utilize Virtual Infrastructure. Available from: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmwarecontrolling-virtual-machine-sprawl-white-paper.pdf [12] Chandramouli, R. (2016, March). NIST SP 800-125B. Secure Virtual Network Configuration for Virtual Machine (VM) Protection. Available from: http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-125B.pdf



54

CLOUD HYPERVISOR AND SECURITY [13] Cisco System. (2017, April 26). WHITE PAPER – Cisco Virtual Topology System: Data Center Automation Next-Generation Cloud Architectures. Available from: https://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/virtualtopology-system/white-paper-c11-734904.html [14] Davie, B., Gross, J. (2012, March). Nicira Networks, Inc.: A Stateless Transport Tunneling Protocol for Network Virtualization (STT). Available from: https://tools.ietf.org/html/draft-daviestt-01 [15] Scarfone, K., Soupaya M., Hoffman, P. (2011, April). NIST SP-800-125: Guide to Security for Full Virtualization Technologies. Available from: https://csrc.nist.gov/publications/detail/sp/800125/final [16] Murphy, A. (2006, September). SANS Institute InfoSec Reading Room: Security Implications of the Virtualized Data Center. Available from: https://www.sans.org/readingroom/whitepapers/sysadmin/security-implications-virtualized-data-center-1796 [17] Jorgenson, P. (2012, May 30). PLURALSIGHT: Virtual Networking 101: Understanding VMware Networking. Available from: https://www.pluralsight.com/blog/it-ops/virtualnetworking-101-understanding-vmware-networking [18] Scarfone, K., Souppaya, M., & Hoffman, P. (2011, January). NIST SP 800-125: Guide to Security for Full Virtualization Technologies. Available from: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-125.pdf [19] Westby J. R., & Allen J. H., (August 2007). Software Engineering Institute: Governing for Enterprise Security (GES) Implementation Guide. Available from: https://resources.sei.cmu.edu/asset_files/TechnicalNote/2007_004_001_14837.pdf [20] Chandramouli, R. (2014, October). Draft NIST Special Publication 800-125-A: Security Recommendations for Hypervisor Deployment. Available from: http://csrc.nist.gov/publications/drafts/800-125a/sp800-125a_draft.pdf [21] Xen.org Community (n.d.). Xen Hypervisor is developed and maintained by the Xen.org community and available as a free solution under the GNU General Public License. For more information, see http://www.xen.org/ [22] Hendrick, J. (2014). The SANS Institute InfoSec Reading Room: Security Visibility in the Enterprise. Available from: https://www.sans.org/readingroom/whitepapers/projectmanagement/security-visibility-enterprise-35442 [23] ISTR (2016, April 21). Symantec: Internet Security Threat Report. Available from: https://www.symantec.com/content/dam/symantec/docs/reports/istr-21-2016-en.pdf [24] Broad, J. (2013). Risk Management Framework: A Lab-Based Approach to Securing Information Systems 1st Edition. Integrated Organization-Wide Risk Management: Risk Management and the RMF (Chap. 3, pp 24).



55

CLOUD HYPERVISOR AND SECURITY [25] Wueest, C., Barcena M. B., O’Brien L. (2015, May 1). Symantec Security Response: Mistakes in the IaaS Cloud could put your data at risk. Available from: http://www.symantec.com/content/en/us/enterprise/media/security_response/whitepapers/m istakes-in-the-iaas-cloud-could-put-your-data-at-risk.pdf [26] Groesbrink, S., Almeida, L., de Sousa, M., & Petters, S. M. (2014, April). Towards certifiable adaptive reservations for hypervisor-based virtualization. In Real-Time and Embedded Technology and Applications Symposium (RTAS), 2014 IEEE 20th (pp. 13-24). IEEE. [27] Bharadwaja, S., Sun W., Niamat M., & Shen F. (2011, April). Collabra: a Xen hypervisor based Collaborative Intrusion Detection System. Available from: New Generations (ITNG), 2011 eighth international conference on (pp. 695-700) IEEE [28] Sabahi, F. (2012). Secure Virtualization for Cloud Environment using Hypervisor-Based Technology. Available from: International Journal of Machine Learning and Computing, 2(1), 39. [29] Xavier, M. G., Neves, M. V., Rossi, F. D., Ferreto, T. C., Lange, T., De Rose, C. A., el al (2013, February). Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments. In Parallel, Distributed and Network-Based Processing (PDP), 2013 21stEuromicro International Conference (pp. 233-240). IEEE [30] Chandramouli, R. (2017, September). NIST SP 800-125A DRAFT (2nd): Security Recommendations for Hypervisor Deployment. Available from: https://csrc.nist.gov/CSRC/media/Publications/sp/800-125a/draft/documents/sp800-125Adraft2.pdf [31] Chandramouli, R. (2016, March). NIST SP 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Protection. Available from: http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-125B.pdf [32] Murphy, M. (2006, September 16). SANS Institute InfoSec Reading Room: Security Implications of the Virtualized Data Center. Available from: https://www.sans.org/readingroom/whitepapers/sysadmin/security-implications-virtualized-data-center-1796 [33] Chandramouli, R. (2016, March). NIST SP 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Protection. Available from: http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-125B.pdf [34] DHS Publication (2016, May 11). The President’s National Security Telecommunications Advisory Committee: NSTAC Report to the President on Big Data Analytics: Available from: https://www.dhs.gov/sites/default/files/publications/NSTAC%20Report%20to%20the%20Presid ent%20on%20Big%20Data%20Analytics%20%285-11-16%29-%20508%20compliant.pdf [35] DHS Publication (2013, May). The Department of Homeland Security’s (DHS) National Protection and Programs Directorate (NPPD)? Cyber Risk Culture Roundtable Readout Report. Available from: https://www.dhs.gov/sites/default/files/publications/cyber-risk-cultureroundtable-readout_0.pdf



56

CLOUD HYPERVISOR AND SECURITY [36] DHS Publication (2014, November 19). The President’s National Security Telecommunications Advisory Committee (NSTAC) reported to the President on the Internet of Things, to examine the cybersecurity implications of the IoT within the context of national security and emergency preparedness (NS/EP). Available from: https://www.hsdl.org/?view&did=789743 [37] OIG (2016, August). Office of Inspector General Unclassified: Information Report: Description of Policies and Computer Security Controls for Select Broadcasting Board of Governors Covered Systems. Available from: https://oig.state.gov/system/files/aud-it-ib-1644.pdf [38] CSRC Menu (2017, March 31). NIST Information Technology Laboratory: Computer Security Resource Center: Status of NIST SP 800-53, Revision 5. Available from: https://beta.csrc.nist.gov/News/2017/Status-of-NIST-SP-800-53,-Revision-5 [39] Zheng, M., & Prof. Jain R., (2011, November 27). Washington University in St. Louis: Virtualization Security in Data Centers and Clouds. Available from: http://www.cse.wustl.edu/~jain/cse571-11/ftp/virtual.pdf [40] DISA Unclassified (2011, May 18). Defense Information Systems Agency: Identity Management Division: Attribute Based Access Control (ABAC) Engineering Bluespring v. 2.0. Available from: https://community.apan.org/.../AbacUnclass_5F00_engineering_5F00_blueprint_5F0 [41] HLdata Protection (2015, March 3). Cyber Physical Systems Public Working Group: Preliminary Discussion Draft Framework for Cyber-Physical Systems. Available from: http://www.hldataprotection.com/files/2015/03/NIST-Cyber-physical-FrameworkPRELIMINARY-DISCUSSION-DRAFT.pdf



57

CLOUD HYPERVISOR AND SECURITY ANNOTED GLOSSARY Selected terms used in the paper are defined below. • Hypervisor security76 = the process of ensuring the hypervisor, the software that enables virtualization, is secure throughout its life cycle, including during development, implementation, provisioning, management and de-provisioning. • Layered security, also known as layered defense77 = describes the practice of combining multiple mitigating security controls to protect resources and data. • Standards-Based Security78 = a series of new security standards (ISO/ITU) allow security professionals to talk a common language. For instance, companies are able to prove and show the level of maturity of their security solutions based on their proven compliance of the recommendations defined by the standard. • “Trustworthy” and “assurance.”79 = a cyberspace environment that provides a user with confidence in its security, using automated mechanisms to ascertain security conditions and adjust the level of security based on the user's context and in the face of an evolving range of threats. • Confidentiality, Integrity and Availability80 = also known as the three components of CIA triad, which is a model designed to guide policies for information security within an organization. The CIA triad is a very fundamental concept in security. Often, ensuring that the three facets of the CIA triad is protected is an important step in designing any secure system • Biometric Verification81 = biometric verification is basically verifying that you are who you say you are. If you press your fingerprint on the scanner of verification devices, the device has to access the database of the fingerprints and determine that your biometrics pattern is actually authorized to open whatever it is that you are trying to access. Notably, a biometric verification is not the same as biometric identification; the latter ascertains the identification of a person. Unique verifications include fingerprints, hand geometry, earlobe geometry, retina and iris patterns, voice waves, DNA, and signatures. The oldest form of biometric verification is fingerprinting. • Access Control82 = access control is a security technique that can be used to regulate who or what can view or use resources in a computing environment. There are two main types of access control: physical and logical. Physical access control limits access to campuses, buildings, rooms and physical IT assets. Logical access limits connections to computer networks, system files and data. • Internet of Things (IoT)83 = the Internet of Things (IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that

76 http://searchcloudsecurity.techtarget.com/definition/hypervisor-security 77 https://en.wikipedia.org/wiki/Layered_security 78 http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470745754.html 79 https://definedterm.com/tailored_trustworthy_space 80 http://security.blogoverflow.com/2012/08/confidentiality-integrity-availabilitythe-three-components-of-the-cia-triad/ 81 http://www.biometric-security-devices.com/biometric-verification.html 82 http://searchsecurity.techtarget.com/definition/access-control 83 http://internetofthingsagenda.techtarget.com/definition/Internet-of-Things-IoT

58

CLOUD HYPERVISOR AND SECURITY

•

•

•

•

are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. Hacker84 = a hacker is an individual who uses computer, networking or other skills to overcome a technical problem. The term hacker may refer to anyone with technical skills, but it often refers to a person who uses his or her abilities to gain unauthorized access to systems or networks in order to commit crimes. A hacker may for example, steal information to hurt people via identity theft, damage or bring down systems and, often, hold those systems hostage to collect ransom. Intrusion Detection System (IDS) and Intrusion Prevention System (IPS)85 = IDS (Intrusion Detection System) and IPS (Intrusion Prevention System) both increase the security level of networks, monitoring traffic and inspecting and scanning packets for suspicious data. Detection in both systems is mainly based on signatures already detected and recognized. The main difference between one system and the other is the action they take when an attack is detected in its initial phases (network scanning and port scanning). o The Intrusion Detection System (IDS) provides the network with a level of preventive security against any suspicious activity. The IDS achieves this objective through early warnings aimed at systems administrators. However, unlike IPS, it is not designed to block attacks. o An Intrusion Prevention System (IPS) is a device that controls access to IT networks in order to protect systems from attack and abuse. It is designed to inspect attack data and take the corresponding action, blocking it as it is developing and before it succeeds, creating a series of rules in the corporate firewall, for example. Cloud Service Providers86 = a cloud provider is a company that offers some component of cloud computing, typically Infrastructure as a Service (IaaS), Software as a Service (SaaS) or Platform as a Service (PaaS), as well as, to other businesses or individuals. Cloud providers are sometimes referred to as cloud service providers or CSPs. Data Leakage87 = is the unauthorized transfer of classified information from a computer or datacenter to the outside world, which can be accomplished by simply mentally remembering what was seen, by physical removal of tapes, disks and reports or by subtle means such as data hiding. Silos88 = a silo in IT is an isolated point in a system where data is kept and segregated from other parts of the architecture. IT professionals often talk about silos in a negative way, because the free flow of data is so important in most enterprise systems.



84 http://searchsecurity.techtarget.com/definition/hacker 85 http://www.pandasecurity.com/usa/support/card?Id=31463 86 http://searchcloudprovider.techtarget.com/definition/cloud-provider 87 https://www.pcmag.com/encyclopedia/term/61834/data-leakage 88 https://www.techopedia.com/definition/25939/silo

59

CLOUD HYPERVISOR AND SECURITY Appendix: CONTEXT: Selected NIST Issues We present a few observations regarding our chosen enterprise security program protection measures in the context of virtualized infrastructure are in order. In a virtualized infrastructure, the distinguishing networking environment is the virtual network. Hence, the network segmentation89 approaches have to involve some virtual network components such as a hypervisor and virtual switches. Similarly, a viable approach for traffic control90 based on architectural options and based on configuration choices should form part of enterprise security architecture issues for a tiered risk management approach that pertain to standards-based security. The architectural context assists the enterprise in assessing and managing risk. Tiered risk management as presented in NIST guidance is a key aspect of managing enterprise risk [34]. Therefore, the authoritative context elaborated in this paper focuses uniquely architectural issues of NIST tiered risk management approach for cloud hypervisor hosts. The three NIST tiers, which are represented as a three-tiered risk management approach, are NIST Risk Management Tiers 1-3 which include both strategic and tactical issues:91 1. NIST Risk Management Tier 1: NIST Risk Management Tier 1 includes Organization, Enterprise Architecture (EA), Enterprise Security Architecture, Big Data, Cloud Computing, and Internet of Things (IoT). It addresses risk from an organizational perspective with the development of a comprehensive governance structure and organization-wide risk management strategy that includes; the techniques and methodologies, the methods and procedures, the types and extent of risk mitigation measures, the level of risk (i.e., tolerance), methods of monitoring, degree and type of oversight. 2. NIST Risk Management Tier 2: NIST Risk Management Tier 2 includes, Mission/Business Processes, Trust and enterprise systems security engineering. This tier addresses risk from a mission and business process perspective, which is guided by the risk, decisions at Tier1. Some of the risk decisions included are activities that are closely associated with enterprise architecture that defines the core missions and business processes, prioritizing missions and business processes. 3. NIST Risk Management Tier 3: Information Systems: NIST Risk Management Framework; and Network Security as depicted in Figure 19. It addresses risk from an information system perspective and is guided by the risk decisions at Tiers 1 and 2, including the impact of the ultimate selection and deployment of needed safeguards and countermeasures (i.e., security controls) at the information system level. Information security requirements are satisfied by the selection of appropriate management, operational, and technical security controls from NIST Special Publication 800-53. Risk management tasks begin early in the system development life cycle and are important in shaping the security capabilities of the information system [35]. If these tasks are not

89 VMware Docs: Network segmentation splits a computer network into subnetworks, each being a network segment, boosted performance and improving security. 31 August 2016. 90 NIST SP 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Protection. March 29, 2016. 91 NIST SP 800-39: Managing Information Security Risk. March 2011

60

CLOUD HYPERVISOR AND SECURITY adequately performed during the initiation, development, and acquisition phases of the system development life cycle, the tasks will, by necessity, be undertaken later in the life cycle and be more costly to implement. In either situation, all tasks are completed prior to placing the information system into operation or continuing its operation to ensure that; information system-related security risk is being addressed, the authorizing official explicitly understands and accept the risks.



92

Figure 19: Framework Core Structures

Systematically, the Risk Management Framework provides a process that integrates security and risk management activities into the system development life cycle. The riskbased approach to security control selection and specification considers effectiveness, efficiency, and constraints due to applicable laws, directives, Executive Orders, policies, standards, or regulations. Showing in Table 4 below is the risk-based approach of the risk management framework (RMF).

92 NIST Framework for Improving Critical Infrastructure Cybersecurity, Version 1.0. February 12, 2014.

61

CLOUD HYPERVISOR AND SECURITY



93

Table 4: Management Organization Risk for RMF

The Risk Management Framework (RMF), illustrated in Figure 19, provides a disciplined and structured process that integrates information security and risk management activities into the system development life cycle. The RMF operates primarily at Tier 3 in the risk management hierarchy but can also have interactions at Tiers 1 and 2 like providing feedback from ongoing authorization decisions or updated threat and risk information to up to management. Candidate Layered Security Architecture The enterprise security architecture should provide confidentiality, integrity, and availability throughout the enterprise and align with the objectives of the organization. Recognizing this, in February 2015, the Executive Office of the President (EOP)94, specifically the National Security Council, tasked the President’s National Security Telecommunications Advisory Committee (NSTAC) to examine the cybersecurity implications of the Iota within the context of national security and emergency preparedness (NS/EP) [36]. The NSTAC found that Iota adoption will increase in both speed and scope, and that it will affect virtually all sectors of our society. The Nation’s challenge is ensuring that the IoT’s adoption does not create undue risk. Additionally, the NSTAC determined that there is a small and rapidly closing window to ensure that IoT is adopted in a way that maximizes security and minimizes risk. If the country fails to do so, it will be coping with the consequences for generations [37].

93 NIST Framework for Improving Critical Infrastructure Cybersecurity, Version 1.0. February 12, 2014. 94 EOP of Federal Research and Development Funding FY February 2015. Source: https://fas.org/sgp/crs/misc/R43580.pdf

62

CLOUD HYPERVISOR AND SECURITY NIST Guidance for Cloud Computing guidelines assist the architectural aspect of managing enterprise risks, which is depicted in Table 3 and below, is the authoritative documentation: • Guidelines on Security and Privacy in Public Cloud Computing (NIST SP 800145) • Cloud Computing Reference Architecture on Security (NIST SP 500-292) • US Government Cloud Computing Technology Roadmap (NIST SP 500-293) • NIST Cloud Computing Referenced Architecture (NIST SP 500-291) • Cloud computing Synopsis and Recommendations (NIST SP 800-146) Malicious attacks to the main hypervisor can compromise other VMs and the whole system. For example, attacks include injecting malicious code or triggering a bug in the hypervisor by a malicious VM. An attacker may also try to reach and compromise operating systems in intermediate layers of the recursive virtualization stack95. Attacks may eventually affect integrity and availability of the described multi-layer architecture. Keep in mind that we are talking about malicious attacks, which only target the software part to the architectural structure, then, we are not considering hardware security. In fact, this paper assumes that malicious code cannot tamper with hardware etc. Consequently, the ultimate goal is to increase the protection of the main hypervisor and other user VMs, against attacks originating from malicious attackers, and interposing layers of defensive VMs between the users of VMs layer and the hypervisor does this

96

Table 5: candidate Enterprise Layered Security Architecture





95 Defense-in-depth is the security technique that place diverse and successive defending layers and mechanism in the way of the adversary, which makes virtualization environments much more resilient to attacks. In Proceedings of the 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks Workshops, DSNW ’11, pages 117–122, 2011 96 Harold Podell, Johns Hopkins University: Enterprise Security Architecture: Selected: Standards-Based Security Issues: Research Paper and Systems Risk Analysis Guidance, v3.3, August17, 2017.



63

CLOUD HYPERVISOR AND SECURITY



97

Figure 20: FIPS 199 Risk/Impact Profiles

The SP 800-53 Rev4 provides a folder of security and privacy controls for federal information systems and organizations as shown in Figure 21. Furthermore, it processes for selecting controls to protect all type of organizational operations, including diverse set of threats, even human errors (both intentional and unintentional). The security and privacy controls are customizable and implemented as part of an organization-wide process that manages information security and privacy risk. The controls address selected security and privacy requirements across the federal government and critical infrastructure, derived from legislation, Executive Orders (EOs), policies, directives, regulations, standards, and/or mission/business needs. SP 800-53 also describes how to develop specialized sets of controls, or overlays, tailored for specific types of missions/business functions, technologies, or environments of operation. Finally, the folder of security controls addresses security from both a functionality perspective (the strength of security functions and mechanisms provided) and an assurance perspective (the measures of confidence in the implemented security capability). Addressing both security functionality and assurance helps to ensure that information technology component products and the information systems built from those products using sound system and security-engineering principles are sufficiently trustworthy.

97 FIPS PUB 199: Standards for Security Categorization of Federal Information and Information Systems.

February 2004. Also important to know that risk from authentication error is a function of two factors: (a) potential harm or impact and (b) the likelihood of such harm or impact: https://csrc.nist.gov/csrc/media/publications/fips/199/final/documents/fips-pub-199-final.pdf



64

CLOUD HYPERVISOR AND SECURITY



98

Figure 21: NIST SP 800-53: Finding the right fit in IT Security Frameworks

The essential two statements to register here are “trustworthy” and “assurance.” It is important to keep the information systems trustworthy, then use sound assurance practices to monitor, test, and make corrections. Important to realize, NIST 800-53 uses a “Risk Management Framework” (RMF) methodology as shown in Figure 19, the RMF same cycle. This methodology implements and continually monitors the information system from a risk perspective. Revision 5 of SP 800-53 plans on including significant changes to make the controls more consumable by diverse groups including, for example, enterprises conducting mission and business operations [38]: • Making the security and privacy controls more outcome-based by changing the structure of the controls; • Fully integrating the privacy controls into the security control folder creating a consolidated and unified set of controls for systems and organizations; • Separating the control selection process from the actual controls, thus allowing the controls to be used by different communities of interest including systems engineers, software developers, enterprise architects; and mission/business owners; • Eliminating the term information system and replacing it with the term system so the controls can be applied to any type of system including, for example, general purpose systems, cyber-physical systems, industrial/process control systems, and IoT devices;

98 NIST SP 800-53 revision 4: Security and Privacy Controls for Federal Information Systems and Organizations. January 22, 2015.

65

CLOUD HYPERVISOR AND SECURITY • Deemphasizing the federal focus of the publication to encourage greater use by non-federal organizations; • Promoting integration with different risk management and cybersecurity approaches and lexicons, including the Cybersecurity Framework; • Clarifying the relationship between security and privacy to improve the selection of controls necessary to address the full scope of security and privacy risks; and • Incorporating new, state of the practice controls based on threat intelligence and empirical attack data, including controls to strengthen cybersecurity and privacy governance and accountability…" Interestingly, the focus is on the three-tiered risk management approach and to measure the advantages and disadvantages of each theoretical defense mechanism on the architecture and infrastructure as described above. For any type of datacenter infrastructure (virtualized or non-virtualized), there is a consensus that the following are some of the key Network-level protection measures [39]. They are: a. Network segmentation or isolation b. Traffic control using firewalls c. Creating redundant communication pathways, and d. Traffic Monitoring and Prevention using IDS/IPS. Out of the above four networklevel protection measures, the first two networks segmentation and traffic control using firewalls form the foundation for the network configuration of the entire virtualized infrastructure. Hence, in this paper, we have chosen to focus on different approaches or mechanisms used for these two network-level protection measures, by performing a detailed analysis of the advantages and disadvantages of each of the approaches. There are a couple of ways in which such risks can be mitigated. Using the three-key NIST Systems Security Guidance Documents, supported by the Integration of Guidance Silos, which also provide an integration of selected guidance silos from other standards developing organizations (SDOs). In particular, these three NIST documents respectively map to other standards provides as follows: 1. An integration of security controls guidance for the enterprise integration of guidance silos; 2. Reduces the need to focus on security controls from other standards organizations, which makes it less reliance on guidance silos. As a result, the three-key NIST document are listed below: 1. NIST: Framework for Improving Critical Infrastructure Cybersecurity, Version 1.0, February 12, 2014, as shown in Table 1: Framework Core [Mapping NIST Cybersecurity Framework functions to NIST SP 800-53, Rev. 4, COBIT (ICSA (International Computer Security Association): Control Objectives for Information and Related Technologies), and ISO/IEC 27001: 2013 Security techniques controls] 2. NIST: Special Publication 800-53, Rev. 4, Security and Privacy Controls for Federal Information Systems and Organizations, August 2013, includes updates as of January 22, 2015, Appendix H: International Information Security Standards:



66

CLOUD HYPERVISOR AND SECURITY Security Control99. Mappings for ISO/IEC 27001 and 15408 [Common Criteria] Table H-1: Mapping SP 800-53 to ISO/IEC 27001 [security controls]. 3. NIST: Systems Security Engineering: Considerations for a Multidisciplinary Approach in the Engineering of Trustworthy Secure Systems, September 2016. This publication contains a set of systems security engineering process extensions for systems and software engineering, and system life cycle processes. 4. Please note that the ISO 27001 as indicated is mapping and formalized in a few versions of selected NIST security guidance documents. For example, mapping of NIST Cybersecurity Guidance to ISO 27001, and COBIT100. Regarding blueprint for the assessment of enterprise in consideration of attribute based access control (ABAC) issues; it recommended selecting the three-layered security architecture, because this architecture provides a metric when assessing ABAC issues, particularly for cloud security. Surprisingly, this layered security architecture approach is not a formal NIST architecture but rather affiliated to defense information systems agency (DISA). The DISA developed the ABAC Engineering Blueprint, uniquely for the Engineering Blueprint addresses dynamic access control concepts, requirements, functionality, and logical architecture with an acute focus on issues pertaining to making decisions and enforcement, and their interfaces for policy and attribute retrieval [40]. Going back to virtualization and cloud computing, particularly the framework for cyberphysical system (CPS), which addresses the need for cross-property risk analysis, such that, we talk about how critical it is important risk management planning and operations be conducted holistically, rather than discipline-specific silos. As technology moving at a rapid pace, the new CPS will provide the next generation of “smart,” co-engineered interacting components connected over diverse networks. Assuring that these systems are trustworthy in the broadest sense (e.g., reliable, resilient, secure, private and safe) but for sure, it poses unique cybersecurity challenges. Traditional approaches to cybersecurity, privacy, reliability, resilience, and safety may not be sufficient to address the risks to CPS. This produces a need for a cross-property risk management approach that leverages and extends the risk management approaches from historically disparate areas of expertise [41]. A remark to make is that, to support the co-design aspect of CPS, a deeper understanding of the relative significance of, and interactions among, these properties is necessary to ensure the functionality of the CPS is not compromised, such that a system produces unintended outcomes. This cross-property understanding will enable appropriate CPS design trade-offs and complementary cross-property design decisions. Coming from the ideology of building a trustworthy systems, it has been said by F.B. Schneider; Cornell Univ., Ithaca, NY during the IEEE Security & Privacy (2007, Volume 5, Issue: 5 pp. 3-4) that: “the system does what is required despite environmental disruption, human user and operator error, and attacks by hostile parties and not other things”. Consequently, to achieve trustworthiness of a system is greater than the sum of trustworthy parts. The context of CPS is related to the layered security architecture to approximate five-layered

99 The Common Criteria is the driving force for the widest available mutual recognition of

secure IT products: https://www.commoncriteriaportal.org/ 100 The International Computer Security Association (ICSA) and the Control Objectives for

Information and Related Technologies are leading the Framework for the governance and management of IT: http://www.isaca.org/cobit/pages/default.aspx



67

CLOUD HYPERVISOR AND SECURITY security architecture when assessing big data, and Iota architecture and functions with respect to authoritative guidance, below is references use cases: • NIST Special Publication 1500-7: NIST Big Data Interoperability Framework: Volume 7, Standards Roadmap Final Version 1, September 2015;http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1500-5.pdf • NIST: Framework for Cyber-Physical System, Release 1.0, May 2016, Section B.4.3.3: The need for Cross-Property Risk Analysis for CPS, Figure 23: Physical, Analog and Cyber components of CPS. • NIST Framework for Improving Critical Infrastructure Cybersecurity V.1, February 12, 2014 [NIST Cybersecurity Framework]; http://www.nist.gov/cyberframework/upload/cybersecurity-framework021214.pdf • NIST: Framework for Cyber-Physical System, Release 1.0, May 2016, Section B.4.3.3: The need for Cross-Property Risk Analysis for CPS, Figure 23: Physical, Analog and Cyber components of CPS. • NIST Special Publication 800-37, Rev 2: Draft: Risk Management (RMF) for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy, September 28, 2017. NIST Framework for Improving Critical Infrastructure







Figure 22: Cybersecurity Framework Components

101



The National Institute of Standards and Technology (NIST) came up with a framework of improving the structure of Cyber security also known as the Cyber security Framework. This framework provides new details that keep track of Cyber Supply Chain Risks, clarification of important terms and finally introducing measurement method required in cyber security. The core reason of updating the framework is to develop the NIST’s voluntary guidance that is aimed at guiding and minimizing cyber security risks in an organization. This framework was later published in February 2014 due to the collaborative processes that involved the government agencies, academic institutions and industries following a directive from the president. The main aim was to come up with a voluntary framework that could help an organization in managing cyber security risks in

101 Framework for Improving Critical Infrastructure Cybersecurity: Improving Critical Infrastructure Cybersecurity January 2016

68

CLOUD HYPERVISOR AND SECURITY the nation’s Critical Infrastructure such as the electric power grid and the bridges. This framework has also been employed in various countries across the world. Modern systems such as CPS include physical, cyber elements and analogue. Cyber components usually proliferate due to their favorable combination of cost, lifecycles, supportability, capability and flexibility102 as shown in Figure 23. The use of cyber components more so in CPS that requires higher resilience, safety and reliability may pose a serious security challenge. Unlike the physical and analogue components that can be subjected to vigorous test with their results being generated through direct observation, the most probable behaviour of cyber component quite exorbitant while testing various configurations and the operating conditions subjected to them. This implies that the cyber components are more likely to consume huge share of the overall system budget than either the physical or the analogue component. Safety and Analysis of Cloud Hypervisor security and concept of System Risk Privacy also poses a serious challenge as the field lacks the necessary technology to the privacy risk and the objectives, which can propagate system designs, and curbing risks involved in cloud computing. Various organizations have been using principle-basedmechanisms such as Fact Information Practice Principe and Privacy Designs to solve the problem of privacy information systems. The principle allows an organization to put into consideration various aspects of handling private information. CPS is mainly used to achieve the required behaviour through an appropriate allocation of requirements in every element through co-designing process.







Figure 23: Physical, Analog, and Cyber Components of CPS100103

NIST: Cybersecurity Framework Feedback The second conceptual will depend on the guidance below to provide a suitable standard of measurement to examine the cyber security risk management for any mobile enterprise.

102 Hwang, J. Y. et al. (2008, January). Xen on ARM: System virtualization using Xen hypervisor for ARMbased secure mobile phones. In Consumer Communications and Networking Conference, 2008. CCNC 2008. 5th IEEE (pp. 257-261). IEEE. 103 Harold Podell, Johns Hopkins University: Enterprise Security Architecture: Selected: StandardsBased Security Issues: Research Paper and Systems Risk Analysis Guidance, v3.3, August 17, 2017



69

CLOUD HYPERVISOR AND SECURITY In Feb. 2014, NITS published the guidelines for improving Critical Infrastructure Cybersecurity known as the NIST Cybersecurity Framework. The framework is designed in a manner that complements and organizes an institution’s already existing risks and security management programs and practices. The NIST’s Cybersecurity framework is aligned with security and privacy standard. An institution is capable of linking the already existing approaches to the framework’s main functions such as Identify, Protect, Detect, Respond and Recover. Users of NIST Cybersecurity Framework NIST Cybersecurity Framework is available in the public domain for download and free use by government institutions and industries. When it was first published in the year 2014, the main object of this framework was to operate national critical infrastructures and since then the framework has been referenced by various business and organizations. Updates to the NIST Cybersecurity Framework The last time the NIST Cybersecurity Framework was updated was on 22nd of June 2017. On 10th of January 2017, NIST issued a drafted version of Cyber security Framework. The draft version 1.1 provided new and up to date details of managing Cyber Supply Chain Risks; clarify new terms and introducing standard methods of measurements for cybersecurity. On 11th of May 2017, the presidential executive order the strengthening of cybersecurity of the federal networks and the Critical Infrastructure was state requiring the United States Federal Agencies to used the NIST cybersecurity framework to demonstrate and establish the risk and security management practices. On 12th of May 2017 NIST announced to have come up with a drat NISTIR8170; this was the cybersecurity Implementation Guidance for Federal Agencies; following the presidential executive orders on Cybersecurity. This draft was used as companion guidance in explaining eight use cases for government Application of Cybersecurity Framework. CISQ’s Statements The Consortium for IT Software Quality (CISQ) supports the NIST’s efforts in the development of Cybersecurity Framework. CISQ has presented comments in an open review periods. CISQ has tremendously contributed to the NIST Cybersecurity Framework are automatic source code standard that measures the size and the structure of a software. The Automatic source code metrics make it possible to measure the reliability and the security of the software is certain regular interval at every release cycle. The updates in version 1.1 of the NIST Cybersecurity Framework enhances the following • Formal agreement of the baseline requirements for partners and supplies • It monitors the Cyber risks that are similar to operation risks or the financial risks • Promote metric measurement



70