Security Risk Metrics: Fusing Enterprise Objectives and ... - CiteSeerX

Proceedings of the 2005 IEEE Workshop on Information Assurance and Security United States Military Academy, West Point, NY

Security Risk Metrics: Fusing Enterprise Objectives and Vulnerabilities K. Clark, J. Dawkins, J. Hale

Abstract— Automated scanners are unable to generate the information required to properly assess a network’s risk. Although scanners may identify high risk exposures, they fail to determine how those exposures will affect an organization’s objectives. Such an assessment requires an auditor to identify the objectives and their relationship to network hosts. Mission Trees allow security auditors to map relationships between an organization’s objectives and its assets. Synthesizing this data with a vulnerability scanner lends itself to creating meaningful enterprise security metrics.

I. I NTRODUCTION Corporations rely on information technology to conduct business and transact payments. Weaknesses in this infrastructure have serious consequences on the organization it supports. A comprehensive understanding of these weaknesses and their effect on the organization mitigates this threat. Traditionally, this is accomplished through security assessments, network scans, and penetration testing [1], [2], [3], [4]. Unfortunately, conventional tools lack the capability to understand how changes in security funding and policy enforcement affect risk levels [5], [6], [7]. Moreover, they do not accurately reflect the relationship between IT assets and their importance to an organization. Information gleaned from these relationships can be used to maintain a proper balance between an asset’s value and the cost of a security control [8]. The most appropriate conduit for analyzing security spending is through risk metrics. A risk assessment produces risk metrics that summarize the current security state. If an assessment is conducted regularly, then changes in security funding will be reflected by the metrics they produce. The remainder of this paper is organized as follows: Section 2 provides background information regarding risk assessment methodologies. Section 3 elaborates the mission assessment process, while Section 4 illuminates the vulnerability assessment process. Section 5 presents an applied application of the risk assessment methodology conducted on an actual network, and Section 6 demonstrates risk mitigation. Finally, Section 7 presents a conclusion and outlines future work. II. BACKGROUND Although essential to risk management, little agreement exists concerning the generation and usefulness of security metrics. [9] states that information assurance metrics should clarify the relationship between assessments and secure software engineering metrics. Other research has focused on using

0-7803-9290-6/05/$20.00 ©2005 IEEE.

the Common Criteria as a benchmark for evaluating systems [10]. [11] proposes developing a metrics program stemming from several assessment methods, e.g. NIST Security Assessment Framework, and tracking the number of incidents over time versus changes in compliance with security requirements. However, [12], [13] believe that a reliable security metric is a chimera and propose other venues like software engineering metrics. On the other hand, automated network scanners have been developed to provide automated technical risk analysis and metrics [14], [15]. Penetration testing may also be used to provide technical risk analysis [4]. Generally, the majority of work in risk assessment metrics may be categorized as either policy-based or technical assessments. A. Policy-Based Assessments Conventionally, risk assessment techniques engage policy based metrics, most notably [16]. By only evaluating highlevel policies and procedures, these methodologies scale well for both small and large organizations and achieve an understanding of an organization’s priorities and goals. However, they do not assess technical flaws in security, but assess policy flaws in its implementation. As a result, they have little understanding of how technical flaws can impact an organization. While it is important to assess high-level policy, an adequate assessment methodology should also evaluate the technical aspects of security. Though an enterprise may have proper security policies and controls in place, it may contain significant technical vulnerabilities. Consequently, the policy based analysis may result in an acceptable security assessment when, in fact, the network may be quite susceptible to technical vulnerabilities. B. Technology-Based Assessments Technology-based assessment tools are capable of scanning for host information and known vulnerabilities, allowing automated security audits to locate vulnerabilities. Additionally, they provide basic qualitative analysis to a network by classifying vulnerabilities, e.g. high, medium, or low risk. Unfortunately, these tools have limited analytical capabilities. Since they are confined to analyzing technical attributes, they cannot understand the impact, with respect to an organization, of a vulnerability being exploited. For example, two hosts on a network may have identical vulnerabilities identified as high

388

Proceedings of the 2005 IEEE Workshop on Information Assurance and Security United States Military Academy, West Point, NY risk. If the assessment is limited to technical analysis, both of these hosts will have the same assessment. However, the organization may value the two hosts differently; one may be the primary webserver, and the other might be a rarely used test server. Within this context, the webserver should have a higher risk than the test server, even though they have the same vulnerabilities. C. Fusion Risk assessments can be improved by bridging the divide between policy and technology based assessments. Evaluating an organization’s purpose, policies, and assets supplements important information obtained by a vulnerability scanner. This data produces a risk metric representing both high-level policy and the technical aspects of an organization’s security. To meet this challenge, an enterprise risk assessment framework has been developed to model the relationship between IT assets and their contribution to an organization’s objectives. The framework allows an assessor to prioritize important objectives and associated assets into a mission tree. Vulnerabilities are used to identify threats existing on assets that lead to mission failure. The objectives a host accomplishes determine the impact of a vulnerability. Though two hosts may have the same vulnerabilities, a host charged with high priority objectives will be at greater risk than those with low priority objectives. The importance of the objectives the host accomplishes determines threat to the overall enterprise.

represented multiple times within the mission tree. Formally, a mission model is defined as M = {O ∈ Θ}, where O = {c ∈ A ∪ Θ}. Figure 1 shows an example mission tree. B. Mission Valuation The nodes within a mission tree are evaluated using a topdown approach. An organization’s mission is identified as the highest valued node and the objectives beneath it are assigned partitions of that value. In this case, it is easiest to think of objectives as having percentages, such that the mission is 100% of the value. The criticality of the objectives then determine the importance of their assets. In environments with many diverse hosts, such a process can decrease the sheer number of assets to assess by only investigating those which primarily contribute to a specific goal. Since not all objectives are equally critical, the nodes within a mission tree are weighted differently. The mission’s root is assigned the value 100, VM = 100. The objectives beneath it, o ∈ O, will intrinsically be less important, o ∈ O s.t. Vo ≤ VM . Thus, a representation of an asset’s rating is based upon the importance of the objectives it accomplishes. The more important the objective, the more important the assets that are used to complete it. In Figure 1, VObjective1 = 45, VObjective2 = 30, VObjective3 = 25 and so, VM = 100. Let Ov = value of parent node where cv ∈ O = value child node, and Ov = cv1 + cv2 + . . . cvn .

III. M ISSION M ODELING Mission modeling provides a management-oriented evaluation of an organization that seeks to understand the relationship between assets and objectives. Many risk assessments begin by identifying the priorities of the target organization [17]. The purpose of the pre-assessment phase in the NSA’s InfoSec Assessment Methodology is to identify customer needs and gain an understanding of critical systems and information. Typically, this information is gained through interviewing management and reviewing mission statements. However, it does not establish a relationship between assets and objectives. To achieve a more comprehensive understanding of how IT assets contribute to a goal, an organization’s missions and objectives will be modeled using mission trees. A. Mission Tree In a mission tree, the root is the ultimate goal of an organization. For instance, a corporate entity may identify its most important mission as increasing the company’s stock value, or a government agency may identify its primary mission as defending America. Beneath the Mission, M, lies the objectives which contribute to its success. Let Θ be the set of objectives and A be the set of assets. An objective, o ∈ Θ, is a specific goal that, when accomplished, will contribute to the success of its parent. The leaf nodes of the mission tree identify the IT assets, a ∈ A, of the organization. Assets are grouped within the tree by the goals they accomplish. Some assets may accomplish several different missions and, thus, may be 0-7803-9290-6/05/$20.00 ©2005 IEEE.

Fig. 1.

Generic mission tree evaluation.

C. Confidentiality, Integrity, and Availability Within this framework, an objective can only be successfully completed if the assets it depends on are available. If they become unaccessible, then the objective cannot be completed and the mission not accomplished. Essentially, the rating given to a particular asset in the mission model is a qualitative rating of availability. Therefore, an objective cannot be met if its assets are unavailable, and the more important the objective, the more important that its assets are available. Security policies are classified within the mission tree as objective nodes. As with all other nodes, policy nodes have values based upon maintaining certain security requirements, i.e. maintaining confidentiality or integrity. Some policy nodes operate just like a typical objective node. For example, policies related to firewall management have a firewall asset as their child node. Other policy objectives constrain assets. The children of policies concerning server access are the servers

389

Proceedings of the 2005 IEEE Workshop on Information Assurance and Security United States Military Academy, West Point, NY B. Vulnerability Evaluation

Fig. 2.

Generic risk tree.

specific to the policy. In this example, the ratings given to the child nodes represent how important it is that the particular asset maintains the security policy. Coincidently, it also reflects how the compromise of a system hinders the particular policy from being successful. An organization may evaluate the importance of maintaining confidentiality or integrity in the mission model by placing higher values on those particular objectives. If it is essential for data on certain assets to remain confidential or maintain integrity, then they will have higher values as well. IV. R ISK T REES Vulnerability scanning tools scan network elements for known vulnerabilities and configuration weaknesses. Although useful for identifying security flaws, such tools provide only limited risk analysis. To improve their capabilities, a manner for evaluating threats and assets must exist. Combining the analytical abilities of the mission model with the vulnerability discovery capabilities of a network scanner, a more effective risk assessment can be conducted. A. Critical States By valuating enterprise objectives and the manner in which they are accomplished, the mission model assesses the importance of each host in completing a particular mission. Within this context, vulnerabilities are only critical when they lead to a condition where an objective cannot be successfully accomplished, called critical states. The key is to identify these critical states and the vulnerabilities which lead to them. Critical states may be identified based upon the type of objective the host accomplishes. For example, hosts which provide a service must maintain availability, so a state of Denial of Service would be critical. Similarly, hosts which must contain confidentiality would have unauthorized access as a critical state. States where malicious entities can execute arbitrary code or gain root access are threatening for all objectives and will always be critical states. Once the critical states are identified, they can be added to the mission tree to create a risk tree, shown in Figure 2. Rather than indicating an organization’s priorities, a risk tree demonstrates the level of risk associated with an asset or objective. 0-7803-9290-6/05/$20.00 ©2005 IEEE.

Evaluating the impact of each vulnerability determines the risk for a host. The impact is defined as how an exploit of a vulnerability will hinder the host’s mission. If the exploit does not retard the asset’s objective, then the vulnerability is irrelevant and has no impact. In the case of a vulnerability that prevents a host from completing its mission, the impact will be the loss of the host. The loss of the host is equal to the value placed upon it in the mission model, Vc . Therefore, each vulnerability, ι, has an impact equal to the value of the host ´ Pnin the mission model, ι = Vc . So the risk for each host, a = i=1 ιi . The evaluation is represented in Figure 2. Asset 1 contains two vulnerabilities which lead to critical states, since Asset 1 has a value of 10 within the mission tree, each of its vulnerabilities will have an impact of 10. Consequently, the risk level of Asset 1 is 20, which raises the risk to its objective. The degree of risk to an entire enterprise can then be determined through the risk attributed to each of its objectives. V. A S AMPLE A SSESSMENT An on-line company with roughly 350 hosts on its network was evaluated using this methodology. In addition to building a mission model, a network scanning tool developed by Visionael [15] called Security Audit, was used. The results of the assessment, minus sensitive information, have been included in this section. A. Mission Assessment The mission model is built using information acquired by conducting interviews with IT management. This data gives assessors an understanding of the business, its goals, assets, and how they relate. Figure 4 shows the results from this initial information. The number within the box represents the asset’s value, and the number outside of the box is the number of children. In this case, the primary mission of the company is to meet their fiscal objectives for the year. They may accomplish this if they can successfully conduct transactions, meet customer privacy requirements, manage their business, and prevent fraud. The company conducts transactions using four different types of systems: CSRs, PBX network, Kiosks, and over the Internet. The assets of the CSR objective are shown in Figure 4. CSR’s are Customer Service Representatives who work for the organization and transact payments. The CSR cloud contains 27 hosts of similar configuration and purpose. Rather than evaluating all 27, one host will be evaluated and its results be applied to the remaining hosts. The privacy requirements objective contains assets which process customer information and are required to maintain specific security requirements to keep this information private. These hosts are placed beneath the privacy requirements objective. Likewise, hosts involved in the transaction process were placed beneath the prevent fraud objective and were assessed accordingly. Although there are close to 350 hosts in the network, not all of them actively participate in the primary objectives of the enterprise. To increase scalability, the assessors concentrated on

390

Proceedings of the 2005 IEEE Workshop on Information Assurance and Security United States Military Academy, West Point, NY

Fig. 3.

Fig. 4.

Company risk tree.

Company mission tree evaluation.

assets which made significant contributions to the company’s primary goals. Ideally, one would prefer to have all of the network assets within the mission model, but the assessment was conducted over a two week time frame and did not allow for that level of detail. Despite this abstraction, the results obtained from the assessment remain viable, because the loss of those hosts would not hinder the primary objectives of the company. If they could, then they would be contained in the mission model. B. Risk Assessment A vulnerability scan was then conducted on the network using the Visionael Security Auditor software. Combining the results from the mission model and the vulnerability assessment required identifying the critical states on each host where a vulnerability could prevent its mission from being accomplished. Figure 3 shows the result of this process upon the web transaction objective. Due to the fact that the objective requires all of its assets to be functioning, a denial of service would hinder its accomplishment, so it was added to the tree as a critical state. Likewise, if a malicious entity were to gain root access or be able to execute arbitrary code, it would be possible to prevent the host from providing its service, and so it was also considered a critical state. For the privacy requirements objective, a state where an attacker gains unauthorized access to data was identified as a critical state. The Visionael scans of each host in the mission tree were then searched for vulnerabilities leading to the identified 0-7803-9290-6/05/$20.00 ©2005 IEEE.

critical states. If found, they were added to the risk tree beneath the critical state they lead to. If no vulnerabilities were found, then the critical state was left empty. The risk posed by a vulnerability to a host is displayed at the bottom of its box and is determined by the host’s value within the mission tree. Within the Web objective, the asset WebServ has a value of 1 in the mission tree. Therefore, the impact of any vulnerabilities leading to a critical state will be 1. Since WebServ has two vulnerabilities, its risk level is 2. Interestingly, it was discovered that some states could be reached through vulnerabilities on other hosts, shown in Figure 3. A vulnerability was identified on host ServProd that leads to a denial of service on SQLServ. The vulnerability did not hinder ServProd from meeting its objective, so it is not one of its children. However, since it affects SQLServ, it is one of its children, even though it does not contain the vulnerability. VI. R ISK A NALYSIS After conducting a risk analysis as described in the previous section, the assessment team was able to determine that the target organization had a risk level of 83, shown in Figure 3. The assessment was then broken down by individual objectives and, ultimately, specific hosts. It demonstrated which of the organization’s objectives are at greatest risk and assisted security personnel in identifying the offending hosts and vulnerabilities. Mission trees and risk trees present an organization with the opportunity for scalable and effective risk metrics to make cost-effective security decisions. The organization faces the greatest risk to its conduct transactions objective. Within the objective, the CSR and PBX objectives are the primary culprits, shown in Figure 3. One way to reduce the risk level, is by patching all 27 of the hosts within the CSR cloud. Each CSR host contains 6 vulnerabilities; mitigating their threat involves patching 162 vulnerabilities for a 16 point drop in risk. In other words, each mitigated vulnerability provides a .1 drop in risk, or .6 points per host. Mitigating them decreases the risk level of the CSR objective from 24 to 8. Unfortunately, the required cost of manually patching 162 vulnerabilities on 27 different machines may make this method unfeasible.

391

Proceedings of the 2005 IEEE Workshop on Information Assurance and Security United States Military Academy, West Point, NY A. Host Trees A more viable method is to identify a single host containing vulnerabilities that increase risk throughout the enterprise. The host named SQLServ is used to accomplish numerous objectives and it contains two vulnerabilities which, if exploited, could prevent it from meeting any of its objectives. Therefore, it is a natural candidate for mitigation analysis.

Using this method, it is possible for an auditor to identify which vulnerabilities exist on the most hosts. Additionally, it can be used to conduct cost-benefit analysis on mitigation techniques. Identifying vulnerabilities that create the greatest risk in the organization and then sorting those results by the fewest number of occurrences would maximize mitigation and minimize cost.

Fig. 6. Fig. 5.

Vulnerabilty tree.

Host tree.

A host tree has been created to visualize this predicament, shown in Figure 5. The root of the tree is the name of the host, its immediate children are all of the objectives that the host helps to accomplish. Beneath those objectives are critical states and the vulnerabilities which lead to them. Using values from the risk tree for each vulnerability, it is possible to see how vulnerabilities existing on a single host contribute to a network’s risk. For instance, the RDP vulnerability on SQLServ is responsible for raising the CSR’s risk by 3 points. Ultimately, it raises risk for the conduct transactions objective by 6 points and makes up 13 points of risk for the entire network, two of which are collapsed under the privacy requirements objective. The ARP cache poisoning vulnerability carries identical weight. Mitigating these vulnerabilities requires patching a separate vulnerability on two different machines. Both vulnerabilities account for 26 points of the total risk. Patching SQLServ and ServeProd is more cost-effective than patching 27 CSR hosts and yields a greater reduction in risk. B. Vulnerability Trees Analysis may also be conducted on specific vulnerabilities to view their impact on an enterprise by using vulnerability trees. A vulnerability tree uses the risk tree to locate all instances of the target vulnerability within an enterprise and discover how it inflames the enterprise’s risk. The root of the tree is the name of the vulnerability being assessed, its children are all of the hosts which have the vulnerability; the children of each host are the objectives they accomplish. The example shown in Figure 6 demonstrates that the RDP vulnerability is located on hosts SQLServ and VoiceLog, and shows its affect on their objectives. The vulnerability is responsible for 3 points of the risk on the CSR objective, which can be verified by cross-referencing with Figure 5. Likewise, it shows the RDP vulnerability is the cause of 13 points of risk on SQLServ. Adding its effect on the other affected host, VoiceLog, produces an additional 16 points of the risk to the organization. 0-7803-9290-6/05/$20.00 ©2005 IEEE.

VII. C ONCLUSION The risk assessment methodology described in this paper synthesizes high-level enterprise assessment with detailed technical analysis to generate security metrics. Despite providing in-depth host and vulnerability dissection, it remains scalable for large networks. Using these metrics, an organization can prioritize mitigation venues, assess host risks, and protect important objectives. These risk metrics help security personnel make cost-effective security decisions and demonstrate how those decisions will affect the organization’s security posture. A. Future Work Scalability could be increased by automating the identification of critical states and vulnerabilities from the mission model. In addition, an increase in utility requires a way to differentiate between likely and unlikely attacks. One way this could be done is through use of multi-stage attack trees described in [18]. Identifying a possible threat, i.e. insider, hacker, worm, etc., and then enumerating the paths in which it could achieve a critical state would be helpful in determining likelihood. Using fault tree analysis, it could be shown that attacks which require multiple exploits to achieve success would be less likely to occur than those which require only a single step. Risk rating could then be provided for various types of adversaries and their most likely attack plan.

392

R EFERENCES [1] J. I. Alger, “On assurance, measures, and metrics: Definitions and approaches,” in Proceedings of the 1st ISSRR Workshop, ACSAC, 2001. [2] P. Bicknell, “Security assertions, criteria, and metrics: Developed for the IRS,” in Proceedings of the 1st ISSRR Workshop, ACSAC. [3] N. Seddigh, P. Pieda, A. Matrawy, B. Nandy, J. Lambadaris, and A. Hatfield, “Current trends and advances in information assurance metrics,” in Proceedings of the Second Annual Conference on Privacy, Security and Trust, pp. 197 – 204, Privacy, Security, and Trust, 2004. [4] B. Wood and J. Bouchard, “Red team work factor as a security measurement,” in Proceedings of the 1st ISSRR Workshop, ACSAC, March 2001. [5] B. S. Yee, “Security metrology and the monty hall problem,” in Proceedings of the 1st ISSRR Workshop, ACSAC, April 2001.

Proceedings of the 2005 IEEE Workshop on Information Assurance and Security United States Military Academy, West Point, NY [6] J. McHugh, “Quantitative measures of assurance: Prophecy, process, or pipedream?,” in Proceedings of the 1st ISSRR Workshop, ACSAC, May 2001. [7] R. B. Vaughn, “Are measures and metrics for trusted information systems possible?,” in Proceedings of the 1st ISSRR Workshop, ACSAC, May 2001. [8] R. T. Mercuri, “Analyzing security costs,” Communications of the ACM, vol. 46, pp. 15 – 18, June 2003. [9] D. J. Bodeau, “Information assurance assessment: Lessons-learned and challenges,” in Proceedings of the 1st ISSRR Workshop, ACSAC, May 2001. [10] A. Hunstad, J. Halberg, and R. Andersson, “Measuring IT security - a method based on common criteria’s security functional requirements,” in Proceedings from the fifth IEEE Systems, Man and Cybernetics Information Assurance Workshop, IEEE, IEEE, 2004. [11] G. Rogers and B. Stauffer, “An approach to InfoSec program metrics,” in Proceedings of the 1st ISSRR Workshop, ACSAC, March 26 2001. [12] D. McCallam, “The case against numerical measures for information assurance,” in Proceedings of the 1st ISSRR Workshop, ACSAC, May 2001. [13] E. A. Schneider, “Measurements of system security,” in Proceedings of the 1st ISSRR Workshop, ACSAC, May 2001. [14] Nessus. http://www.nessus.org. [15] “Visionael security audit,” tech. rep., Visionael Corporation. http://www.visionael.com/. [16] M. Swanson, N. Bartol, J. Sabato, J. Hash, and L. Graffo, “Security metrics guide for information technology systems.” National Institute for Standards Publication 800-55, July 2003. [17] “Infosec assessment methodology,” tech. rep., National Security Agency. http://www.iatrp.com/iam.cfm. [18] J. Dawkins and J. Hale, “A systematic approach to multi-stage network attack analysis,” in Second IEEE International Information Assurance Workshop (IWIA’04), pp. 48–58, IEEE Computer Society, 2004.

0-7803-9290-6/05/$20.00 ©2005 IEEE.

393