Addressing the Need for Independence in the CSE Model Presented at: CICS 2011 2011 IEEE Symposium on Computational Intelligence in Cyber Security Paris, France Robert K. Abercrombie, Ph.D. Erik Ferragut, Ph.D. Frederick T. Sheldon, Ph.D. Michael R. Grimaila, Ph.D.
April 12, 2011 The submitted manuscript has been authored by a contractor of the U.S. Government under contract DE-AC05-00OR22725. Accordingly, the U.S. Government retains a nonexclusive, royalty-free license to publish or reproduce the published form of this contribution, or allow others to do so, for U.S. Government purposes.
Agenda For Today’s Presentation
Addressing the Need for Independence in the CSE Model
• Background – Identification of Risk – in the context of system security risk and mission assurance
• CSES Modification for Current Investigation • CSES Foundations - One Enabling Technology - Expanded Explanation: Application of Mean Failure Cost – Stakes Matrix: Stakeholders versus Requirements – Dependency Matrix: Requirements versus Components – Impact Matrix: Components Failure versus Threats – Mitigation Costs Matrix: Verification Costs via Requirements or Components
• Expansion of Method • Enumerating Disjoint Events • Conclusions and Future Directions
2
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Background of Risk & Risk Management in context of Mission Assurance • Risk and Risk Management – per ISO 73 2009 – Risk is formally defined as “the effect of uncertainty on objectives” – Risk management involves the “coordinated activities to direct and control an organization with regard to risk”
• Organizations typically use Risk Management to: – Identify and mitigate risks to assure their organizational mission – Provide a documented, structured, and transparent process to identify critical resources, estimated threats, and vulnerabilities. • The intersected set of threats and vulnerabilities cause harm (risks) to those identified resources.
3
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
CSES Modification - Current Investigation • CSES combines the subject domain of information security and risk management – CSES is a methodology for estimating security, security costs to stakeholders as a function of possible risk postures.
• Current CSES addresses independent Events – – Analysts create matrices that capture their expert opinion, and then use those matrices to quantify costs to stakeholders. – In situations where the underlying events exhibit significant dependencies, the current approach is not appropriate. – Significant dependencies are likely to arise with any detailed modeling of a complex system of components, such as an enterprise network. 4
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Introduction of Dependence & Disjointness • Dependence – Two events are dependent if the probability of one event is altered by knowing whether the second event occurred.
• Disjointness
Independence P(AB) = P(A)P(B)
A
AB B
Disjointness P(AB)=0
– Two events are disjoint if they cannot both occur simultaneously.
A
• Mathematically:
1. Disjoint events are dependent , and 2. Independent events cannot be disjoint (see figure). 5
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
B
Implication of Dependence & Disjointness • While dependence needs to be addressed, the mathematics of CSES exploits the disjointness of the underlying events. – In particular, matrix products are used to compute probabilities (a computational application of the Law of Total Probability provided that the events are disjoint).
• An improvement to CSES continues to use matrix products as an efficient way to compute outcome probabilities, but directly addresses dependence among events by enumerating all possible combinations. • Although this yields a potentially intractable number of event combinations to consider, we argue that the dependency structure among these events will generally greatly simplify the manual work required. 6
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Process of the Automated Tool – a cascade of linear models • Determination of a series of matrices: – – – –
7
Stakes Matrix by the Stakeholders Dependency Matrix by the Architects Impact (Threat) Matrix by the Verification and Validation (V&V) Group Mitigation Costs Matrix by the Analysts and V&V Group
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Stakes Matrix: Stakeholders vs. Requirements
• Premises necessary for MFC estimation:
• Best represented with 2 dimensional matrix: – Rows: Stakeholders – Columns: Requirements – Entries: Stakes
Stakeholders
– A stakeholder may have different stakes in different requirements – A requirement may carry different stakes for different stakeholders
Requirements R1 R2 R3 … Rn S1 S2 S3 … Sm
cost that stakeholder Si would lose if the system failed to meet requirement Rj Probability that the system fails to meet requirement Rj
8
Managed by UT-Battelle for the U.S. Department of Energy
FCi,j
Addressing the Need for Independence in the CSE Model
Stakes Matrix: Stakeholder vs. Requirements
The Stakes Matrix below illustrates an example of how Failure Cost (FC) is derived. The FC entry at row i, column j, represents: • the cost that stakeholder Si would lose if the system failed to meet the security requirement Rj.
Security Requirements R1
R2
R3
…
Rn
S1 S2 Stakeholders
S3 …
FC i
j
Sm Stakeholder
9
•Requirements •Requirement 1: Safety record
•Requirement 2: Timeliness
Passengers
•Personal Safety
•Convenience, Scheduling
Airline company
•Liability for Loss of Life •Reputation of airline
•Reputation with for timeliness / Public relations
Aircraft Manufacturer
•Liability for Loss of Life •Reputation of aircraft
•Zero
Accident Insurance of Aircraft
•Premium owed for loss of aircraft
•Zero
Life Insurance of Passenger
•Value of life insurance
•Zero
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Dependency Matrix: Requirements vs. Components
Requirements
C1 C2 R1 R2 R3 … Rn
Components C3 … Ck Ck+1
(Ri|Ej)
• Links the probability of failing a particular requirement with the probability of failure of a component of the system • Simplifying hypothesis: assume that violations affect no more than one component at a time • Let Ei, for 1≤i≤k, be the event: failure of component Ci
▫ Event Ek+1: no component has failed
Probability of failing requirement Ri given component Cj fails Probability of requirement Ri failing 10
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Probability of component Cj failing
Dependency Matrix: Requirements and Components
An analysis of the system architecture, by architecture subject matter experts, can lead to the derivation of conditional probabilities that link the probability of component failures with the probabilities of failing to meet specific requirements.
R1
This information can be represented in a 2 dimensional matrix, which we call the Dependency matrix.
R2
The term (Ej) represents the probability of event Ej
R3
The term (R|Ej) represents the probability of failing to satisfy requirement Ri, given hypothesis Ej (i.e., that event j has occurred).
In the table there exists a component event Ej for a requirement Ri where the probability of failure to satisfy requirement R exists ( (Ri|Ej) ):
Components C1
Requirements
C2
…
C3
Ck
( Ri | E j )
… Rn
Components Requirements
Processing Components
Login Component
Secure Storage Component
User Profile Analysis
Freedom from Insider Threats
0.01
0.6
0.2
0.98
Protection of Critical Data
0.01
0.2
0.98
0.2
Access Control
0.01
0.98
0.4
0.1
The Dependency Matrix showing the relationship between Requirements and their distinct respective components and failure results (probability) with Respect to Passenger. 11
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Impact Matrix: Component Failure vs. Threats
Components
T1 T2
C1 C2 C3 … Ck Ck+1
Threats T3 …
Th Th+1
• To assess the likelihood of a particular threat leads to failure of a component: – Set of threats T1, T2,…, Th – Events V1, V2,…, Vh, Vh+1 – Vi, 1≤i≤h: Threat i has materialized
(Ei|Vj)
– Vh+1: No threat i has materialized – Assume that no more than one threat materializes at a time
Probability of component Ci failing given threat Tj materializing
Probability of component Cj failing
12
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Probability of threat Tj materializing
Impact Matrix
(Component Failure vs. Threat Relationship)
Threats T1
T2
T3
…
Th
The Impact Matrix showing Component Failure versus Threats Relationship Grouping •
To assess the likelihood that a particular threat leads to the failure of a component, we consider a set of cataloged threats (or families of threats with common attributes), say T1, T2, T3, … Th, and we consider the events V1, V2, V3, … Vh, Vh+1, where Vi, for 1 ≤ i ≤ h, stands for: Threat i has materialized, and Vh+1 stands for: No threat i has materialized.
•
The probability of threat Tj (which is (Vj)) to the probability of component failure for component Ci (which is (Ei)). To apply this formula, we need to derive the conditional probabilities, which we propose to represent in a 2 dimensional matrix, that we call the Impact matrix.
C1 C2 Components
C3 …
( Ei | V j )
Ck
Threats Components
13
Insider Threats
Intrusions
Denial of Service Threats
Authentication Threats
No Threat
Processing Component
0.2
0.4
0.8
0.8
0.0
Login Component
0.2
0.2
0.2
0.2
0.0
Secure Storage Component
0.2
0.4
0.2
0.2
0.0
User Profile Analysis
0.2
0.1
0.1
0.1
0.0
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Summary of Calculation of MFC Y: vector of size n X: vector of size m A: n×m matrix
ST: Stakes Matrix PR: vector of requirement failure probabilities
DP: Dependency Matrix PE: vector of component failure probabilities
IM: Impact Matrix PT: vector of threat emergence probabilities
14
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Expansion of Method – Practical Application - Inclusion of dependent events
• Expert Matrices - Analysts create matrices just described • Matrix Products – Reviewed for sensitivity analysts • Underlying Assumptions – 1. Probabilities of different matrices are independent. – 2. Entities in each class must correspond to disjoint events.
• Enumerating the Disjoint Events – Addressing the lack of generality of Assumption Two by enumerating a finite set of disjoint events makes CSES applicable even without Assumption Two as follows: • Enumeration, Calculation, Interpretation, and Population
15
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Enumerating the Disjoint Events - Enumeration • If a system contains components A, B, and C, – then there are eight possible component failure states (one of which represents no failures) based on the eight subsets of (A, B, C) that fail. (A, B, C, AB, AC, BC, ABC, and 0). – In general, we have 2k states for k entities.
• We propose that the original sets of requirements, components, threats, and mitigations be replaced by all of their possible combinations. • The resulting matrices, while being exponentially larger, will (by definition) satisfy Assumption Two, thereby making the calculations correct. • However, these larger expert matrices potentially introduce new challenges to the CSES methodology with respect to calculating and interpretating the product matrix, and populating the matrices. 16
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Enumerating the Disjoint Events - Calculation • The n1 × n3 product of an n1 × n2 matrix with an n2 × n3 matrix can be computed in work O(n1n2n3). • For a product of four matrices, the work becomes O(n1n2n3 + n1n3n4) or O(n2n3n4 + n1n2n4), depending on the order of operations. – A subfield of combinatorial optimization has addressed how to best order the computations for efficiency.
• Since we only have a small number of matrices, we can reasonably handle 1000×1000 matrices; significantly larger matrices may pose a problem. • In the near term, we propose limiting the analysis to these relatively small matrices. (This allows for up to 10 independent factors prior to the combinatorial enumeration.) • With additional improvements in efficiency that exploit structural properties and sparsity, we expect to be able to extend the practical matrix size limits. • In particular, since we will be using the dependency structure among the events to populate the matrices, we should be able to use the same structure to computationally simplify the product calculation (future work).
17
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Enumerating the Disjoint Events - Interpretation – Triage (Previous Papers) • A triage process for reducing the number of mitigations that need to be considered. • First, if the mitigation costs can be estimated, we can eliminate all mitigations that are too costly. – These mitigations are the enumerated combinations of the original set of mitigations. – While no original mitigation would be included if it was cost-prohibitive, it is likely (since CSES is being used to support security decisions) that there are combinations of these that are too expensive.
• Second, if there are unacceptably high risks to certain stakeholders, we can eliminate the mitigations resulting in those high risks. – The use of stakeholder utilities allows for a highly expressive means of accounting for arbitrary stakeholder concerns.
• Third, if two mitigations are such that one is both cheaper to implement and costs less to the stakeholders (this is many simultaneous conditions), then the other mitigation can be dropped. – These notions should help to reduce even a large set of mitigations to a small set of “best” candidates for analysis by a human. 18
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Enumerating the Disjoint Events - Population • If each of the four expert matrices is about 1000 × 1000 in size, • then there are 4 million entries that need to be assigned.
• In practice, assigning them individually is inadvisable since – it would take far too long, – the resulting entries are bound to be inconsistently derived, and – any structural relations between the entries must be enforced manually.
• Explicitly outline the relationships between the components. – Using a Bayesian belief network to capture structural (conditional) dependencies satisfies this, since – A Bayesian belief network is a probability model that efficiently captures the potentially highly complex relationships between variables by “factoring” them as conditional dependencies.
19
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Enumerating the Disjoint Events - Population (cont.)
• Why Bayesian belief networks address this problem: – Represents many variables with a small number of parameters (2 examples in the paper), i.e. expressive, – Enforces probabilities to be consistent across all instances of a variable’s value, – Simplifies maintenance to include updating and extending of the probabilities, and – Captures cognitively how experts view dependencies and complexities in risk.
20
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Conclusions and Future Directions • We described CSES and the fundamental shortcoming, – which we referred to as Assumption Two – Entities in each class (other than Stakeholders) must correspond to disjoint events.
• We then proposed a method for addressing that assumption in a general setting via the use of combinatorial enumeration. – The resulting issues from the combinatorial explosion can be addressed by an application of Bayesian belief networks (to incorporate expert opinion) and basic exploratory heuristics (to analyze the results).
21
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Conclusions and Future Directions
(cont.)
• Future work should include: – Continued Validation: • additional implementations of this method, and • exploration of computational enhancements for particular structured matrices, • which would lead to validation via real-world examples.
– Further investigation is warranted in merging our method with a hybrid analytic dynamic forecasting (HADF) methodology • that combines the techniques of analytic hierarchy processes, factor analysis, and spanning trees to the problem of selecting among a set of contingency measures following events which place the organizational mission at risk.
22
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
References •
R. K. Abercrombie, F. T. Sheldon, and A. Mili, "Validating Cyber Security Requirements: A Case Study," IEEE Proceedings of the 44th Annual Hawaii International Conference on System Sciences (HICSS-44), Koloa, Kauai, Hawaii, January 4-7, 2011, Computer Society Press, 2011 (10 pages) .
•
R. K. Abercrombie, F. T. Sheldon, and M. R. Grimaila, "A Systematic Comprehensive Computational Model for Stake Estimation in Mission Assurance," Proceedings of International Workshop on Mission Assurance: Tools, Techniques, and Methodologies (MATTM 2010), 2nd IEEE International Conference on Social Computing (SocialCom 2010) / 2nd IEEE International Conference on Privacy, Security, Risk and Trust (PASSAT 2010), Minneapolis, MN, August 20-22, 2010, Computer Society Press, pp. 1153-1158.
•
A. Ben Aissa, A. Mili, R. K. Abercrombie, and F. T. Sheldon, "Modeling Stakeholder/Value Dependency through Mean Failure Cost," Proceedings of 6th Annual Cyber Security and Information Intelligence Research Workshop (CSIIRW-2010), ACM International Conference Proceeding Series, Oak Ridge, TN April 21-23, 2010.
•
A. Ben Aissa, A. Mili, F. T. Sheldon, and R. K. Abercrombie, "Software Requirements for a System to Compute Mean Failure Cost," Proceedings of 6th Annual Cyber Security and Information Intelligence Research Workshop (CSIIRW-2010), ACM International Conference Proceedings Series, Oak Ridge, TN April 21-23, 2010.
•
Anis Ben Aissa, Robert K. Abercrombie, Frederick. T. Sheldon, and Ali Mili, “Quantifying Security Threats and Their Potential Impact: A Case Study,” in Innovation in Systems and Software Engineering: A NASA journal, in press, 2010.
•
F. T. Sheldon, R. K. Abercrombie, and A. Mili, "Methodology for Evaluating Security Controls Based on Key Performance Indicators and Stakeholder Mission," IEEE Proceedings of the 42nd Annual Hawaii International Conference on System Sciences (HICSS-42), (CD-ROM), Waikoloa, Big Island, Hawaii, January 5-8, 2009, Computer Society Press, 2009 (10 pages).
•
R. K. Abercrombie, F. T. Sheldon, and A. Mili, “Synopsis of Evaluating Security Controls Based on Key Performance Indicators and Stakeholder Mission Value,” in 11th IEEE High Assurance Systems Engineering Symposium (HASE '08), Nanjing, China, 2008, pp. 479-482.
•
F. T. Sheldon, R. K. Abercrombie, and A. Mili, "Evaluating Security Controls Based on Key Performance Indicators and Stakeholder Mission," Proceedings of 4th Annual Cyber Security and Information Intelligence Research Workshop - Theme: Developing Strategies to Meet the Cyber Security and Information Intelligence Challenges Ahead, ACM International Conference Proceeding Series, Vol. 288, Oak Ridge, TN, May 14, 2008.
23
Managed by UT-Battelle for the U.S. Department of Energy
Addressing the Need for Independence in the CSE Model
Oak Ridge National Laboratory: Meeting the challenges of the 21st century
24
Managed by UT-Battelle for the U.S. Department of Energy
Robert K. Abercrombie, Ph.D. Erik Ferragut, Ph.D. Email:
[email protected],
[email protected] U.S. Phone: +011 (865) 241-6537/ http://www.ornl.gov/~abe, Addressing the Need for Independence in the CSE Model http://www.csiir.ornl.gov/abercrombie
