A new assessment and improvement model of risk ...

Int. J. Information and Computer Security, Vol. 1, No. 3, 2007

341

A new assessment and improvement model of risk propagation in information security Suleyman Kondakci Faculty of Computer Sciences, Izmir University of Economics, Sakarya Cad. No. 156, 035330 Balcova-Izmir, Turkey Fax: +90-232-279 26 26 E-mail: [email protected] Abstract: This paper presents an analysis of fault propagation in information security solutions. It presents a unique and efficient approach to security assessment that can be useful for security planners, evaluators, managers, and IT owners to discover and correct weaknesses at any stage of security planning processes. Intuition and qualitative approaches are not adequate to guide accurate risk analysis in information security. In this paper, we present a rather formalised preventive approach to guide the risk management quantitatively. The quantitative approach determines the propagation of the design faults by use of a probabilistic method supported by a scoring scheme. Keywords: risk assessment; knowledge and policy management; security risk propagation; human factor; information and computer security. Reference to this paper should be made as follows: Kondakci, S. (2007) ‘A new assessment and improvement model of risk propagation in information security’, Int. J. Information and Computer Security, Vol. 1, No. 3, pp.341–366. Biographical notes: Suleyman Kondakci was born in Kars, Turkey. He has obtained Electrical & Electronics Engineering from the University of Gazi, Ankara in 1979, Candidatus Magisterii and Candidatus Scientiarum from the Department of Informatics, University of Oslo, Norway, 1984 and 1987. Kondakci has various contributions within information security. He has established the Information Security Test and Evaluation Laboratory at the Scientific and Technical Research Council of Turkey and conducted several security projects for critical systems. He has also contributed to the NATO Information Security Framework. His current interests are telecare systems and their security and security of critical infrastructures.

1

Introduction

Despite the numerous recommendations from international organisations and professional associations, there is no conventional language for information-security risk measurement, nor are there formally established metrics for quantitative risk management. A number of professional and industry associations have published guidance that recognises the importance of the risk management to assure the security of information, CC (2006), ISO/IEC:COBRA (2006), ISO/IEC:FDIS 15408-1 (2005), Copyright © 2007 Inderscience Enterprises Ltd.

342

S. Kondakci

COBIT (2006), NIST (2006), SANS (2006), ISC2 (2006), CIS (2006), ISA (2006), ISACA (2006) and ISSA (2006). As a result of a research project, we have developed an umbrella framework of concepts, which covers the analysis of threat domains, human-factors, improvements, and assessment methods supported/enhanced with quantitative evaluation results. A compound security analysis combined with security test and improvement mechanisms embodies a powerful framework. We refer to the compound framework as CSAIF: Compound Security Analysis and Implementation Framework. As a major requirement, the compound framework should provide reliable approaches to both analysis and implementation of information security solutions. Security providers and evaluators could use the CSAIF framework in their security planning, implementation, and assessment processes. The quantitative risk assessment of CSAIF is a useful simple feature that enables us to track deficiencies even under the security planning. Current methods within the field found so far in the literature do not address the compound security analysis. The idea behind CSAIF is unique in identifying weaknesses systematically and developing a scheme for quantitatively examining the weaknesses in dynamically changing information infrastructures and their constituent services and assets. Our method has advantages of its simplicity and the peculiarity of its proactive approach to early detection of deficiencies in security solutions. Its simplicity will guide system owners to use it as a self-assessment tool and procedure as necessary. Some of the internationally recognised security evaluation standards and approaches are Common Criteria (CC, 2006), and BS 7799 (will be renumbered as ISO/IEC 27002), which is also detailed in ISO/IEC:COBRA (2006). These methodologies define standards and methodologies in information security evaluation and certification. However, they do not define the methodology for risk assessment. The standards specify only that the organisation should use a systematic approach to risk assessment. On the contrary, CSAIF does not specify a broad and general standard. However, it can be used as a complementary risk management system with a specific methodology. CC and BS 7799 standards also encourage using auxiliary systematic approaches to risk assessment methodologies. Hence, a key benefit of the approach presented by CSAIF is that both the CC and BS 7799 certification facilities and others can apply CSAIF in their certification programmes. Security awareness is a known problem in many developing countries. Most of the organisations do not have IT security specialists or managers. They merely rely on solutions provided by ad hoc basis. Most of the users trust rumour-based ‘de facto’ solutions. Another unacceptable fact is that none of the organisations perform regular IT audits or security evaluations. Unfortunately, the internet consumers are not far sensitive. There is a fact that, in many environments users rely on unverified rumour-based ‘de facto’ solutions in an overlapping manner. This is true, at least in Turkey, which contains one of the largest IT consuming population among the developing countries. Turkey has no regulation considering the IT security test and evaluation issues. Especially, Turkey has not yet established the fundamental IT policies. Necessary laws have not been founded to enforce the creation of policies for IT and IT-security. Besides, we do not have laws for enforcing the establishment of the facilities for test and evaluation and certification of information systems. A socio-organisational approach to information systems security management and proposals are presented in Koskosas and Paul (2003). It proposes a framework, which illustrates three important issues in the process of security goal setting. These issues are trust, culture and risk communication. They have also weighed evidence that there is a chain reaction among these issues with a

A new assessment and improvement model of risk propagation

343

subsequent effect on the level of security goal setting. A different view and a rather contradicting discussion to risk analysis and science can be found in Even (2004). Technical approaches seem to be the first envisioned solutions to security. However, the criticality of human decisions and actions are unarguable facts in causing failures even with the superiority of the technical approaches. We will present an analysis of effects of human decisions in systems assurance supported by some statistical inferences. Probabilistic Risk Assessment (PRA) is a topic of wider spectrum. Some of the valuable sources discussing PRA are: Kumamoto and Henley (2000), Bedford (2003), Vose (2000), Everitt (1999) and Bruske et al. (1985). PRA has been applied to diverse fields, particularly to safety analysis of nuclear power plants, to environmental, and to medical and life sciences. PRA is a well-established technique for integrating various reliability-modelling tools, such as Fault Tree, Event Tree, Reliability Block Diagram, and Failure Mode and Effects Analysis (FMEA) to quantify risks numerically. This paper deals with risk analysis in security planning using a top-down probabilistic approach complementary to that of the fault tree analysis. A fault tree analysis is a deductive, top-down method for analysing the system design and performance. It involves specifying a major event to analyse, followed by identifying all of the associated elements in the system that could cause that major event to occur. Fault tree analyses are generally performed graphically using a logical structure of gates, such as AND, OR, and combinations. The method proposed here applies a score-based quantification of vulnerability and strength level analysis in a security planning and/or in operational systems. Thus, it specifies security implementation as the major event to analyse. Security objectives and policies are the associated elements that lead to security implementation of varying qualities. Several useful sources describe risk management in two categories, qualitative and quantitative. Peltier (2001, 2005), McCumber (2005), Walsh (2002), Gallegos et al. (2004), Tipton and Krause (2003) and Herrmann (2002) discuss information security risk management in general. The qualitative method is rather difficult to apply efficiently. It relies more on expertise and intuition of the evaluator. On the other hand, the qualitative approaches use deterministic methods to analyse risks that can easily identify impacts. Moreover, accuracy and applicability of the qualitative approach can be a challenging goal to achieve. In this paper, we focus on the quantitative approach to analyse weaknesses and strength of security solutions or systems. Some of the well-known test and evaluation standards and approaches are guided by the standards defined by Common Criteria (CC) and ISO/IEC 17799:2000 (BS 7799). Details and applications of the CC standard and other related methodologies are given in CC (2006), ISO/IEC:COBRA (2006), ISO/IEC:FDIS 15408-1 (2005), COBIT (2006) and NIST (2006). Some of the risk management approaches are hybrid, i.e., using both the qualitative and quantitative approaches. Two of them are OCTAVE (Alberts and Dorofee, 2002) and GAO/AIMD-00-33 (1999). Carnegie Melon University and the Software Engineering Institute (SEI) developed the OCTAVE approach. The OCTAVE approach is a useful source that addresses both the technical and organisational issues. It defines the security risk in combination of four major parameters, asset, threat, vulnerability, and impact. GAO applies some principles to assess the risk. It bases the assessment on a risk assessment matrix that represents risk levels computed from the severity and probability of occurrence of harmful events. An advantage of the GAO method over to the OCTAVE is that it performs continues monitoring of the evaluation

344

S. Kondakci

process. This ensures rather efficient lifecycle security and has the ability to detect faults in the evaluation system.

1.1 Outline of the paper In the following, Section 2 presents the validation model of security designs. This section provides also an assessment model and criteria used as guidance to overall security design and improvement. Section 3 discusses the risk propagation model based on statistical test methods and inferences to analyse dependency factors and risks in the security planning. It also presents a probabilistic model to explore the conditional dependence among the security planning phases. Section 4 concludes the paper. A special case analysis is given in Appendix A to justify the methodologies presented in the paper. Appendix A discusses further the public awareness of the security aspects by resorting to results of a statistical survey.

2 Validation of security designs The specification of the security objectives and policies are related to human actions. These are hard to control by other means than the human. The evaluation of the objectives can indeed determine the overall procedural security for organisations. The knowledge of an organisation is a fundamental criterion for proper evaluations of the overall technical abilities. We formalise the assessment of the knowledge of an organisation by evaluating the objectives of the organisation. That is, security objectives cover the basic knowledge of the organisation (what to protect, what is security and its importance); overall IT infrastructure, the entire environment, and the assets needed to fulfill the business requirements. Different knowledge levels may lead to different policy definitions. Obviously, different policy definitions may also lead to different security solutions. The procedural evaluations are mostly performed by answering questionnaires that are tabulated in several documents. Thus, there is a hierarchical dependence between the coverage of the security objectives and the determination of the security policy. Hence, the ultimate quality of the protection is also strictly dependent on the security policy. There is a chain of dependence starting from the objectives ending with the security implementation. Therefore, this paper will also emphasise the fact that the security-awareness has an impact on the implementation of the required security mechanisms. Figure 1 illustrates the three major phases of security planning. The process of lifecycle security planning involves at least four phases: •

the initial phase; requirements analysis (identification of objectives)

•

composing a policy and design

•

implementation

•

test and evaluation.

A new assessment and improvement model of risk propagation Figure 1

345

Overall security dependence diagram for systematic approaches. Phase 1 determines the security objectives and requirements, phase 2 determines the policy, and phase 3 implements the security functions

Implementation of the security mechanisms must match the needs, which are indicated by the composed policy. These phases might be fine grained, but we find them satisfactory in the following discussion. Deficiency analysis during the initial phase is an essential task needed to reduce risks that can propagate down to the subsequent phases of the entire security planning. As a grainy classification of the security-awareness, there are two main categories of environments: security-aware and security-unaware (level 0). A security-aware environment is further classified in two levels: 1

aware of only policies

2

designed the policies and implemented security measures.

That is, a level 1 environment is only a knowledgeable environment. Owners of such environments perceive the importance of the entire environment and its security. A system with awareness of level 1 has an idea about its security objectives and policies, but has not been able to implement the security mechanisms. Systems with level 2 are fully aware if they have defined policies and implemented the corresponding security mechanisms.

346

S. Kondakci

As shown in Figure 1, environment A is security-unaware, B is security-aware to some degree (level 1), while C illustrates an overall security-awareness. Environment A could not implement effective security functions because there are no defined security policies, which are dependent on the awareness aspect. Environment C, on the contrary, has a full coverage of the security implementation, because it has systematically applied all of the phases of the security planning. A security solution should be evaluated to reduce the risk propagation. Referring to Figure 1, evaluation of the phases of a security solution is a trivial task. We use a scoring scheme to evaluate and quantify each phase by assigning grades between 0 and 5, where 5 depicts the highest grade. Each assessment result is categorised into three equally quantified levels (sub ranges) such as Loosely Defined (LD), Defined (D), and Well Defined (WD). These sub ranges are defined as LD = {0, …, 1.66}, D = {1.67, …, 3.33}, WD = {3.34, …, 5}. Following the analysis, we match the average evaluation score to one of these categories. This will then determine the category of the system under test, whether the system conforms to LD, D or WD. Additionally; each of the planning phases for the organisation is evaluated in order to reduce the risk propagation. A key benefit of this method is that it lets the organisation to self assess the quality of its security planning.

2.1 Assessment model as a guidance to overall security solution A proper security design should go through a design validation. This will alleviate predicting the fault propagation. Design validation is the task of verifying design efficiency. Validation involves several steps, such as the input data generation, defining the coverage metrics, and evaluating the input data. A coverage metric defines a set of criteria that are used to determine faulty conditions in a test process. The coverage metrics must consider a related set of faulty conditions (or behaviours). Often, the security professionals prepare the coverage metrics and faulty conditions by evaluating the security requirements for the related environment. Coverage analyses are used in a variety of fault detection and test generations to evaluate behavioural designs. For example, hardware design validation and software tests often rely on such approaches. Some useful discussions on coverage analysis applied to specific fields can be found in Hayek and Robach (1996) and Moundanos et al. (1998). The chain of validations that describes relational risk dependence exhibits a hierarchical process flow. The relational risk management task considered here discusses the issues of requirements analysis, determining security objectives, policies and the implementation of required countermeasures (or just measures). Our decision model applies a simple realistic model to expose the relations between the elements of the entire security planning or solution. The quantification algorithm of the scoring scheme further uses the decision result in order to produce numerical values that describe the quality of the security planning. Fault detection is central to our model in which risk propagation is reduced by several stepwise decision processes. The evaluation process assumes a pre-defined attribute of security objectives, which is represented by a set of objective items. Thus, the decision hierarchy starts with assessing the objectives attribute, and continues similarly with the assessment of attributes of policy and the assessment of the implemented security measures. Analogous to the attribute of security objectives, pre-defined coverage metrics of both the policy and measures are also needed in order to complete the decision hierarchy.


347

One can also argue implementing security measures independently of any security objectives and policies. Unfortunately, this is possible and all other ad hoc approaches are possible too. However, we consider here only the systematic approaches that are needed to ensure the protection of information systems of any security classification. Ad hoc solutions can also serve well for very small environments and for protection of loosely classified systems. Nevertheless, this is an assessment model, which can guide decision making that conforms to a set of pre-defined criteria via a process of decision hierarchy. We term this decision making process shortly as assessment. The assessment is performed in four steps. First, an initial set of objective items are collected. The set is also referred to as objective attribute. The collection can merely be a result of a procedural evaluation based on questionnaires. The objective items are then evaluated against a pre-defined coverage metric and scored. The next step is the verification of the objective scores produced so far. During the verification, the score set is arranged in three sub ranges to denote, LD, D, and WD items. The evaluator may now prune the LD items in order to generate applicable policy items. This is necessary for keeping the number of initial faulty states as few as possible. One can also prune only a selected set of the extreme LD values instead of pruning the entire LD data set. The validation model is illustrated in Figure 2. Following a successful validation of the objective items, a set of policy items will be generated from the validated objective items. Similar to the verification of the objective items, the policy items are verified and the resulting LD scores of the policies are pruned if desired so. The final step is to validate the entire solution by generating the items for the countermeasures, which comply with the policy items. Note that, the generation of policy and countermeasure items is trivial where the items are selected from related coverage metrics that are already defined. Figure 2

The validation algorithm executes three processes sequentially, each producing input to its next process. If an undesired (NULL) state occurs the process will be terminated and the validation will be restarted

348

S. Kondakci

The final evaluation will result in LD, D, or WD. Indeed, we assess the final result to determine the quality of the solution obtained from a given set of objectives. Hence, the quality will either be loosely defined, defined, or well defined. The assessment of the final result can be based on a probabilistic approach. By this, we can quickly estimate the dominating sub range (quality), LD, D, or WD. The most weighed sub range will depict the characteristic feature of the solution. We assume that, the values of the sub ranges are binomially distributed. A binomial process has two outcomes, referred to as ‘success’ and ‘failure’. Thus, the probability of observing x LD values (successes) in n outcomes is  n Pr{LD = x} = Pr( x | n, pld ) =   ( pld ) x (1 − pld )( n − x ) , for x = 0,… , n  x

(1)

n n!  x  = x !(n − x)!  

(2)

where

and the size n is given by the sum of the sub range sizes n = sizeof (LD) + sizeof (D) + sizeof (WD).

The corresponding cumulative probability function is x  n F (LD) = F ( x | n, pld ) = ∑   ( pld )i (1 − pld )( n − i ) . i= 0  i 

(3)

The probability pld is the probability of success on each trial for the overall evaluation result that corresponds to the LD sub range. It can be specified as the highest probability for the overall evaluation result that corresponds to the LD sub range. Nevertheless, in most cases pld is empirically determined. The above equations are also applied to determining the probabilities Pr{D = x} and Pr{WD = x} by replacing the specific probability pld with pd and pwd, respectively. We can determine the overall evaluation result by calculating Max (F(LD), F(D), F(WD)).

(4)

This probabilistic approach might, however, require expensive computations because the probability distributions and the cumulative functions must be calculated for all values of x for each of the sub ranges. Alternatively, we can determine the overall system characteristics by use of stochastic ordering. In a stochastic ordering, X is said to be stochastically larger than Y, and denoted by X ≥ Y. st

Thus Pr{ X > x} = 1 − FX ( x ) Pr{Y > x} = 1 − FY ( x )

(5)


349

Pr{ X > x} ≥ Pr{Y > x}, ∀x , that is, do it for all possible values of x.

(6)

and

To complete the stochastic ordering, first we compute the required probabilities, i.e., Pr {LD = x}, Pr{D = x}, and Pr{WD = x}, using equations (1)–(3). Next, we run a chain of pairwise comparisons to determine the final evaluation. Hence, Step 1: Determine dominance of LD ∀x  Pr{LD = x} ≥ Pr{D = x}, if   then select LD. Pr{LD = x} ≥ Pr{WD = x}, ∀x

Step 2: Determine dominance of D Pr{D = x} ≥ Pr{LD = x}, ∀x  if   then select D. Pr{D = x} ≥ Pr{WD = x}, ∀x

Step 3: Determine dominance of WD Pr{WD = x} ≥ Pr{D = x}, ∀x  if   then select WD. Pr{WD = x} ≥ Pr{LD = x}, ∀x

The final evaluation, as mentioned earlier, will either be LD, D, or WD. We can prune a solution that gives undesired evaluation result, and rework a new solution by modifying either the objectives or the policies, or both of them. It is known that, a binomial experiment becomes a multinomial experiment if each trial produces more than two possible outcomes. Thus, obtaining x outcomes for LD, y z outcomes for D, and z outcomes for WD will occur with probability pldx , pdy and pwd . From here, the joint probability distribution of the random variables LD, D, and WD representing the number of occurrences for x, y and z is given by Pr(LD = x, D = y, WD = z ) =

n! z pldx pdy pwd x! y !z !

(7)

where x

y

z

pld + pd + pwd = 1,

and x + y + z = n.

For, example, in a single experiment, by using equation (7) we can compute the number of occurrences for LD values, D values, and WD values. Thus, the characteristic feature of the solution at hand or the assurance level of the system under evaluation is now determined.

2.1.1 A special approach: planning protection for critical systems Security of critical systems needs special caution. To be rather strict, we prefer not to proceed with the implementation if the organisation had not been able to provide

350

S. Kondakci

adequate security objectives. In this extreme case, we generate a security plan from scratch. To do so, we examine the general requirements to collect information on the existing assets and generate a set of appropriate objectives. The generation of objective set is completely dependent on the classification of the environment at hand. For example, the environment might be desired to comply with a higher protection level. In this case, we should classify the environment at least within a restricted category. It should be noted that, in general, we have unclassified, classified, restricted, secret, and top secret categories of security classifications. The following pseudo-code illustrates the assessment of a rather classified (strict) security solution. The approach treats the system owners by their knowledge. For the security-unaware system owners, we prefer running scenario-based simulations to illustrate various security solutions and their risks and protection potentials. For rather knowledgeable owners we can omit the simulation and just present some appropriate solutions that match the basic requirements. The assessment considers a valid set of security objectives in order to proceed. The code illustrates also the interpretation of assessment criteria together with the simulation algorithm used for decision making in the overall security planning. That is, the simulation algorithm can also generate successful security solutions.

This pseudo-code is simply a part of the prototype simulation, which we developed for the risk assessment of any security planning process. The prototype simulation is only an aid to security designers and management in decision making situations in which uncertainties dominate. In particular, the first phase of a security planning requires a detailed analysis of security objectives in order to determine appropriate security policies.


351

Obviously, the algorithm illustrates the flow of the hierarchical risk assessment process or evaluation of the strength of a given solution. It tries first to define (or generate) security objectives that are stored in the attribute SO_object. Each object, under consideration, contains an attribute that describes its parameters (called items). Notice that, the boolean function DefineAndVerify(SO_object) can check attributes of objectives and policies separately whenever needed. Upon a successful definition and verification of the security objectives, it tries to define and verify a security policy attribute that is stored in the security policy object SP_object. Likewise, upon successful definition and verification of the policy attribute, it proceeds with defining the security measures to implement the policy. As a final operation, after a successful definition of the security measures, it tries to evaluate the required security measures. The verification of the attributes should not be intermingled with a binary decision making. The verification at first determines a set of quantitative values used to score the attribute, which is in turn evaluated to decide whether a further action in the decision hierarchy will be taken or the assessment will be interrupted. As illustrated in Figure 2, when a test validates to NULL, the solution will be discarded. For example, Ob = DefineAndVerify(SO_object) returns true if the verification finds that the value of SO_object is in the range of defined or well defined scale. Otherwise, it returns false and discards the next operation. This is necessary because one cannot proceed to build security measures if there have not been defined any security policies for the environment under consideration. Finally, the designed security measures are evaluated, and upon failure of the evaluation, a new alternative of security planning is generated and reassessed.

3

Risk propagation model of security planning

The model discussed here will assess risks and determine the efficiency and statistical characteristics of a security planning process. Via the statistical inference, we will try to judge whether the observational data would pertain to a certain probability model. The model will certainly fit a hypergeometric distribution. Hence, we apply the hypergeometric distribution to estimate the tendency of the strength of the initial security planning. The study of the public awareness is required in order to produce statistical data that can be used to justify the validation model and the risk propagation model presented in this paper. In an effort for the analysis of the public awareness of efficient security planning, we envisioned two major parts to the assessment: quantitative (scoring and probabilistic) evaluation approaches and a set of empirical data gathered from a statistical survey. The survey is required to draw an overall picture of the public tendency, hence, to study relations between the overall picture and the assessment approaches discussed earlier. The survey, Kondakci (2003), has investigated security planning tendencies, long-term (or lifecycle) security planning, testing and evaluation cultures and habits, dependence on trademark offerings, and dependence on qualitative assessment arguments. The research results show that most of the quality arguments were only based on sector rumours. The leading companies, A, B, or X, for that matter, were the most considerable references and reasons of the less knowledgeable organisations to invest in that specific ad hoc solution. This is an important fact of the public picture reflecting the general approach to security solutions of the IT community in Turkey, which has a relatively large population of IT and internet consumer among the

352

S. Kondakci

developing countries. Figure 3 illustrates the risk propagation model starting with the identification of security objectives ending with the implemented security mechanisms. As shown in Figure 3, each circle identifies the union result achieved at the related phase of the security planning process at hand. The sizes of the sets (circles) are gradually decreasing. Figure 3 also illustrates the fact that the implementation of security mechanisms will hardly reflect all of the protection mechanisms indicated in the security policy. Ideally, we can choose (or generate) a random set of solutions to determine the statistics. To be rather realistic, we use a set of organisations to simulate the population. This model can be conveniently applied to a given set of security attributes or solutions, such as objectives, policies, and implementations. Considering this assumption, for example, organisations with well defined security objectives will represent a given set of well defined security objectives. Figure 3

The risk propagation model. Relationships of the processes are expressed in probabilistic factoring. The factors represent the risks that are propagated from the previous process

Suppose that Ω = {O1, O2, …, ON} is a defined set of mutually exclusive security objectives events extracted from a given random population of organisations as modelled in Figure 3, such that at least one of the required objective events from this set always occurs. Recall that, security objective is a piece of data showing the overall knowledge of the organisation. An organisation with a good security objective has a full coverage over all existing assets, knows the functions and importance of the assets, risks, and impacts of the risks. A number of organisations, Zo = n(O), having defined the security objectives is derived from a given number of organisations, n(R). Further, from n(O) the number of organisations having appropriately defined their policies is given by Zop = n(O … P). In turn, Zm = n[O … (M … P)] depicts the number of organisations having defined security measures in accordance with the respective policies. It is obvious that, the quality of any security policy P is dependent upon the existence of the defined or well defined security objectives O. The implementation of any security measure M is dependent upon the existence of a defined or well defined security policy.


353

That is, the better the security objectives the stronger the security solution. Thus, the probability Pr(P) assumes (or depends on) the occurrence of O and Pr(M) assumes the occurrence of P. Hence, the conditional probability of P given O, Pr(P|O), is expressed by Pr( P | O) =

Pr( P ∩ O) Pr(O)

(8)

and the conditional probability of M given P, Pr(M|P) is expressed by Pr( M | P ) =

Pr( M ∩ P ) . Pr( P )

(9)

Regarding the survey results, that are discussed in Appendix A, the user awareness of information security exhibits a hypergeometric probability distribution Pr(O = o) given by  N  R − N   R Pr(O = o) =       , max(0, n + N − R) ≤ o ≤ min(n, N ). o  n − o   n 

(10)

The probability is positive, when o is between max(0, n + N – R) and min(n, N). Specifically, our sample population consists of R organisations that are divided into two groups according to those who possess security objectives, denoted by N, and those who do not, denoted by R – N. Recall that N ⊂ R. We then let O denote the number of organisations having defined their security objectives in the random sample space of n organisations as depicted in the sampling scheme shown in Figure 4. The number of successes, i.e., the number of organisations with security objectives, selected in this sample is denoted by o. It is also observed that with large size of samples the distribution is binomial, which is a well-known statistical fact. In order to compute the hypergeometric probability distribution of variable O by equation (10), the sampling scheme selects o elements from the subset of N and selects n – o elements from the subset of R – N, respectively. Figure 4

A sampling scheme to estimate the population tendency of a random population of size N. We choose n solutions, where o are security-aware and n-o are security-unaware

354

S. Kondakci

Similarly, events from a defined or well defined security policy set P = {P1, P2, …, PM} will only occur under the condition that at least one O event has occurred. This joint relationship is illustrated in Figure 3. First, we define the probability distribution for organisations that have defined both of the Objective (O) and Policy (P) attributes. Via these we will first calculate Pr(Y = y) for the y defined security objective items selected from set n(O) as shown earlier in Figure 3. Recall that, the conditional probability Pr(P|O), the probability of P policy events dependent on the occurrence of O objective events, is given in equation (8). The probability Pr(Y = y) is given by  Yp   N − Yp  Pr(Y = y ) =      y  p − y 

N   , max(0, p + Yp − N ) ≤ y ≤ min( p, Yp ), p

(11)

where Y is the hypergeometric random variable for which the number of y successes is being computed, N is the new population size adhering to a set of security objectives (O), Yp is the number of compound events composed from the objectives and policy set, i.e., set (O … P), p is the new sample size under evaluation. Thus, as illustrated differently in Figure 3, the probability distribution Pr(P = p) for the events n(O … P), which have also defined P policy attributes, can be calculated by if Pr(O = o) = 0 0, Pr( P = p − y ) =  Otherwise Pr(Y = y ).

(12)

Referring to the hierarchical flow given above, the hypergeometric probability distribution of security measures can be expressed as  Z m   Z op − Z m  Pr( Z = z ) =     z   m − z 

 Z op   m  , max(0, m + Z m − Z op ) ≤ z ≤ min(m, Z m ). (13)

Hence, the final probability distribution for the events [O … (M … P)] reflecting the probability distribution of the security measures is given by 0, if Pr( P = p ) = 0, Pr( M = m − z ) =  Otherwise Pr( Z = z ).

(14)

Where M is the new population derived from the set (O … P), Zm is the number of compound events computed from the objectives, policy, and measures set, i.e., [O … (M … P)], m is the new sample size under evaluation, and Z is the hypergeometric random variable for which the number of successes z is being computed. This depicts the number of organisations having successfully implemented their security measures. This analysis is closely discussed in Appendix A in order to justify the verification and risk propagation models.

4

Summary and conclusions

Generally, we have an intuition that knowledge and its proper usage are the key aspects that cause serious impacts on engineering processes. However, impacts and risks in information security are owing to many factors, which are human, external (unknown), and technical factors. Intuition and qualitative approaches are not adequate to guide


355

accurate risk analysis in information security. In this paper, we have presented a rather simple approach to guide the risk analysis quantitatively. Consequently, the results of our research are practically applicable in the evaluation of information security, security development projects, and multidisciplinary projects as well. Analysis of security requirements (or objectives), design of security policy and countermeasures are all conditionally dependent aspects that can be best described by a hierarchy of joint processes. We have shown that, loosely defined security objectives will lead to loosely implemented security solutions. We have mainly divided the security planning process into three phases, where each phase was quantitatively evaluated. To pinpoint the weaknesses encountered in each phase, the evaluations were necessary in order to eliminate the propagation of the design faults. We have numerically determined the propagation of the design faults by use of a probabilistic method and a scoring scheme. With the scoring scheme we were able to classify assets and assign grades to weaknesses and risks for the assets under evaluation. To simplify the determination of risks encountered for each asset, we needed to classify the assets by assigning a weight to each asset. The numerical weight of the asset depicts indeed the importance (or the classification level) of the asset. We have also presented some statistical inferences about human-related risk factors that are reflected onto security solutions. We have applied some hypotheses tests to justify the power of the statistical inferences. The crucial fact is that the internet is growing sporadically with drastically evolving threat space that span over many dimensions. Security tests conducted to evaluate operational effectiveness and suitability in local environments and to identify any operational deficiencies or needs for modifications are vital for the globally interconnected IT environments. Unfortunately, it is evidential that, a very small portion of the organisations from the sample survey in Turkey performs regular (or any at all) security test and evaluations. This might also be a fact in other underdeveloped and developing countries. Turkey as a massive IT consumer has even no government regulation on IT policy, usage, and security test and evaluations. It was shown that regular security test and evaluations were necessary for the government organisations, and indeed for all others, to improve the security-awareness, hence to strengthen the security measures. Because Turkey and eventually other loosely protected regions take part in the global internet connectivity, they should conduct the necessary operations that could withstand the threats directed from them into the internet. Most obviously, to achieve the requirements of information assurance conforming to the international standards and practices, we have a rather far distance to go.

References Alberts, C.J. and Dorofee, A.J. (2002) Managing Information Security Risks: The Octave Approach, ISBN:0321118863, Addison-Wesley, Harlow, UK. Anderson, D.R., Sweeney, D.J. and Williams, T.A. (2005) Statistics for Business and Economics, ISBN: 0-324-20082-X, South-Western College Pub, Mason, OH, USA. Balakrishnan, N. and Rao, C. (2001) Handbook of Statistics 20: Advances in Reliability, ISBN: 0-444-500078-2, North Holland. Bedford, T. (2003) Probabilistic Risk Analysis: Foundations and Methods, ISBN: 0-52177320-2, Cambridge University Press, London, UK. Bruske, S.Z., Wright, R.E. and Geaslen, W.D. (1985) Potential Uses of Probabilistic Risk Assessment Techniques for Space Station Development, NASA STI, USA.

356

S. Kondakci

CIS (2006) The Center for Internet Security, www.cisecurity.org, accessed 2006. COBIT (2006) Control in Business Objectives for Information and Related Technology, V. 4.0, http://www.isaca.org, accessed 2006. COBRA (2006) Consultative Objective and Bi-functional Risk Analysis: Tools, ISO/IEC 17799 Compliance and Security Risk Analysis Approach, http://www.bspsl.co.uk/17799/, accessed 2006. Common Criteria (CC) (2006) Common Criteria and the Common Evaluation Methodology, V. 3.0, http://www.commoncriteriaportal.org/, accessed 2006. Even, T. (2004) ‘Risk analysis and science’, International Journal of Reliability, Quality and Safety Engineering, Vol. 11, No. 1, pp.1–15. Everitt, B. (1999) ‘Chance rules: an informal guide to probability’, Risk and Statistics, Copernicus. Gallegos, F., Manson, D.P., Senft, S. and Gonzales, C. (2004) Information Technology Control and Audit, 2nd ed., ISBN: 0849320321, Taylor & Francis CRS Press LLC, FL, USA. GAO/AIMD-00-33 (1999) Information Security Risk Assessment: Practices of Leading Organizations, United States General Accounting Office (GAO), http://irm.cit.nih.gov/itmra/ gaoguid.html, accessed February 2006. Grimmett, G. and Stirzaker, D. (1992) Probability and Random Processes, 2nd ed., Oxford University Press, New York. Hayek, G.A. and Robach, C. (1996) ‘From specification validation to hardware testing: a unified method’, International Test Conference, pp.885–893. Herrmann, D.S. (2002) Using the Common Criteria for IT Security Evaluation, ISBN:0849314046, Taylor & Francis CRS Press LLC, FL, USA. ISA (2006) The Internet Security Alliance, www.isalliance.org, accessed 2006. ISACA (2006) Information Systems Audit and Control Association: ISACA, http://www.isaca.org/, accessed 2006. ISC2 (2006) International Information Systems Security Certification Consortium: (ISC)2, www.isc2.org, accessed 2006. ISO/IEC:FDIS 15408-1 (2005) ISO/IEC, FDIS 15408-1: Information Technology – Security Techniques – Evaluation Criteria for IT Security, http://www.gammassl.co.uk/ist33/, accessed 2005. ISSA (2006) Information Systems Security Association: ISSA, http://www.issa.org/, accessed 2006. Kondakci, S. (2003) ‘Controlling security risks in large computer networks’, International Journal of Computational Intelligence– ICSP’ 2003, ISSN 1304-2386, Vol. 1, No. 2, pp.7–10. Koskosas, I.V. and Paul, RJ. (2003) ‘A socio-organizational approach to information systems security risks’, International Journal of Risk Assessment and Management, Vol. 4, Nos. 2–3, pp.232–244. Kumamoto, H. and Henley, E.J. (2000) Probabilistic Risk Assessment and Management for Engineers and Scientists, 2nd ed., ISBN: 0-7803-6017-6, Wiley-IEEE Press, London, UK. McCumber, J. (2005) Assessing and Managing Security Risk in IT Systems: A Structured Methodology, ISBN: 0849322324, Taylor & Francis CRS Press LLC, NW, FL, USA. Moundanos, D., Abraham, J.A. and Hoskote, Y.V. (1998) ‘Abstraction techniques for validation coverage analysis and test generation’, IEEE Transactions on Computers, Vol. 47, pp.2–14. NIST (2006) NIST Frequently Asked Questions On ISO/IEC 17799:2000 for Information Security Management, http://csrc.nist.gov/publications/secpubs/otherpubs/reviso-faq.pdf, accessed 2006. Peltier, R.T. (2001) Information Security Policies, Procedures, and Standards: Guidelines for Effective Information Security Management, ISBN: 0849311373, Taylor & Francis CRS Press LLC, NW, FL, USA. Peltier, R.T. (2005) Information Security Risk Analysis, 2nd ed., ISBN: 0849333466, Taylor & Francis CRS Press LLC, NW, FL, USA.


357

SANS (2006) The SANS (Systems Administration and Network Security) Institute, www.sans.org. Tipton, H.F. and Krause, M. (2003) Information Security Management Handbook, 5th ed., Vol. 2, ISBN: 0849332109, Taylor & Francis CRS Press LLC, NW, FL, USA. Vose, D. (2000) Risk Analysis: A Quantitative Guide, ISBN: 0-471-99765-X, J.Wiley & Sons Ltd, London, UK. Walsh, J. (2002) Asset Protection and Security Management Handbook, ISBN: 0849316030, Taylor & Francis CRS Press LLC, NW, FL, USA.

Appendix A A.1 A case study of correlation between awareness and implementation Human related risk factors depend often on organisational cultures. Some organisations prioritise the systematic definition of policies for IT and IT security, while others find them unnecessary. They merely rely on ad hoc safeguards based on trademark rumours or ‘de facto’ solutions. We have randomly chosen 100 government and 100 private organisations to study their security solutions consisting of only the major security aspects integrity, confidentiality, and availability. Table 1 summarises the evaluation data for a network of 20 assets. For this statistical study, the data are aggregated in three entries (rows), i.e., a triplet of , for each asset under evaluation. That is, we evaluate each asset by its Objective (o), Policy (p), and Implementation (i) items. Hence, the entries of the triplets will be the final evaluation scores varying between 0 and 5 (5 being the highest score). From these per-asset evaluation values, an overall Average Score (AS) is computed for each of the rows given in Table 1. For example, if a network contains 20 assets, then, 60 values are aggregated, 20 for each entry of the triplets. The number of overall average scores is naturally three for the entire evaluation of 20 assets, e.g., = . Here, 3.04 gives the quality of the objectives, 2.68 gives the quality of the policy, and 2.55 gives the quality of the final implementation. Table 1

Summary of the result of a sample evaluation of 20 assets Asset no. 2

3

…

20

AS

O

3

0.8

5.0

…

3.8

3.04

P

2.8

0

5.0

…

2.5

2.68

I

2.5

1.0

–

…

3.8

2.55

Score

1

Here, the entire network is graded with a triplet of average scores, i.e., AS = = . As seen, the average score of the implementation is decreased. This is owing to the reflection of weaknesses from the objectives onto the policies, and the weaknesses propagated from the policies to the implementation. Table 2 summarises the results of this master study. As shown in Table 2, objective values (%O) are out of 100. The entries of the other columns are as explained below.

358

S. Kondakci

Table 2

Security-awareness of 100 government and private organisations Government organisations

Item

%O

O…P

Private organisations

P…M

%O

O…P

P…M

Integrity

16

12

5

58

48

20

Confidentiality

82

65

48

70

60

41

Availability

45

38

26

81

72

58

Here, %O represents the percentage of organisations having defined their objectives, O … P represents the number of organisations with defined policy, and which have also defined their objectives. Likewise, P … M gives the number of organisations with adequate countermeasures, which have defined their policies. Thus, there are only 16 government organisations that are aware of the integrity. Out of these 16 government organisations, 12 have determined security policies and 5 of the 12 organisations have presented adequate security measures. Note that, the confidentiality has gained higher respect among the respondents, because many of them think that security can be only obtained by encryption. In particular, considering the confidentiality, some of the government and defence organisations invest more in the cryptography and encryption equipment. Availability is respected higher among the private companies, because a significant number of them run businesses like finance and banking, logistics and transportation, and internet service providing. Naturally, for such operations maximum availability is always the major requirement. The results are illustrated in Figure 5. As seen, the trend is similar to the propagation model illustrated in Figure 3, where the strength of the policy and countermeasure are decreasing. This justifies the model of conditional relationships between the objectives and policies and between the policies and implementations. Furthermore, regarding the fault propagation, the design verification model shown in Figure 2 produces results that correspond densely with the survey results. Figure 5

Effect of objectives on policy and effect of policy on countermeasures

The propagation factors that are shown on the chart are 0.86, 0.66, 0.82 and 0.68. The first factor, 0.86, depicts the propagation factor from objectives (security awareness) to the policy design, the second factor, 0.66, depicts the propagation factor from the policy design to the final security implementation for the private organisations. The corresponding factors for the government organisations are 0.82 and 0.68. It should


359

be noted that, these factors are specific to the current experiment. The factors are used to determine the quality (or strength) of the solution at hand. For example, a perfect security objective attribute scored 5.0 will lead to a reduced policy attribute score of 4.3. That is, 0.86 × 5.0 = 4.3. The implementation score will be 0.66 × 4.3 = 2.84, which shows a dramatic decrease in the strength of the final implementation. We have run several experiments on the collected survey data and calculated the distributions. We found that the propagations were hypergeometrically distributed. Experimentally determined propagation factors had shown very small deviations from the population values shown on the chart (Figure 5). It can be assumed that the awareness on the security objectives is binomially distributed. Nevertheless, this initial distribution model has no impact on the posterior distribution of the propagation. Because, in general, security designers do not make choices based on numerically estimated successes of the defined objectives. Most of the designers define the necessary policy intuitively. For the survey data, we calculated the probability distributions of the security objectives and measures. Both the hypergeometric and their cumulative distributions of the evaluation data for the security measures and corresponding objectives are depicted in Figure 6. Figure 6

Probability distributions and relations of security objectives and implementations for 20 government and private organisations (a) objective and implementation distributions and (b) cumulative objective and implementation

(a)

(b)

360

S. Kondakci

Figure 6(a) shows the distributions, and Figure 6(b) shows the same distributions in a cumulative form. It should be noted that, these values get quickly closer with better coverage of security-awareness. The government organisations exhibit larger correlation between the objectives and final countermeasures compared to the private organisations. Security-awareness among the government organisations is significantly low. On the other hand, the private sector exhibits a narrower variance and more knowledgeable companies. Figure 7 illustrates the correlation between the objective and implementation evaluations. Figure 7

Correlation between the security-awareness and the implemented countermeasures

By intuition and the figures shown above, it is obvious that the more the security awareness the better the security solutions. First of all, system owners should be aware of the importance of information security so that they can efficiently perceive threats and eventual countermeasures. Otherwise, security providers do often install their own products and solutions, without much attention to the actual requirements of the owners. Especially, in Turkey security solution providers, or in general IT providers, consider their tasks like a boutique business rather than the technology and service provision. Hence, most of the IT providers do not employ professional staff, but marketing staff. Some of the respected companies and some critical government organisations (e.g., defence department) outsource their IT and security projects to providers in foreign countries for incredibly high costs. Of course, the expenses are covered by very high taxes taken from the registered business and manpower. This is a serious problem causing negative cash flow from a developing country like Turkey, which is already head over heels in debt. Many of the developing and/or underdeveloped countries do not possess legal technology policies and standards to guide their IT communities. Besides, with this IT policy (or non-policy), the uncovered impact will continue to expand. Naturally, the ultimate impact is such that the necessary know-how is never going to be achieved nor improved.


361

A.2 Justification of assessment and risk propagation models In Section 2, we have presented the assessment and improvement model for security solutions (see Figure 2). In Section 3, we presented the risk propagation model. These two models are commensurate with each other. The assessment model validates a given solution and guides the security designer for improvements. The risk propagation model illustrates the risk factors by use of the probabilistic approach. In this section, we give further details to justify the methodologies presented so far. For the evaluation of security items, we apply Bernoully trials to determine the entry-level distributions. Each evaluation process produces a set of triplets called evaluation scores, i.e., as defined earlier. To estimate a distribution from this set, we choose a sample set of size n, which is proportionally large enough compared to the overall population size. Recall that, we have divided the entire measurement scale into three equal categories (sub ranges), i.e., LD = {0, …, 1.66}, D = {1.67, …, 3.33}, WD = {3.34, …, 5}. Here, LD represents loosely defined, D defined and WD well defined solutions. Note that, the individual solution can be any of the security-planning phases. The Bernoully trial we applied here is also useful in determining the category of a given security attribute. For example, we can determine the quality of an objective or policy item; whether characterised as loosely defined, defined, or well defined. The Bernoully method is specifically time-efficient when a large set of assets will be evaluated. The idea is actually similar to quality evaluation of a mass production line. For this, we do not need to evaluate hundreds of assets; instead we can choose a subset of them to determine the overall security category of the large network. Example: Consider a sample evaluation of n organisations or solutions. A sample evaluation of this kind is already tabulated in Table 1. We choose, first, only objectives from the table (first row) one by one and evaluate each of the objective scores. What is the number of evaluations having scores less than or equal to x? Let us assume x ≤ 3.33, which represents the highest score for the category of defined objectives. Then, we select the policy scores of the corresponding entries (second row) and evaluate them to determine whether the objective and policy evaluations correlate. Further, we chose the implementation scores of the corresponding entries (third row) and evaluate them to determine whether the policy and implementation evaluations correlate. Note that, we repeat the experiment n times independently. We measure the score of the ith objective in ith trial, i.e., 1, X i ≤ x i = 1, 2,..., n 0, X i > x,

ζi = 

(15)

where Xi denotes the score of the ith object. Then, it is obvious that Sn = ζ1 + … +ζn is the number of security objectives for which the score does not exceed the value of x. Apparently, we have a Bernoully trial with success probability of P ≡ P{ X ≤ x} = F ( x), 0 ≤ x ≤ 1,

(16)

 n P{Sn = i} =   ( F ( x))i (1 − F ( x)) n −i , i = 0, 1, 2, … , n. i 

(17)

362

S. Kondakci

Thus, the probability that the number of security objectives, y, for which the score does not exceed the value of x is y n P{Sn ≤ y} = ∑   ( F ( x))i (1 − F ( x)) n − i . i =0  i 

(18)

Basically, F(x) is a distribution function on [0, 1]. In a special case, F(x) may be uniformly distributed, i.e., 0, x < 0  F ( x) =  x, x ∈[0,1] 1, x ≥ 1, 

(19)

or it can take the form 0, x < 0  F ( x) =  x 2 , x ∈ [0,1] 1, x ≥ 1, 

(20)

and so on. Now, considering the data from Table 1, we have n = 20. Formerly, it was empirically found that the probability of a defined objective was 0.48. Based on these values we can estimate the probability of defined or loosely defined objectives in a new trial. Thus, for the uniform distribution, the probability f(x) that 10 defined or loosely defined objectives are found in the sample of 20 objectives is 0.17, and the corresponding cumulative distribution, F(x), is 0.66. Figure 8 shows the plots of the functions f(x) and F(x). Figure 8

Distribution of objective evaluations for n = 20, p = 0.48


363

Furthermore, we have drawn out at random 12 entries from each row of Table 1 to compute the hypergeometric distributions of the objectives, policies, and the implementations for x < 3.34. Equations for the hypergeometric distributions are given in Section 3, equations (10)–(14). Recall that, the values between 0 and 3.34 cover the sub ranges Loosely Defined (LD) and Defined (D). The security items graded between 3.34 and 5.0 are Well Defined (WD). Figures 9 and 10 show the probability, f(x), and the cumulative probability, F(x), of each experiment. Figure 9

The probability functions of loosely defined or defined objectives and their effects on policies and implementations

Figure 10 The cumulative probability functions of loosely defined or defined objectives and their effects on policies and implementations

For the overall analysis of correlations, 20 items from each data set have been assessed, i.e., objective, policy, and implementation. Figure 11 shows the correlations between objective scores, and policy scores. Figure 12 shows the correlation between the policy scores and the implementation scores. As shown in the scatter diagrams, we have higher correlation between the objective items and policy items. Though the correlation between the policy and implementation items is not very high, we can easily see the effect of the policy reflected onto the final implementation. This discussion elaborates the facts that design faults propagate through the steps of the security planning processes if not intervened using the assessment and validation methodology given in Section 2.

364

S. Kondakci

Figure 11 Correlation between objective and policy scores

Figure 12 Correlation between policy and implementation scores

A.3 Power test One of the major socio-cultural findings was that, private organisations were more security-aware than that of the government organisations. Establishing a hypothesis test that, in turn, will compare two population proportions, given as p-values, can prove this conjecture. In conjunction with this, useful inferences can be drawn about the population on the basis of the proportion parameters p and pˆ. We know sample sizes of both populations (government and private), but not the population standard deviations. We can not therefore calculate standard error accurately. On the other hand, applying the hypothesis test together with a power test, validity of our conjecture can be verified with higher accuracy. We can choose between two hypotheses, H0 and H1, in the decision making process. Thus as for the test, we have H 0 : p po = pso , H1 : p po > pso .


365

We will try to reject the null hypothesis H0 in favour of the alternative hypothesis H1 if the assumed decision rule can show its power supporting the conjecture. In the process of hypotheses testing we rely on the survey data summarised in Table 1. We will thus consider a standard model with a couple of independent random samples of nso and npo observations with their associated proportions pso and ppo. Here, the variables are subscripted with so for the government and with po for the private organisations, respectively. We assume that proportions can be approximated as normally distributed random variables, which is, by the Central Limit Theorem, true for large samples. During the test, we also assume α = 0.05 as the significance level. The value of α can be changed from the most popular value 0.05 in order to observe different power functions. Detailed statistical formulae are given in various statistics textbooks, e.g., Balakrishnan and Rao (2001), Anderson et al. (2005) and Grimmett and Stirzaker (1992). Hence the decision rule will reject H0 in favour of H1 if Z=

pˆ po − pˆ so ( pˆ 0 qˆ0 / n po ) + ( pˆ 0 qˆ0 / nso )

> Zα

(21)

where Zα = Z0.05 ≈ 1.645 is obtained from the standard normal table, and pˆ 0 is given as pˆ 0 =

nso . pˆ so + n po . pˆ po nso + n po

.

Figure 13 depicts power functions of varying sample sizes of organisations. Figure 13

Power curves for test of H0 : pso = ppo vs. H1 : ppo > pso

(22)

366

S. Kondakci

The power of the test at the sample difference of proportions for 100 government and private organisations each, i.e., ∆ = pˆ po − pˆ so = 0.212 (or 21.2%) equals 92%, and the power equals to 50% when the population proportion difference is about 11.63%. We achieved the expected result, the larger the sample size, the greater the power of the test.