A Practical Approach to Multi-Hazard Risk Assessment

Section on Risk Analysis – JSM 2010

A Practical Approach to Multi-Hazard Risk Assessment 1

2

Xunguo Lin , Richard G. Jarrett 1 CSIRO Mathematics, Informatics and Statistics, GPO Box 664, Canberra, ACT 2601, Australia, E-mail: [email protected] 2 CSIRO Mathematics, Informatics and Statistics, Private Bag 2, Glen Osmond, SA 5064, Australia, E-mail: [email protected] Abstract The aim of this paper is to provide a broad overarching structure for risk assessment which is practically useful, as objective as possible and which avoids logical inconsistencies. While much has been written about the potential flaws in the risk assessment process, a strong, quantitative and disciplined approach to risk assessment can not only remove some of the problems, but can also bring the remaining problems into sharper focus. This implies that effort can be directed towards ensuring that these remaining important issues are given the attention they deserve. In particular, the paper proposes that both consequence and likelihood scores can be determined respectively as logarithms of objective measures of consequence (in terms of cost or surrogates of cost) and rate of occurrence over a time and spatial scale appropriate to the situation. These numerical scores can be added to provide a risk score, which can easily be mapped onto an expected cost. The proposed assessment of consequence and likelihood is rooted in values which, in theory at least, can be supported or validated by data. Such a formalism does not pretend that individual hazards have consequences and likelihoods that can be accurately determined. Rather, it provides a risk matrix which is unambiguously defined and accepts that individual hazards placed within it will have levels of uncertainty associated with their exact position, due either to lack of data or to varied opinions among experts. Within this context, the paper offers a number of innovations. Consequences often relate to a variety of categories, for example economic, environmental and social aspects. A consequence matrix, describing the severity (rows) for different categories (columns) of consequences, is proposed and a new methodology provided to produce an overall consequence score for each hazard assessed. Likelihood is carefully defined as a rate which applies to a particular region and time scale. Risk can then be defined in an unambiguous way. If a number of hazards are identified and assessed in each of a number of regions, the methodology allows for the combination of hazards to determine the risk associated either with all hazards for a particular region, or with a particular type of hazard across all regions.

Key Words: Quantitative risk assessment, risk matrix, AS/NZS 4360

1. Introduction The Risk Matrix approach used in risk assessment is a table that has categories of "severity/impact/consequences" for its rows and categories of "probability/likelihood/

456


frequency" for its columns. Identified hazards are then placed within that Risk Matrix by choosing the appropriate row and column for that hazard. A typical Risk Matrix is shown in Table 1. The Australian and New Zealand Standard on Risk Management (Australian Standard, 2004) was first developed in 1995, and has an accompanying Handbook (Handbook 436, 2004) which contains such a Risk Matrix and extensive discussion about how to use it. The need and the background for such a standard is described by Cross (1995), together with a summary of the risk management process. Risk itself is determined by the combination of consequence and likelihood, with the result that gradations of risk are determined by essentially diagonal contours in the Risk Matrix, as shown by the shading in Table 1. Here, we follow Swallom (2005) in arranging the categories so that increasing consequences and likelihoods correspond to moving up and to the right, so that the table is arranged in the same way as the corresponding graph. Anderson (2006) claims to have identified more than 800 examples of Risk Matrix through a search on Google, including many different versions of the matrix. Table 1: A General Risk Matrix → Likelihood → ↓ ↓Consequences↓

IMPROBABLE REMOTE OCCASIONAL PROBABLE FREQUENT (E) (D) (C) (B) (A) VERY HIGH

CATASTROPHIC(4) HIGH

CRITICAL(3) MEDIUM

MARGINAL(2) NEGLIGIBLE(1)

LOW

There have recently been criticisms levelled at Risk Matrices (Cox, 2008a) in relation to their suitability and reliability in adequately ranking the risk of a variety of hazards. Many of these criticisms related to a certain degree of arbitrariness in the definition of the scales for consequence and likelihood. In Section 2, a more explicitly defined Risk Matrix is proposed, in which categories for both consequence and likelihood are scaled in, usually equal, logarithmic steps. This provides a standardised and well-defined measuring stick against which individual hazards can be judged. There is no suggestion that this implies greater accuracy in the assessments; it is aimed rather at reducing the uncertainty that may arise from the use of more subjective scales, and thus restricting uncertainty to our knowledge (or lack of it) about the hazards. Consequences often arise in a variety of ways, many of which are not expressible in purely monetary terms. Suggestions have been made, such as in Handbook 436 (2004) to identify a number of consequence categories and then produce a Consequence Matrix which, in each row, provides descriptions appropriate for that consequence score in each category. Section 3 enlarges on this approach. Likelihood can also be expressed as a likelihood score that moves by 1 unit for each 10fold change in likelihood. It is proposed in Section 4 that probability be replaced by rate of occurrence, in which case the likelihood score can be extended arbitrarily in either direction.

457


The use of an overall consequence score and a likelihood score to calculate a risk score is given in Section 5. Section 6 provides a method to combine different hazards in different regions and discusses important issues such as possible dependence between consequence and likelihood. In Section 7, the conclusions of this paper are given.

2. The Proposed Risk Matrix Several researchers, including Swallom (2005) and Anderson (2006), have argued for the use of logarithmic scales in both the rows and columns of the Risk Matrix so that successive categories on each axis of the matrix correspond to an order of magnitude increase in consequence or likelihood. This approach has been used by the authors on a number of quantitative risk assessments in recent years. Table 2: The Risk Matrix with "Logarithmic" Scores

Consequence Score Catastrophic 10 [9.5, 10.5) Major 9 [8.5, 9.5) Moderate 8 [7.5, 8.5) Minor 7 [6.5, 7.5) Insignificant 6 [5.5, 6.5)

-4 [-4.5, -3.5)

Likelihood Score -3 -2 -1 [-3.5, -2.5) [-2.5, -1.5) [-1.5, -0.5)

0 [-0.5, 0.5)

Medium 6 [5, 7)

High 7 [6, 8)

Very High 8 [7, 9)

Extreme 9 [8, 10)

Extreme 10 [9, 11)

Low 5 [4, 6)

Medium 6 [5, 7)

High 7 [6, 8)

Very High 8 [7, 9)

Extreme 9 [8, 10)

Very Low 4 [3, 5)

Low 5 [4, 6)

Medium 6 [5, 7)

High 7 [6, 8)

Very High 8 [7, 9)

Negligible 3 [2, 4)

Very Low 4 [3, 5)

Low 5 [4, 6)

Medium 6 [5, 7)

High 7 [6, 8)

Negligible 2 [1, 3)

Negligible 3 [2, 4)

Very Low 4 [3, 5)

Low 5 [4, 6)

Medium 6 [5, 7)

Table 2 provides an example with logarithmic scales (based on powers of 10), and categories based on integer steps. For example, if cost is measured in dollars, then a consequence score of 8 would correspond to a consequence of $108 = $100M, and would be considered to cover consequences that were thought most likely to fall in the range [107.5, 108.5) = [$30M, $300M), where the square bracket implies inclusion of the startpoint in the interval and the round bracket the exclusion of the end-point. It is assumed that events placed beyond either end of the scale are included in the most extreme category. Generally, hazards would be provided with integer consequence scores, but more precise estimates could be used if thought appropriate. Similarly, if likelihoods are assessed on an annual basis, then a likelihood score of 0 represents events that occur on average 100 = 1 time per year, with a range [10-0.5, 100.5) = [0.3, 3) times per year. Note that there is an important distinction between probability, which is confined to lie between 0 and 1, and the representation here of a rate which can take any non-negative number. Further, since the rate of events may not be constant over

458


time, a time scale appropriate to the situation should be chosen. In the simplest case of a Poisson process, a change in time scale would just shift all likelihood scores up or down by the log of that change. For example the likelihood scores for a rate per month would be log(1/12) = −1.08 relative to those for an annual rate. The risk score is the sum of the consequence score and the likelihood score. It has a natural interpretation as the log of the Risk or Expected Cost per year associated with a particular hazard, where Risk = Expected Cost = Consequence × Likelihood. For example, if the likelihood were a rate of 0.1 events per year and the consequence were $C, then, on average, for nine years out of 10 the cost would be $0, while one year in 10 the cost would be $C. Averaged over the 10 years, the cost would be $0.1C per year, which is referred to as the Expected Cost. Expected Cost is related to an insurance premium, without the margin to cover costs/profits. Cox (2008a) argues that Risk Matrices should fulfil a number of key criteria based on their ability to effectively and reliably order the risk of the various identified hazards. He demonstrates that, when row and column categories are based on a subdivision of the interval [0,1] and risk is the product of these two numbers, these criteria may not be fulfilled, thus rendering the risk matrix inefficient at determining appropriate risk priorities. The formulation above fulfils the requirements of Cox (2008a) for weak consistency and betweenness, provided the risk levels used have step sizes of at least 2 between them. For example, if the highest risk level were defined as any cell where the central risk score is 8 or 9, and the next risk level had a central risk score of 6 or 7, and so on, then no cells with risk levels two steps apart are contiguous, thus fulfilling Cox's definition of betweenness. The top risk level can have risk scores as low as 7, and the third highest risk level (central values 4 or 5) can have scores as high as 6, while the risk level between them, which can take values in the interval [5, 8), has points with scores both higher than the smallest of the level above (7) and lower than the largest of the level below (6), thus fulfilling Cox's definition of consistent colouring. Cox (2008a) concludes that only two possible colourings of the 5 x 5 risk matrix satisfy his conditions. This, however, applies to the situation in which each of the axes is marked off in 0.2 increments from 0 to 1, and the contents of the cells are the products of the axes. The proposal here differs in that axes have equal logarithmic increments, allowing a risk score to be calculated as the sum, rather than the product, of the quantities on the two axes. This representation provides a 5 x 5 table with 5 risk levels, which still fulfils all of Cox's (2008a) conditions. Such risk levels would provide 5 levels of risk in Table 2, where each level covers a range of essentially 2 units on the log scale, corresponding to a 100-fold step in risk between adjacent levels. This would generally be too coarse in practice and we would usually prefer the risk level steps to be rather smaller. Accordingly, we have coloured Table 2 with risk levels that go in steps of 1 unit, or 10-fold changes in risk, which we believe to be more appropriate. The defining of risk levels and the cutoff values applied to them depends on (i) the actions that would be taken for each risk level, and (ii) agreement that this is the action one would want to take if a hazard fell into that risk level. Table 3 provides a typical

459


description of action from a report by the United States General Accounting Office (GAO, 1998) that would be taken for certain risk levels, so those involved in the risk management process need to agree the appropriate cutoff point for each action envisaged. Table 3: Examples of Actions Associated with Risk Levels Risk Level Extreme High Medium Low

Action Required Unacceptable (reduce risk through countermeasures) Undesirable (management decision required) Acceptable with review by management Acceptable without review

The step size can be reduced on either or both the consequence and likelihood scale. For example, the steps between consequence (and likelihood) categories could be 0.5, corresponding roughly to a 3-fold step up in consequence (and likelihood) between successive categories. Cox's (2008a) conditions would then be met by having risk levels that went in steps of 1 unit, corresponding to 10-fold changes in risk. The appropriateness of this would depend very much on the situation, in terms of the range of consequences and likelihoods being considered, and the supposed accuracy with which they could be determined. The Risk Matrix defined in this way has a number of distinct advantages: 1. Such a risk matrix "ties" both consequence and likelihood scores to real, measurable quantities. This provides more objectivity, even when expert elicitation is used, by tying assessments to real numerical values. Furthermore, where data is either available or can be collected into the future, the data can provide estimates, provide validation for previously determined values, or, in cases of sparse data, augment expert elicitation data. 2. It can be readily adapted to the situation. For example, likelihood here has been described in terms of expected number of events per annum, but could equally be the number of events per 10 years (thus adding 1 to the likelihood scores proposed), monthly (essentially subtracting 1), or daily (subtracting 2.5). Similarly, consequence scores could be considered as log base 10 of the cost in millions of dollars (thus subtracting 6). 3. The likelihood and consequence scores can readily be extended in either direction if hazards of relevance fall into such categories. 4. Generally, few if any hazards will be in the extreme corners of the risk matrix, since good process design generally mitigates the most extreme hazards at an early stage, and hazards of very low risk are usually ignored. Most risks will then occur on a broad diagonal from bottom left to top right in Table 2. If so, then the refinement which provides risk levels that have steps of 1 unit (a 10-fold increase) is preferable. 5. The former comment, and Cox's (2008a) conditions, suggest that 1 unit steps in consequence and likelihood scores do not give sufficient resolution to set the risk level increments to 1 unit. That implies that we should either use categories with 0.5 unit steps or provide explicit values for consequence and likelihood scores.

460


It may be advantageous to move away from the concept of a risk matrix and to consider continuous scales on the axes, in order to provide a risk graph. This has the additional advantage that hazards can be located on the graph, possibly surrounded by a region, such as a 95% probability region, within which they are considered most likely to lie, such regions being determined from data and/or expert opinion. Consequences often cannot be measured in terms of dollars alone, and there are often other types of consequences that need to be considered. This aspect is addressed in the next section.

3. The Consequence Matrix Consequences of particular hazards, where they are strictly monetary, can be assessed and estimated. This may be possible even in cases where there is no previous data. More difficult is the situation where consequences are not describable in numerical terms. Environmental impacts, for example, might be monetary in some cases, as measured by the price that would be paid to clean up a spill, but expressing a loss of species on a monetary scale is more contentious. One way to resolve this is to develop a Consequence Matrix in which columns represent different categories of consequence and rows are labelled with the different consequence scores. The entries in the table provide, for each consequence category, a verbal description of what would be required to achieve that consequence score for that category. A version of this is provided in Table 6.2 of Handbook 436 (2004) although the proposal here is to make this more robust and, in a sense, quantifiable. In a particular government-based risk assessment in which the authors were involved, there were seven such consequence categories, labelled "Death, injury or illness", "Economic", "Social", "Environmental", "Symbolic", "External", and "Reputational". These form the columns of the Consequence Matrix. Table 4 shows examples of typical descriptions for moderate level consequence for each of these seven consequence categories. The dot points in the table provide agreed descriptions of consequences for each consequence score. There are three important features of the Consequence Matrix proposed here: • •

•

An increase of 1 unit in consequence score corresponds to a 10-fold increase in the economic consequence category. For other consequence categories, judgment and agreement between stakeholders are needed to provide descriptions for each consequence score that represent events of similar "importance" or "magnitude" regardless of the consequence category. The score allocated in a given consequence category is the highest score for which one or more of the dot points in Table 4 is expected to be realised.

By this means, a score of, say, "8" in any column of the matrix could be regarded as being of similar importance. Each of the categories is to some extent related to the others. Seven categories are used not to create duplication but rather to assist in defining the potential consequences of a particular hazard.

461


Table 4: Examples of Descriptions for Consequence Categories Category Death, injury or illness

Economic/ Business

Social

Environmental

Symbolic

External Reputational/ Public Image

Description for Moderate Consequence (Consequence Score = 8) • Multiple fatalities; • Mass very seriously ill or seriously ill casualties; • State and/or Territory health system fully committed with local health system overwhelmed; and/or • Australian Government assistance considered, State and/or Territory response required. • Impact of $30−300 million; Impacts on business include: • Travel to and within Australia disrupted; • Substantial disruption to industry and/or commerce; and/or • Disruption of one or more national industry sectors with recovery likely to last one to three months. • Significant and/or short-term challenge to Rule of Law, lasting up to 1 month; • Civil liberties, freedom of speech, association, movement, or religion denied or restricted for up to 1 month locally, and threatened at state level for up to 7 days; • Widespread disruption to, or destruction of, the state physical and communications infrastructure, and other essential services (including critical social infrastructure) for up to 7 days; • Isolated disruption to, or degradation of, the local education system up to 1 month, or specifically isolated disruption to state system up to 7 days; and/or • Extreme disruption to local participation in community, arts, cultural, sporting, and leisure activities for up to 1 month, or significant disruption at state level for up to 7 days. • Damage to a conservation value of a marine bioregion, including species, communities or areas identified as of particular conservation significance where recovery extends from three to 10 years; • Introduction of exotic marine/terrestrial pest species resulting in a localised incursion with substantial long-term environmental impacts or a widespread incursion with environmental follow on effects where recovery extends from three to 10 years; • Industrial scale harvesting or trade in, any native species; • Exacerbation/causation of probable long-term decline in an important population of, or habitat for, a listed species; • Commercial harvesting of, trade in, or removal from the biomass of any listed species; and/or • Killing or removal from the biomass of individual representatives of a species listed as critically endangered. • Significant reparable damage to a nationally important symbol that is internationally recognised; • Significant irreparable damage to a nationally important symbol; and/or • Destruction of a locally important symbol. • Setback and damage to bilateral relations; and/or • Frequent and deliberate challenges to Australia’s sovereignty by a foreign state. • Major criticism and temporary damage to the government’s parliamentary reputation; • Inquiry with detrimental findings and significant criticism leading to temporary damage to the government’s reputation; and/or • Moderate damage to Australia’s national business reputation.

462


An overall consequence score is required which does not diminish the effects of any high consequence scores. Use of the maximum of the scores is one alternative; however, it does not acknowledge the different consequence implications between a case where 8 occurs in just one category compared to the more extreme case where 8 occurs in more than one category. A shift of 1 unit in the consequence score for Economic Impact corresponds to a 10-fold increase in the cost of an event. For example, an "8" corresponds to an economic impact of $30M-$300M, for which a central value, on the log scale, is $100M (=$108). If a score of "8" in other consequence categories is regarded as implying an event of a similar order of magnitude, then these might be regarded as also having a similar "cost". Adding the numbers after taking a power of 10, then corresponds to adding the implied costs, and taking the log of the sum takes it back to a consequence score again. This leads to a formula for combining the consequence scores as a three-step process: 1. Take 10 raised to the power of each consequence score (ignore any categories for which the impact is assessed as Nil). 2. Add these numbers together to give an overall total. 3. Find the power of 10 that gives this total (the “log” of the total). Table 5: Example of Calculations of Overall Consequence Score Death, injury Economic EnvironReputational/ Overall Hazard Social Symbolic External or /Business mental Public Image Score illness 9 6 6 6 6 6 9.003 Example 1 6 Example 2

6

9

6

9

9

6

6

9.478

The overall consequence score for a particular hazard will then be slightly higher than the maximum score across a row, but will be increased if that maximum score occurs in more than one consequence category. Table 5 shows two worked examples, where in each case the maximum of the individual consequence scores is 9. For example, if the consequence scores are 6, 9, 6, 9, 9, 6, 6 in the seven categories, then the overall consequence score is Log10 ( 106+109+106+109+109+106+106 ) = Log10 ( 3,004,000,000 ) = 9.478. Clearly, this is preferable to the average score of (6+9+6+9+9+6+6)/7 = 7.28, which would downweight the scores of "9" to such an extent that the hazard would be regarded as having quite minor consequences. It is also preferable to just using the maximum score, since it adequately recognises that high severity in more than one category is "worse" than having it in just one category. The maximum score achievable when the highest individual score is 9 would be log10(7×109) = 9.85, when all seven categories are scored at 9. It is more important that the consequence categories cover the different types of consequences envisaged, rather than being seen as independent of one another. The main argument for this is that, even if a consequence category were repeated (that is, fully dependent on each other), the overall consequence score would only be minimally

463


inflated. For example, if we added an eighth consequence category to the example above which simply repeated, say, the second consequence category, the overall consequence score would only be elevated from 9.478 to 9.602, an increase much less than the likely levels of uncertainty in the estimates.

4. The Likelihood Likelihood can often be assessed using past data, either internal to the organisation or external, which provide estimates of the frequency of various types of events. Where historical data exists, likelihoods scores can generally be estimated more accurately at the higher end of the scale than the lower end, since there will be more events within a given time window. As a result, it may be useful to consider categories of unequal sizes, as shown for example in Table 6. Again, a 1-unit step in the scale is a 10fold step in rate of occurrence, and the scale could be extended in either direction if events at a higher or lower rate were required to be identified. Note that the words used to describe the likelihood categories (such as "Likely") should be chosen to be suitable to the particular application. Likelihood must be based on the same event for which the consequence score has just been determined. As an example of the potential difficulty, consider fishing. If the likelihood is taken to be the number of fishing boats, say, over a defined period of time, the consequence must relate to the impact of a single boat over that time. If the event is loss of a fishing ground due to over-fishing, and the consequences related to that are evaluated, the likelihood is the likelihood that the fishery would be lost given the current level of fishing. Table 6: Proposed Likelihood Scores Based on Annual Rates Likelihood

Description

Almost Certain Very Likely

The event will occur on about a monthly basis The event will occur two or three times a year The event will occur on an annual basis The event will occur every few years The event will occur from time to time Aware of an event like this occurring elsewhere

Likely Possible Unlikely Rare

Indicative Average Likelihood Recurrence Interval Score 10 events or more a year Two to three events a year

Indicative (log) range

1

(0.75, .)

0.5

(0.25, 0.75]

0

(-0.25, 0.25]

One every three years

-0.5

(-0.75,-0.25]

One event in 10 years

-1

(-1.5, -0.75]

One event in 100 years

-2

(-2.5, -1.5]

One every year

The rate of occurrence is related to 10 raised to the power represented by the likelihood score, in particular Rate of occurrence (/yr) = 10Likelihood Score.

464


The reference period of a year can also be changed to suit the application. By defining Likelihood so clearly, the opportunity is created to calibrate Likelihood assessments against available data. Other methods such as expert opinion may have to be used in cases where no data is available, although of course the absence of such events over a given time period does itself constitute data. Likelihood is harder to evaluate in cases of low event frequency, especially in situations where such events have never happened before. Security risks, such as terrorist attacks, often fall into this category and make it particularly difficult to estimate the likelihood of an event (Cox, 2008b). Dealing with likelihood in situations of intentional harm where there are forces or individuals who are trying to find ways of breaching the systems of controls is the subject of a separate paper by the current authors.

5. Quantification of Risk There is considerable disagreement over the degree to which consequence and likelihood can or should be quantified. Even in the Australian Standard (2004), the full range from qualitative to quantitative is considered permissible. Our view is that the risk assessment should aim to provide quantitative estimates, even if there is uncertainty in those estimates. The primary motivation for this is an issue of "measuring instruments". It is our strong view that the "measuring instruments" need to provide a clear and unambiguous yardstick against which consequence and likelihood can be assessed. Van Duijne et al. (2008) point to this problem when they say "scale values such as ‘unlikely’ and ‘improbable’ can be interpreted as almost similar, yet risk assessors are forced to choose one of these options if both values appear on the same scale." If there is no precise definition of what these scale values represent for either consequence or likelihood, then the uncertainty in the assessment will contain two components of variation − the first related to people's differing interpretations of the measuring instrument and the second related to their uncertainty in where to place a particular event on the scale. The aim of precisely defined categories is to remove the first of these sources of variation, so that discussions need centre only on the uncertainty in placing the event on a well defined scale. There are other important reasons for defining the scales in quantitative terms: • • • •

Where the risk assessment is more qualitative, it is difficult to combine consequence and likelihood appropriately to give an overall risk score; that is, how consequences should be weighted relative to likelihoods. Expected Cost provides a direct calculation of Risk and hence defines how the two factors are combined. It provides a mathematical formalism that justifies how contours of equal risk should be drawn. Where Expected Costs are calculated in the presence and absence of proposed (and costed) mitigation activities, a true cost-benefit analysis of the value of the mitigation can be assessed. Expected Cost is not always appropriate as the measure of Risk. Van Eijndhoven and Ravenzwaaij (1989) consider "societal risk" where "to allow for the relative unwantedness of accidents with large consequences, it has been decided that a consequence n times greater must correspond to a chance n2 smaller". In this case, the Risk Score would be revised to

465


Risk Score = 2 × (Consequence Score) + (Likelihood Score) giving the consequence score a greater weight and effectively making the diagonals in Table 2 steeper. This increased weight to the consequence score is also suggested as an option in Handbook 436 (2004, p.49). The conversion of risk scores into risk levels requires a mapping into levels which will determine the type of actions that need to be taken. Table 3 provides one example, drawn from GAO (1998). It is a matter of governance to determine at what risk score each such action would need to be initiated and hence the boundaries for the risk levels. This will provide a series of diagonal contours in the Risk Matrix, as shown in Table 2, corresponding to the boundaries between different risk levels. With a relatively fine subdivision of risk scores, namely steps of 1 unit, nearly all of the cells in the table contain hazards which could, according to their position within that cell, be allocated to one of several different risk levels. The risk levels shown in Table 2 are for a hazard whose consequence and likelihood scores are in the middle of the corresponding bands. If, however, we consider the full range of values that may occur within a table entry, then other risk levels are possible. For example, if the Medium risk level is "6" with risk scores in the interval [5.5, 6.5), then each cell labelled Medium in Table 2 can take scores in the range [5, 7), and so particular entries, in the corners, could be coded as either Low or High. However, on the assumption that hazards are placed uniformly and independently on each of the consequence and likelihood score scales, the probability is only 1/8 each of having a low or high risk rating and 6/8 of having the medium risk level. A move to consequence and likelihood scores on a continuous numerical scale, rather than integer steps, would reduce this problem, although uncertainty surrounding the estimate of each score would still imply that the "best estimate" of the risk level was subject to variation.

6. Combining Different Hazards in Different Regions Risk scores are typically determined for a range of hazards. These hazards might well be geographically dispersed and of different types. There are a number of ways in which the risks for different hazards across geographical regions might be combined. For example, the overall risk across all hazards might be compared between regions, or an overall assessment of the risk associated with a particular type of hazard might be required. Having consequence, likelihood and risk scores defined as in this paper allows the risks to be combined, producing interpretable results.

6.1 Combining Risk Risk scores are determined for each identified threat. In some cases, they are also determined separately for each of a number of sites or geographical regions. For example, a company might have factories in each of a number of locations and wishes to undertake an overall risk assessment covering all regions. The need is then to combine risk scores (i) across all threats within a region, (ii) for a given threat across all regions, or (iii) across both threats and regions. These calculations follow in the same way as for combining consequence scores described in Section 3: 1. Take 10 raised to the power of each risk score. 2. Add these numbers together to give an overall total.

466


3. Find the power of 10 that gives this total (i.e., the "log" of the total). This is based on the premise that adding the expected costs across threats or regions is sensible, and then taking the log of this number reduces the numbers back to the risk score scale: Expected Cost = Sum{10(Consequence Score + Likelihood Score) } = Sum{10(Risk Score) } and the combined Risk Score would just be the log of this Expected Cost. Consequences and likelihoods essentially provide a probability distribution for cost, where, for each hazard, there is a certain probability distribution for cost which can be turned into an expected cost. Summed over all the different hazards, this gives an overall Expected Cost. If the occurrences of the different hazards are correlated with each other (either positively or negatively), then this has implications for the variance of the Cost, but it has no effect on the expected value of the Cost.

6.2 Combining Likelihood and Consequence Scores Forming a combined Risk Score across a number of hazards begs the question as to whether it is possible to associate a likelihood score and a consequence score with that combined set of hazards. Assume it is possible and meaningful, then suppose there are p such hazards, with consequence scores Ci and likelihood scores Li (i=1,...,p). Then the risk scores are Ri = Ci + Li and the combined Risk Score is R = log{ Sum(10Ri) } = log{ Sum(10 Ci+Li) }. In a similar way, a combined likelihood score can be obtained by noting that the rate of occurrence for a set of threats is the sum of the rates of occurrence of the individual threats. Since the likelihood score is the log of these rates, the combined likelihood score is given by L = log{ Sum(10Li) }. This is true even if the processes generating the individual threats are not independent. The combined consequence score C for the set of hazards should satisfy the rule that R = L + C, from which it follows that C = log{ Sum(10 Ci+Li) } − log{ Sum(10Li) } = log{ Sum(10 Ci+Li) / Sum(10Li) }, so that the combined consequence is a weighted average of the consequences of the individual hazards, weighted by the rates at which they occur. The combined likelihood score provides a rate of occurrence for a series of events which may have very different consequences. The combined consequence score is a summary statistic − an expected value − for a distribution of consequence scores represented by the set of hazards it summarises. The number 10C is in fact the mean of a probability distribution where consequences 10Ci each occur with probability mass 10Li / Sum(10Li ). While this may represent a formal view of the process, it is more complex for several reasons. Firstly, the consequences assigned to the individual hazards are themselves likely to be estimates whose uncertainty needs to be quantified. It is then possible that the consequence for each hazard is represented by a probability distribution, so that the consequences related to the combination of hazards is a more complex, possibly

467


multivariate, probability distribution. Second, this poses some difficulties when either or both of consequences and likelihoods cover several orders of magnitude. It is not, for example, very useful to say that, when the hazards are combined, we will have one event per year with an average cost of $105 if that is a summary of two hazards, one of which has a cost of $108 with a probability of 0.001, and the other has a cost of $102 with a probability of 0.999. It is thus important to keep an eye on which hazards contribute to the risk. As before, we recommend setting a lower limit to the size of consequences considered.

6.3 Obtaining Risk Scores for Each Region or for Each Type of Event In principle, the hazards that apply in a region can be identified, separately for each region. In combining these across hazards or across regions, it is necessary first to assess the likelihood and consequence scores, separately for each hazard in each region. This will provide a table like Table 7 in which the rows are the possible hazards for each region, and the columns represent, in turn, the overall consequence score, the likelihood score and the risk score for each hazard in each region. In the example, it is assumed that there are four hazards which may occur in each of two regions. In practice, it is likely that some hazards will occur in only some of the regions. These scores can be converted to $-values in the sense described earlier and rates of occurrence. Table 7 then shows a "Cost per event", and a "Rate" in terms of the expected number of events per year. The product of these two gives the expected cost per annum related to that particular hazard in that region, shown in the last column of Table 7. Table 7: Example of Four Hazards Across Two Regions Region Hazards Cons Score Lik Score Risk Score Cost per event Rate A A A A B B B B

1 2 3 4 1 2 3 4

7.0 8.8 7.7 7.9 7.3 7.5 7.1 7.8

0.0 -1.0 0.0 -1.0 0.0 -0.5 -1.0 -1.0

7.0 7.8 7.7 6.9 7.3 7.0 6.1 6.8

Exp Cost

$10,000,000 $630,957,344 $50,118,723 $79,432,823 $19,952,623 $31,622,777 $12,589,254 $63,095,734

1.0000 0.1000 1.0000 0.1000 1.0000 0.3162 0.1000 0.1000

$10,000,000 $63,095,734 $50,118,723 $7,943,282 $19,952,623 $10,000,000 $1,258,925 $6,309,573

$59,617,155 $24,746,363 $14,976,312 $175,614,749 $46,706,953 $71,264,279 $45,389,807

2.2000 $131,157,740 1.5162 $37,521,122 2.0000 $29,952,623 0.4162 $73,095,734 1.1000 $51,377,649 0.2000 $14,252,856 3.7162 $168,678,862

Summaries A B A+B A+B A+B A+B A+B

all all 1 2 3 4 Overall

7.8 7.4 7.2 8.2 7.7 7.9 7.66

0.3 0.2 0.3 -0.4 0.0 -0.7 0.57

8.1 7.6 7.5 7.9 7.7 7.2 8.23

This simple example has some hazards with huge costs that are infrequent (e.g. Row 2 of the table), but also some lower cost hazards that happen more often (e.g. Row 1). The costs and rates of occurrence can then combined as shown to provide summaries based on

468


combining either (i) threats within each region, (ii) each threat across all regions, or (iii) all threats in all regions. These are shown under the "Summaries" in the second half of Table 7. The last row of Table 7 for example, cumulates over all threats and all regions to give an overall assessment. In particular: • • • •

The Expected Cost per year, $169M, is the sum of the first 8 entries in that column. The overall rate of occurrence, 3.72 per year, is the sum of the first 8 entries in the "Rate" column. Dividing the first by the second gives an average cost per event of $45M. This is essentially an average of the costs, weighted by the rates of occurrence. Reversing the formulae for these three numbers provides, respectively, the Risk Score, Likelihood Score and Consequence Score in the last row.

The $169M is like an insurance premium − costs in individual years would vary widely. For example, the "Rate" column gives the (average) number of events per year in each category, so that, on average, there will be one event costing $10M, one costing $20M and one costing $50M, with others occurring less frequently. Once in 10 years an event costing $631M will occur. So this is definitely just an average figure.

6.4 Potential Dependence Between Consequence and Likelihood There may be some dependence between consequence and likelihood. There are several ways in which this might arise and it is important to recognise whether this will have an impact on the Expected Cost. Firstly, there may be a common covariate which describes the dependence. In credit risk, researchers such as Pykhtin (2003) and Rösch and Scheule (2005) have noted that the probability of default goes down as the size of default increases, so that likelihood and consequence are negatively correlated. In such cases, stratifying against the underlying variable − the stage of the economic cycle or the size of the original loan − is likely to lead to independence between the estimations of probability of default and size of default within each stratum. Secondly, dependence between estimation of consequence and estimation may appear as part of expert elicitation. In finance, this is often dealt with by restricting attention to events related to losses greater than some figure, such as $1M, with smaller events being managed through normal bank processes. This is likely to remove the artificially induced dependence identified above and significantly reduces the amount of effort to undertake a full risk assessment. Thirdly, dependence may also occur when hazards in different regions have some dependence on each other. The Expected Cost may be unaffected by such dependence but other measures such as the standard deviation of the estimated cost or the Value at Risk may be markedly affected.

7. Conclusions Any risk assessment that defines a risk level based on a quantitative assessment of consequence and likelihood has an implied mathematical relationship. By providing a risk matrix with quite explicit categories, the nature of the implied mathematical relationship becomes evident and objective. Such uncertainty as then exists is confined to the placement of hazards into a well-defined structure, where the uncertainty is related variously to lack of data, the variety of views among participants, uncertainty about the future and other possible sources.

469


Consequences cannot always be assessed purely on a $-value. This paper offers a solution which allows other categories of consequences to be put onto the same scale and then shows how the consequence scores across these categories can be combined into an overall consequence score. A significant advantage of the proposed approach is that the assessment of consequence and likelihood is rooted in values which, in theory at least, can be supported or validated by data. This has three major benefits. Firstly, it helps to create more objectivity, even when expect elicitation is used, by tying the assessments to real numerical values. Second, where data are either available or can be collected in the future, they can be used to provide estimates, to provide validation for previously determined values, or in cases of sparse data, to augment expert elicitation data. Third, the risk levels obtained are interpretable, for example, in insurance terms. The methodology, since it is described in terms of consequences and likelihoods, might be thought to apply strictly to those situations where hazards arise through accidents or natural causes. Further work has been undertaken by the authors to extend the methodology to the situation where hazards also arise through intentional acts, such as criminality or terrorism. Concerns have been expressed about the appropriateness of the methodology for these situations, particularly in relation to the assessment of likelihood. However, the contention is that risk and risk control must increasingly be prepared to consider hazard across both purposeful and non-purposeful acts, in order to make appropriate use of finite risk control resources.

References Anderson, K. (2006). A synthesis of risk matrices, Newsletter of the Australian Safety Critical Systems Association, 8-11, December. Australian Standard (2004). Risk Management. AS/NZS 4360:2004. Standards Australia. Cox, L.A., Jr. (2008a). What’s wrong with the risk matrices? Risk Analysis, 28(2): 497512. Cox, L.A., Jr. (2008b). Some Limitations of “Risk = Threat × Vulnerability × Consequence” for Risk Analysis of Terrorist Attacks. Risk Analysis, 28(6): 17491761. Cross, J. (1995). The risk management standard. The Australian Journal of Emergency Management, 10(4): 4-7. GAO (1998). Combating terrorism: threat and risk assessments can help prioritize and target program investments. United States General Accounting Office, Report Number GAO/NSIAD-98-74. (http://www.gao.gov/archive/1998/ns98074.pdf) Handbook 436 (2004). Risk Management Guidelines - Companion to AS/NZS 4360:2004. Standards Australia. Pykhtin, M. (2003). Unexpected recovery risk. Risk, 74-78, August. Rösch, D. and Scheule, H. (2005). A multifactor approach for systematic default and recovery risk. The Journal of Fixed Income, 63-75, September. Swallom, D.W. (2005). A common mishap risk assessment matrix for United States Department of Defense aircraft systems. Proceedings of the 23rd International System Safety Conference, San Diego, California, August. Van Duijne, F.H., van Aken, D. and Schouten, E.G. (2008). Considerations in developing complete and quantified methods for risk assessment. Safety Science, 46: 245-254. Van Eijndhoven, J.C.M and van Ravenzwaaij, A. (1989). Optimizing risk analysis relating to external safety in the Netherlands. Risk Analysis, 9(4): 495-504.

470