an aid to better decision making

iDEPEND

6/29/2013

AN AID TO BETTER DECISION MAKING

CAMBRENSIS

Getting to Grips with Uncertainty | David Slater 1 Modelling Basics 1 ds

06 Feb. 14

1. (Bayesian Net) Dependency Modelling using iDepend What is Dependency Modelling? To start at the beginning, a dependency model is based on goals and objectives, and the prerequisites to satisfy these goals. In other words it is a positivist, top down approach working from goals to requirements. This is in strong contrast with other methodologies which focus on faults, disasters and failures. There are a number of advantages in the positivist approach, not least being that it is easier and more intuitive to think of goals and requirements. Senior management people are more comfortable working with them than with disasters The principle behind Dependency Modelling is that ideally the analyst should not have to work out all the things that could go wrong with his enterprise. His responsibility should be limited to providing a sufficiently accurate model of the enterprise, and software will do the rest.While we can't totally achieve that ideal, Dependency Modelling goes a long way towards it. So in Dependency Modelling we build an abstract model of the enterprise which software then analyses. In reality we have to build the model in such a way as to help the software along a bit. Dependency Modelling is a way of analysing the risks to an enterprise - anything for which a suitable model can be built. It works by analysing the model. However it doesn't work by dealing with risk directly, but rather by studying success and what we want to achieve, and it's much more interesting and less scary than thinking what could go wrong. It is based on the idea that risk is about goals and all risk springs from the fact that achieving our goals depends on many things some of which we can't control, or predict, or in some cases, even understand. It can be applied to any complex system such as an organization, a company, a project, a machine, a doctrine, a diagnostic, a government, a charity, a business model, or a village fête. Since there is no word that adequately covers all these, we'll use enterprise. We won't call it a system since that could mean the computer used to do the Dependency Modelling.

Why Dependency Modelling? There are a large number of advantages of building a dependency model, such as: 

If forms a language to discuss risk with other people.



If forces us to understand and articulate what we are trying to achieve.



Any misunderstandings we have are made visible to ourselves and others, and are thereby more likely to be uncovered.



It allows us to analyse risk. 2

Modelling Basics ds 0.1

6-Feb-14

What is RISK? If we use a standard (say ISO 27000) then we are encouraged to fill in risk registers and plot Probability /Impact graphs (PIGS). But does that really meet the needs of modern managers?

Risk category

Current Solutions

Description of category

Agreement

Risks that relate to contractual agreements on supply of discharge, permits in place, etc.

Business Continuity Plan

Risks difficult to manage by the Site due to the external character of the risk, e.g. extreme weather conditions.

CAPEX

Risk that are fairly easily dealt with as long as the necessary funding is available

Compliance

Risks that are related to current or future non-compliance of the site operations.

External stakeholders

Risks that by its nature is dependent upon relationship (goodwill) with external stakeholders

Off-site infrastructure

Risks that have to be dealt with in terms of off-site infrastructure upgrades

On-site infrastructure

Risks that have to be dealt with in terms of on-site infrastructure upgrades, but likely to have low cost investment need only

On-site storage capacity

Risks that relate to lack of or limited onsite storage capacity

OPEX

Risks that are expected to cause increase in operational expenditures only. Typically cost increase for water supply and wastewater discharge is put in this category.

Disasters you cannot afford

Other

Risk that could not immediately be categorised within any of the above categories.

04/03/2013

Obiectivo Rome

3

Probability versus Severity – Aside Before we get started here's a quick aside. Many people are confused by the concept of probability. To be specific they confuse the probability of an event with the severity of the event, but these are quite different ideas. The severity of say, an earthquake might be measured on the Richter Scale. This indicates how much damage is likely to be caused by an earthquake of that severity. The probability of an earthquake is nothing to do with its severity, but rather it measures how likely it is to happen at all.

Even the designers of the Titanic were confused about the difference. The vessel was 3 Modelling Basics ds 0.1

6-Feb-14

believed by some to be virtually unsinkable, and as a result it was fitted with only enough lifeboats for half the passengers. Now a moment’s thought shows that either it could sink or it couldn't. If it could sink it needed enough boats for everyone. If it couldn't then it didn't need any boats. There was never a logical argument for half the boats. Yet an otherwise intelligent designer confused likelihood with severity. In the context of future risk we need to say what we mean by probability. When an event takes place that could have different possible outcomes, the probability that it will have a particular given outcome is simply the proportion of times when it actually does have that outcome. So if a certain event can have six possible outcomes, called say outcome number 1 to outcome number 6, and if say the proportion of occasions on which outcome number 6 occurs is 41%, then we would say that outcome number 6 has a 41% (or 0.41) probability of occurring.(Of course if the event consists of rolling a dice and the probability of landing a 6 turns out to be 41% we would say the dice is loaded!) When we come to discuss the probability of achieving a new goal that has never occurred before we can't easily use this idea because there is no history of successes and failures, nor probably any way to carry out repeated experiments and measure the proportion of occasions when the goal is achieved. So we need some way to visualise what probability means in this context. This thought experiment may help. Suppose we somehow have access to thousands of parallel universes, all very similar, and in each one a similar entrepreneur attempts to bring a similar venture to fruition. Suppose that in 73% of these universes the venture is deemed successful, then we would say that the probability of a successful outcome to this venture is 73% (or 0.73).Importantly, it's nothing to do with the degree of success, but rather whether the outcome falls into the category called success. That's the difference. How could we come up with such a figure if we can't do the imaginary experiments? Well we can estimate it another way if the outcome is determined by the various things it depends on. If we understand the statistics of those dependencies it turns out that we can actually calculate the probability that the outcome is successful. Or to be more precise, we can get a computer to do it for us.

ISO 31000 – Goal oriented, Enterprise wide, Risk Management Risk (for Managers) is now defined as – “Uncertainty on Objectives” Because Managers are focussed on achieving “Objectives” - concentrating on the things they know they depend on to achieve them; and identifying and understanding the things that might hinder them. So “uncertainty” in predicting these critical dependencies is a real problem – a “RISK” Classic Risk Methodologies (27000) only address “Failure”- (not uncertainty!) - in terms of PIGS - “consequences” and “likelihood”- This is not the same! Therefore we need a new methodology that is tailored to meeting the real needs of managers. A new Objective Driven, Dynamic, Dependency Defining approach is now available to us – Dependency Modelling – It is now the de facto (Open Group) Standard for ERM

4 Modelling Basics ds 0.1

6-Feb-14

2. But Before We Start! Risk Projects When various people attempt to get together to build a model of an infrastructure they need to agree upfront on many aspects of policy, otherwise the model will be a façade and conclusions based on analysing it will be meaningless. This does not just apply to groups of people. Even a single person building his first model can become unstuck because he didn't give sufficient thought to various overarching considerations at the outset.

Good model, bad model Having a flawed model is better than not having a model at all, because without a model there is little hope of any misunderstanding coming to light. The concept that a flawed model is usually better than no model is not the sole preserve of risk analysis. Indeed there is no field of science that can claim its theories are totally correct. Our state of knowledge is simply the best we have at any one time, and errors are put right as and when they are discovered. Aristotle's laws of motion were replaced by Newton's, which in turn were replaced by Einstein's General Theory of Relativity, and are likely to be overturned by a Theory of Everything. But in their own times the earlier ones worked well enough. Without a model people tend to use fallacious reasoning such as “why should I give up smoking when I could be killed by a truck?” If the questioner had a model showing his life goals and how they depend on his relationships, wealth, health etc., and the error would be immediately obvious.

Evaluating a model Without knowing the background intention for a model it's impossible to determine if it's sensible. Moreover the vast majority of illogical models stem from a failure to set out the ground rules. This probably results in turn from the model being constructed in a rush. But without access to the background intention and the ground rules, anyone attempting to check out the correctness of a model will usually have to spend days interviewing the model's author to find exactly what he meant by the various entities in it. So here's a quick guide on how to go about building a sensible model

Philosophical plane Decide on the philosophical plane for the model. For instance is your model to be based on real-world entities like availability of power, workforce, materials, buildings and general infrastructure, or is it to be based on 5 Modelling Basics ds 0.1

6-Feb-14

metaphysical entities such as political climate, willingness, motivation, flexibility, perceived threat etc. These and many other ideological partitions can all make perfectly logical and tractable models, but you need to pick one, stick to it, and have every paragon defined consistently with that decision, or you will become confused and the model will appear ridiculous to an observer.

Timeframe Decide on a common timeframe for everything in the model. Without a time-frame probabilities are meaningless. Assuming independence of time periods, something that has an only a 1% probability of failing within a day has a 26% probability of failing within a month and a 97.4% probability of failing within a year. Statistics are totally meaningless without a time-frame.

Now we can Start! •

Start by clearly defining our Objectives (in a logical and consistent way)

•

Then “Map” the critical dependencies (risks) onto those Objectives

•

Build a hierarchical model of our risk environment

•

It must be noted, however, that these critical dependencies will each then have their own individual or even unique objective.

•

To fully appreciate the use of this tool, it is therefore necessary to understand how each “objective” (goal) is directly attributed

•

(by a statement or fuller explanation of how we know what is the desired state and how we will know we’ve achieved it?)

What do Goals and Objectives actually mean? Define precisely what each objective means. You really need to prepare an essay describing it. Then don't allow that definition to drift as the model develops. It sometimes helps to call them “PARAGONS” - just to highlight their special status. For instance if your head paragon is concerned with a nuclear power station, then what aspect are you talking about? Do you mean its effectiveness as a generator of electricity, or perhaps its reliability as a generator of electricity, its effectiveness as a supplier of employment to the local community, its effectiveness at containing leakage of radioactive material, its effectiveness in keeping out terrorists, its effectiveness as visitor centre or a wildlife sanctuary? These are all quite different paragons that can succeed or fail independently of each other, each having quite different dependencies. In other words there is no such thing as The Power Station Paragon, but there could be a paragon for the effectiveness of the power station in supplying a varying demand of between 300MW and


6-Feb-14

1GW without interruption for one month. This goes for every paragon in the model. We know that paragons are always abstract, but that doesn't mean they are vague. Quite the opposite in fact.

Some Definitions that may help?

PARAGON?

Goal

Objective

Meaning:

Something that one's efforts or The purpose toward actions are intended which an endeavour to attain or is directed. accomplish; purpose; target.

Example:

I want to achieve success in the field of genetic research and do what no one has ever done.

I want to complete this thesis on genetic research by the end of this month.

Action:

Generic action, or better still, an outcome toward which we strive

Specific action - the objective supports attainment of the associated goal.

Measure:

Goals may not be Must be measurable strictly measurable and tangible. or tangible.

Time frame:

Longer term

Mid to short term

Suppose your power station depends in some way on the (integrity of its) buildings, and on something called (effectiveness of) procedures, and on something called (effectiveness of) maintenance. You need to think through how you are going to relate these entities to each other. For instance you could say that buildings, procedures and maintenance are all dependencies of the power station. Or you could instead say that maintenance is a dependency of buildings, not of the power station directly. The worst mistakes are to confuse the two, or use both, or not to make upfront decisions about these things, or to have different parts of the model use different conventions. When checking out someone else's model it's trivially easy to spot such inconsistencies. 7 Modelling Basics ds 0.1

6-Feb-14

3. Case Study 1 – Home Alone! Let’s look at a simple example to which we can all relate, (An Englishman’s home is his castle?).We can analyse this using a dependency model! What are the sort of things that this requires to satisfy the goal (Paragon) of my “home’s OK”? I may need (satisfactory?) entertainment, and may need to be able to eat at home. In the UK in winter, a source of heating is always necessary for an OK home! We can also add the requirement for working sanitation to be essential to ensure comfort; these could constitute a set of “needs” to achieve our goal of an OK Home.

Then similarly we can identify a set of requirements (dependencies) on which our nutrition depends. Obviously we can repeat this for the rest of the dependencies.

But we can go delving down into the dependencies until we reach those over which we have 8 Modelling Basics ds 0.1

6-Feb-14

no control – (we have to accept an externally determined status – such as mains electricity, water, weather etc.). These are called “leaves”.

If we were using iDEPEND to build this simple model we could then do much more than all agree that this is a reasonable “Mind Map”. We will have in effect produced a Bayesian Belief Net, which when we feed in estimates of the probabilities of these “Leaf” states being satisfactory or otherwise, can produce quantitative estimates of the states of all the nodes, sensitivity plots of the effects of different “”Leaves” and plotting of different Failure Modes through the tree. It also allows us to see the effects of different failure scenarios (“What if”) by deliberately setting nodes to “fail” 9 Modelling Basics ds 0.1

6-Feb-14

4. Now do it with iDEPEND • Access the web portal – www.idepend.heroku.com • Sign up with a user name and password • Developing basic help menus – Browse • If all else fails read the books • Open Standard, Manual, Papers (Download) • On the Dashboard, create model Home OK, • On that modelling page, enter an entity “Home OK”, • “Add” it to the entity list (Left Hand Column), • Drag and drop on the modelling space! • Looks like this (below)

• Easy isn’t it?

Now add entities Entertainment, Nutrition, Heating and Sanitation. Drag and drop in turn on to the root entity “Home OK”


6-Feb-14

Now on the Root Home OK click Edit

A dialogue box opens Here we can click to add a full description of what our concept of an OK Home really means, or We can specify whether we need all (AND Logic) or any (OR Logic) of the dependencies to be OK to be satisfied. When we get more familiar we can create custom probability “Tables” for every possible states of the dependencies. For this model click AND.


6-Feb-14

Now we can add the entities for the second layer and again specify the Logic.

(Spare TV set is OR)

Now we can add the rest of the entities right down to the “Leaves”

On each Leaf, click EDIT 12 Modelling Basics ds 0.1

6-Feb-14

For each leaf, we can now enter/ estimate a probability that it is likely to be in a satisfactory (On) or failed (Off) by entering numerical values in the table or using the slider. A full description of what this means can also be entered as previously. Additional features allow you to share this status information with others who have this leaf in their models. And finally we can input the actual real life status of say the “Mains Electricity”, (availability of?), as an “External Feed”. We should check at this stage that we have completed all the inputs and their are no red “Default” settings showing on the model. If we have missed a Logic choice or a probability estimate, the tool will flag it for us by holding up the next step until we have! Now we can click on “Reports” on the header bar and click “Default Set”. The tree will light up with the red (Failure) and green, (Success) bars which indicate the probability ratio of each node.

Facilitators Acclimatisation Course

17

The second standard report is the Three point sensitivity plot (Below)


6-Feb-14

The above plot is illustrative (showing a selection of leaves) and your plots will depend on the probability numbers you have actually put in for these leaves. The vertical line (intercept on the y axis) is the probability that the Home is OK (on our definition of OK), with the individual nodes listed with their probabilities in brackets The red Bars show the effect of just failing that node (Some are 100% critical while others have a relatively small impact. Finally the green bars show the extent of potential improvement in the overall resilience of our Home if each node was 100% positive The last report shows on the tree the path of each of these failure modes, in exactly the same way as in Boolean Fault Trees and is very helpful in understanding the vulnerability of the different dependencies and where to focus on countermeasures most effectively. All of the displays are now dynamic in that changes can be made in entities (add, subtract, relocate), Logic, or more detailed probability data are instantly displayed and fresh reports can be generated on demand. On this demonstration version, the best way to save and utilise the reports is to use screen shots.


6-Feb-14

5. Let’s now look more closely at our Home OK Model

Interdependencies in a home There is a main goal, called home OK and it depends upon the provision of entertainment, heating, nutrition and entertainment. Entertainment (by which we merely mean the ability to watch TV) in turn depends on the availability of TV, an antenna and electricity. Electricity in turn depends on mains supply and wiring. (Notice that after its first appearance, electricity is shown with three dots, meaning that the dependencies of electricity are the same as before, but for compactness are not repeated.) 15 Modelling Basics ds 0.1

6-Feb-14

The rest of the diagram should be more or less self evident. Only the names of the interdependent entities are shown, neither their nature nor all the types of relationships, but nevertheless these details are present in the model and can be viewed and changed. We'll come back to this later. Notice that all the entities in the boxes are goals in some sense, in other words they are something we must have, desire or need. They can also represent things we already have and the goal is not to lose them. None of them represents a problem. With this formulation problems are merely unsatisfied goals.

Terminology Notice that some of the entities have dependencies - i.e. entities joined to their right - the things they most immediately depend on. For instance entertainment has the dependencies TV, antenna and electricity. Conversely electricity etc. have entertainment as a dependent. The main goal, home ok can also be thought of as the root (as in a tree), while lesser goals with dependencies, such as entertainment are sometimes called branches. Finally some entities (e.g. mains supply) don't have dependencies at all, so to extend the tree metaphor they are said to be leaves. Leaves are very important. Sometimes they are called uncontrollables to emphasize that they are usually beyond our control. For the same reason they can be described as given. So although some of our goals depend on a mains electricity supply, nevertheless this is something we are simply "given", and as far as we are concerned it is an uncontrollable because we can't change its properties. It turns out that all risk is due either to uncontrollables or to incomplete information and we shall return to this when we come to measure risk. (Note that some authors use a dynastic metaphor whereby if C has a dependency P then P is said to be a parent of C, while C is a child of P. In other words dependency=parent and dependent= child.)

Probability of achieving goals The probabilities of achieving all the goals are shown as coloured bands in this diagram below: This model is a simple one where each entity has only two possible states - success shown in green and failure shown in red. The widths of the colour bands show the probabilities that each entity will be found in these states. It is possible to have as many states as we like, but for simplicity in this model all entities have just two. States are always arranged in increasing order of desirability with the first state representing failure and the last state meaning success, and any in-between states, if any representing some intermediate condition.


6-Feb-14

Probabilities of achieving all the goals

Data for the model These bands result from calculations based on just two pieces of information supplied as part of the model:  

Base probabilities for the leaves Branch relationships

Leaf base probabilities A leaf has a-priori probabilities specified for each state. For instance mains supply may have two states called (say) off and on with probabilities of (say) 0.1% and 99.9% respectively, meaning that during a particular period (of perhaps a week) there is a 0.1% probability of a non-trivial power outage and a 99.9% probability of uninterrupted service. We can't change the properties of the mains supply, so it's given and uncontrollable. (Note that sometimes we will use percentages and sometimes decimal fractions to express probabilities. So 0.5% could also be written as 0.005.)


6-Feb-14

Branch relationships There are three kinds of branch relationships called AND-type, OR-type and CUSTOM-type. The AND and OR types are indicated by small words above the links The branch called electricity is an AND-type. By this we mean that for electricity to be in its on state it needs both mains supply AND wiring to be working (i.e. to be in their success state). The branch TV has an OR-type relationship. We have a functioning TV if either the main TV OR the spare TV, or both are working. It is important to grasp that AND-type relationships enormously increase risk while OR-type relationships greatly reduce it. In other words all relationships are not born equal. There is an infinite number of possible relationship types, and all the ones that are not AND or OR types are lumped together under the heading CUSTOM-type. But we can make a lot headway with just two states for each entity, and just the two types of relationship, AND and OR.

What if something changes? The model is completely under our control so we can experiment with making changes and see what difference it makes. That is one of the ways we come to understand the risks to our enterprise. For instance we can specify that the spare TV is not working. Here's how the top part of the diagram changes:

With spare TV not working We see that the red sections of home OK, entertainment and TV have all increased a bit. The spare TV was an uncontrollable. We can also specify that one of our goals is not achieved. For instance if instead of spare TV we specify that entertainment was found to be not achieved, then the picture changes very dramatically:


6-Feb-14

Entertainment not achieved Now we see that our main goal home OK fails. But looking more closely we see that other entities, for instance electricity and antenna have increased probabilities of failing too. Why is this? It's because the model has automatically adjusted itself to take account of the fact that entertainment failed. It provides in effect an explanation totally consistent with all the available data. In other words it's almost as if the model is trying to explain the likely causes of the failure of entertainment. This is made possible because iDEPEND contains a Bayesian engine to recalculate all the statistics.


6-Feb-14

6. Modelling Basics How do I Start? Let’s start with something simple. We will deliberately avoid anything involving threats to national infrastructures because we want to concentrate on the principles of modelling. Suppose we want to visit friends who live several hundred miles away. Our goal is a successful visit, and we're going to limit our analysis just to the journey. The issues we're concerned with here are possible last minute cancellation by either party, or possible travel problems. So the success of our journey will depend on, for example:  a car that works properly  the availability of fuel  the state of the traffic  possible road closures  ourselves avoiding being taken sick at the last moment  our friends avoiding being taken sick at the last moment We will call these the dependencies of our goal. (To extend the terminology, a goal is the dependant of its dependencies.)

Building Block If we break the model into manageable building blocks, each one then makes sense in its own right and has its own logic. Here's the structure of a typical building block:

Building block This is called a “paragon and its immediate dependencies” – there may be more than two dependencies of course. (But there can’t be just one – then it is effectively just an extended single paragon!) If a model is an essay, then such a building block is a sentence.

But a Paragon is? An Abstraction We have seen that paragons represent an abstraction and there is not a one-to-one correspondence between a paragon and any physical entity. If we are concerned with say a nuclear power station it might be imagined that we could somehow devise the nuclear power station paragon. Not so.

Which has Aspects There are many possible abstract goals associated with a nuclear power station and we could be interested in any one of them. For example, one aspect might be its success as a source of electricity. Another might be its ability to prevent the release of radioactive 20 Modelling Basics ds 0.1

6-Feb-14

particles into the environment; another might be its effectiveness as a supplier of local employment; another with the speed with which it could respond to sudden demands; another might relate to its cost-effectiveness as an energy source and so forth. Each of these different aspects would need to be the subject of the definition of a different paragon. Each such paragon would have different dependencies and have different statistics. It's all part of getting the definition right. A particularly common error when building models is to use one paragon to represent different aspects of the same physical entity. We repeat, there is no such thing as the nuclear power station paragon.

And has States A paragon must have at least two, distinct, named, ordered states, indexed in increasing order of desirability, which typically represent degrees of attainment of the underlying goal, from non-attainment (failure) to full-attainment (success). This is necessary to allow the automatic inference of whether a paragon has succeeded or failed and to measure risk, and to give meaning to conditional probability tables.

Which have Statistics These states must have meaningful, statistical probabilities which, if the paragon has immediate dependencies, are conditional on the states of those immediate dependencies. Values of 0 or 1 are meaningful, permitted probabilities for paragons with dependencies but not for uncontrollables, since in the latter case they are equivalent respectively to nonexistence and non-dependence. The statistics may relate to the reliability of something, such as the effectiveness of a countermeasure, in which case it would be the proportion of occasions in which it succeeds. Alternatively it may the absence of some undesirable state of affairs, such as absence of attempts to break in. Such statistics must be related to a clearly stated period of time, because in a short enough time-frame nothing happens, while in a long enough time period everything possible will certainly happen. To say the probability of no break-ins is 0.99 is meaningless. Do we mean during a second, a day, a week, a year, a millennium?

Must have Logic A good test of whether we have the definitions of paragons logically consistent, and to check if the building block in which it is embedded is logically coherent is the Cover-Up Test. For example we might be modelling the security of a vault and wish to include the relationship between attempted break-ins and countermeasures. We might introduce the paragons no break-ins, no attempts and attempts averted in this building block:

Break-in model This diagram is our statement about the relationship between prevention, attempts and countermeasures. 21 Modelling Basics ds 0.1

6-Feb-14

The definitions of these paragons might be something like this: no break-ins

The state of affairs whereby either there are no attempted break-ins to our vault or such attempts are successfully averted by countermeasures.

no attempts

The state of affairs whereby no attempts are made to break into our vault.

Attempts averted

The state of affairs whereby all attempted break-ins are dealt with in such a way that no break-ins attains the success state.

Notice that this is a tautology - a set of self-reinforcing statements with redundancy. It is the redundancy that guarantees the correctness.

The Cover-Up Test This is simply to cover up one of the definitions and ask yourself (or someone else) to supply the missing wording in such a way as to make the set of definitions a tautology.This may seem rather obvious, but it is common for beginners to throw together ill-conceived models and expect them to provide accurate answers. Note too, the abstract nature of the paragon definitions. There is no paragon representing a vault or a countermeasure for instance. They are all states of affairs. (Notice that if we were analysing the risks to a government, our list would probably include items relating to ministries, departments, armed forces, police divisions, taxation, health services etc. At the other extreme if we were analysing the workings of a motorcycle our concerns would be with the parts of the engine, the fuel tank, brakes, tyres and so forth.)

7. Case Study 2 - Juries and DNA A good way to illustrate the importance of this logical “Building Block” is to look at a classic example of confusion, even by learned counsel!”Imagine that a defendant is on trial solely because his DNA was found at a crime scene. The prosecution states - quite correctly - that the probability that an innocent person's DNA would match is one in a million. The jury is therefore invited to conclude that the defendant is almost certainly guilty. Though this argument has convinced many juries, nevertheless this last piece of reasoning is quite wrong. Even in a country with as few as (say) 10 million people there will be about 10 people whose DNA matches in that country alone, so the defendant is probably innocent.

Model the building block :


6-Feb-14

We can build a simple model of this situation with just three entities. Our main "goal" is DNA found, meaning that DNA is found at the crime scene, and this could be caused either by the guilt of the accused, or a one-in-a-million chance. We simply make the goal DNAfound depend on guilt OR chance.

DNA paradox The a-priori probability that a person about whom we know nothing is guilty is set at 1 in ten million, and the a-priori probability that their DNA will match by chance is set at 1 in a million. Note that by cunning labelling the red and green bands show the probabilities that the words in the boxes are false or true, respectively. The probability report diagram above is when nothing else is specified. It's all red. In other words in absence of any special extra knowledge, there probably isn't a DNA match, the defendant is probably not guilty, and chance doesn't cause a match. This is not the situation in the courtroom.

The probability report diagram above is when we know the defendant is guilty. DNA is found, but it wasn't caused by chance. Again this not the actual problem faced by the court, but the prosecution might want the jury to believe it is.

The court actually faces the problem illustrated in the last probability report diagram. Here we have included the extra information that DNA was found (i.e. the DNA-found goal was achieved). So the DNA-found entity is totally green because we know that's true. But when we look backwards into the causality we see that there is about a 10% probability that the dependent is guilty (the green in the guilt box) and about a 90% probability this was caused by chance (the green in the chance box). So iDEPEND is not fooled for a moment. This kind of reasoning - reverse inference - is very important for risk analysis. It's also extremely difficult to get right by intuition. Making even simple inferences from statistics is fraught with danger, and that's the last thing we need when carrying out a risk analysis.


6-Feb-14

8. Case Study 3 - Boy/Girl paradox Here's another simple example of how difficult statistical reasoning can be without help and how easy it is using a Bayesian model. It's the Boy-Girl paradox as follows. Consider these two questions. Q1 : A man tells you he has two children, at least one of whom is a boy, what is the probability that he has two boys? Q2: A man tells you he has two children and the elder is a boy, what is the probability he has two boys? Some people can't see the difference between these two questions, arguing how can knowing the birth order make any difference? But in fact they are not only different questions but the answers are different too. Here's why. There are just 4 possible families of two children, (1) Boy-Boy, (2) Boy-Girl, (3) Girl-Boy and (4) Girl-Girl, where in each pair the elder child is given first. Q1 tells us we're dealing with a family of type 1, 2 or 3, and of these only type 1 is two boys, therefore the probability is 1/3. Q2 tells us we're dealing with a family of type 1 or 2, and again only type 1 is two boys so the probability is 1/2. This problem is easy to model.

The left image shows the raw probabilities when we don't have any extra information. The two elements "elder is a boy" and "younger is a boy" each has 0.5 probability of being true. The element "at least one boy" is just an OR relationship. The element "two boys" is an AND relationship. We see there is a 0.25 probability that the man has two boys. The centre image shows the situation of Q1 where we are told that the man has at least one boy. We see that the element "two boys" has only a 1/3 chance of being true (the green section).The right image shows the situation where we are told that the elder child is a boy. We see that now there is a 0.5 probability that he has two boys. The point is it's a no-brainer with a Bayesian network.


6-Feb-14

9. Building a Model! The car trip list is not supposed to be comprehensive; it's just a starter to illustrate some ideas. You many disagree with our choice of dependencies, but we picked them mainly for the insight into modelling they provide and not because we really want to know about the risks to a trip.But we can now use it to start building our model The success of our trip depends then on all these things, so we could draw a diagram like the one below.

2-layer trip model

In a more realistic situation the success of a goal would probably depend on many more than just seven items, but this will do for the moment. We can view the entity trip successful as a sort of goal with various degrees of achievability. It might for instance be fully achieved, partially achieved, slightly achieved or totally unachieved, or any degree of achievement we care to come up with. But for the moment we'll keep things simple and limit it just to two possibilities: failure and success. But bad and good, or no and yes, negative and positive, or 0 and 1 would do equally well. We can also view the seven dependencies – such as car OK, fuel available and so on - as sort of minor goals in their own right, just like trip successful. These too have various degrees of achievability, and again for the moment we'll limit the possibilities to just failure and success. It is important to note that these are not fundamental limitations, just convenient simplifications for the present discussion.

A list still isn't a good model Now it turns out that this simple, 2-layer model representing the relationships would not be much use in risk analysis because of the crude way it lumps all the dependencies together. In this form it's no better than a list. But that's the least important reason. For reasons we'll soon make clear, a much better model would be this:


6-Feb-14

Better model

What do we want from the model? Chance(s) of Success Now we need to ask ourselves “what do we want from our model?” One thing must surely be to calculate the chances we will achieve the goal. Think of this as the “probability that the paragon trip successful will be found in the state we called success”.

Causality We might also want to able to find the most likely cause of failing to achieve success, and the most important elements in our model from a success point of view, and so forth. We'll show how to answer all these questions from our model in due course. And of course we'd like to know the risk to our goal. In passing we note that there is a completely different kind of risk information we can derive from our same model called Failure-Mode Analysis. If we regard these diagrams as describing the relationship between cause and effect, then cause is on the right and effect to the left. Putting it another way, causality flows from right to left. So car OK is one of the causes which effect the top goal/paragon, trip successful. In Failure mode Analysis, the lack of a successful trip can be traced back to the failed state of car OK.

Consistency Another way to view the definition of a paragon is to observe that each paragon is defined by its relationship with its dependencies.

Roads OK Take for example the paragon roads OK in the above fragment. We can write a definition for car OK to be anything we like, but in reality the paragon is defined as

Roads_OK = Weather OK AND Traffic_OK AND Roads_Open 26 Modelling Basics ds 0.1

6-Feb-14

Clarity /Logic (AND and OR relationships) There are two kinds of relationships whose conditional probability tables are of a type that occur sufficiently frequently in models to merit special names. They are AND and OR relationships. They can only be used when the paragons involved have only two states. The diagram below illustrates the idea. In both diagrams a heavy weight is suspended from the ceiling by three thin rings. In the left diagram – the AND relationship – the rings form a chain with the links in series. In the right diagram – the OR relationship – the rings are arranged in parallel. We assume that the only components likely to fail are the thin rings.

AND

vs.

OR

The names come from the fact that the left diagram requires the first ring AND the second ring AND the third rings all to function properly without breaking, while the right diagram requires only that either the first OR the second OR the third ring functions. The AND model will fail if any ring fails, whereas the OR relationship will succeed if at least one succeeds.

Countermeasures - OR good, AND bad A vitally important point to grasp is that clearly: an OR relationship reduces risk while an AND relationship increases it. This observation is both shocking in its implications yet beautiful in its elegance. Note that for this statement, and for many other statements to be true, all paragons must represent something positive, and low numbered states must represent something bad, and high numbered states represent something good. This fussiness is of course just another of the reasons we insist on a strict definition for paragon, and why we don't just say goals. We'll define paragons more fully shortly. If the dependencies are D1 and D2 and the goal is G, and p0 and p1 are the probabilities that the goal will be in states 0 and 1, then the tables for the two relationships are like this:

D1 0 0 1 1

AND relationship D2 p0 0 1 1 1 0 1 1 0

p1 0 0 0 1

D1 0 0 1 1

OR relationship D2 p0 0 1 1 0 0 0 1 0

p1 0 1 1 1

(Note that in the D1 and D2 columns the 0s and 1s are just names of states, while in the p0 and p1 bold italicised columns they are numbers measuring probabilities.) 27 Modelling Basics ds 0.1

6-Feb-14

These are of course just special cases of conditional probability tables. In the later diagram called 4-layer trip with 2 cars all the relationships are AND type except car OK which was OR type. Countermeasures can often be represented as OR relationships. Suppose we have a threat, such as the possibility that bad guys may break into our premises, and a countermeasure such as an access control system and we want to represent this idea in a dependency model. Firstly we decide what we are trying to achieve. In this case it's no break ins. Note particularly that it's not an access control system – that's just a mechanism to try to achieve the goal. Secondly we turn the threat into an asset by labelling its paragon no attempts (i.e. no attempts to break in). This is the desirable state of affairs that nobody happens to try to break in. We give it a success probability. Thirdly we then create a paragon to represent the effectiveness of the countermeasure, say access control ok and give it a success probability. Lastly we create an OR relationship:

no break-ins It is a beautifully logical and concise statement of the relationship between the goal, the threat and the countermeasure.

Modelling Basics This Successful Trip model is now multi-layered - three actually - and now consists of these small building blocks, but why is this better? Here's why. 1. A multi-layer gives much us more insight. This will become clear in due course. 2. Surprisingly, it takes a huge amount of effort from us to specify a two-layer model. 3. It's computationally infeasible to do the calculations for a two-layer model with more than a handful of paragons. To understand the problems with 2 and 3 we need to look at how the calculations are done. Here again is a 3-layer version of our model


6-Feb-14

3-layer trip model We have introduced 3 extra paragons, vehicle OK, roads OK and people OK, and the model now contains 11 of them. However we now have a model that is much easier to specify. For instance vehicle OK can be specified by a table with only 8 numbers because it forms a cluster together with its two immediate dependencies, car OK and fuel available. The small table for vehicle OK will now look something like this:

States of dependencies

vehicle OK state probabilities

car OK

fuel available

0

1

0 0 1 1

0 1 0 1

0.9 0.8 0.7 0.1

0.1 0.2 0.3 0.9

Notice that we've put two entries in each row, by having two columns, one for each state of vehicle OK. This form of table is called a Conditional Probability Table because it gives the probabilities for one paragon's states (roads OK) in terms of the states of its immediate dependencies, car OK and fuel available. So for instance the probability that vehicle OK will be in state 1 under the condition that car OK and fuel available are in states 1 and 0 respectively is 0.3.Specified in this way the total number of table entries for each paragon is as follows Paragon#

Table Entries

car OK fuel available weather OK traffic OK roads open we are OK our friends OK vehicle OK roads OK people OK trip successful Total

2 2 2 2 2 2 2 8 16 8 16 62 29


6-Feb-14

Continuing with our model, if we have 2 cars we can take account of this in the model too by introducing the paragons car 1 OK and car 2 OK as “OR” dependencies of car OK: If either of our two cars is functioning then car OK is in its success state. Again we can develop the dependencies of “Fuel Available” and “Roads Open” as being affected by “Industrial Harmony”. Similarly “We are OK” and “Our Friends are OK” could well depend on “No Epidemics”. The model can then be drawn as below:

4-layer trip with 2 cars But while the addition of the fourth layer OR Dependencies “Car 1 OK” And “Car 2 OK” are correct and are a valid countermeasure improving our resilience, the other two Harmony and Epidemic, however, cannot just be added as single dependencies as shown, they become additional third level “AND” dependencies and thus increase the risk of our journey being OK!

Specific meaning of paragon We pause to make a vital point. We must not confuse the paragon we labelled industrial harmony with the everyday meaning of that phrase. Here industrial harmony is the name of a paragon whose associated goal, if articulated would be something like: The state of affairs whereby the likelihood of roads being open and fuel being available are not adversely affected by industrial action. By definition, if our paragon industrial harmony is found to be in its failed state, then since roads open has zero probability of being in its good state unless industrial harmony is in its good state too, roads open will fail. You could find yourself having arguments with non-experts about whether lack of industrial 30 Modelling Basics ds 0.1

6-Feb-14

harmony always entails road closure. This can be resolved by observing that the inevitable logic of our paragons is not true of the everyday use of these phrases, but comes about because of the very specific meaning we've attached to these paragons. The key point here is that there is a huge difference between the technical significance of the paragon industrial harmony and the same phrase used in everyday speech. There is only really a problem because the labels are necessarily short (to fit into a small box on a diagram), but the meaning behind them is complex. In short, we must beware of confusing the meaning of a paragon with its label. The label is merely a hint or reminder of the actual definition of the paragons. Another reason for being fussy about the definition of a paragon is that it affects the answer. How is this? It's because the statistics we give for the paragon depend on the definition. To avoid later misunderstandings we're going to introduce a new word. We loosely described trip successful and car OK and all the other entities as “sorts of goals”. This turns out to be misleading. These entities are much more specific than “goals”. Moreover for most people the word “goal” is loaded with unhelpful associations. In fact there is no word in English that adequately describes what the entities trip successful, car OK, etc. represent in our model, so we're going to invent one. The word is paragon. As well as a name, paragons have a meaning expressed through their definition. The name cannot convey much information, but it alone appears on the diagram. It is important not to confuse the definition of a paragon with its name, which merely serves as a shorthand reminder. Paragons must be precisely defined. When we said they were abstract we didn't mean vague. A well defined paragon would be along the lines of To prevent attackers or their agents from entering company premises, or introducing malicious software into company computers or networks at any time between midnight January 1st 2011 to midnight January 1st 2012, or to detect their presence in a sufficiently timely manner as to be able to prevent any of the losses catalogued in list 1. We've already seen an example where we described the significance of the paragon industrial harmony as the state of affairs whereby the likelihood of roads being open and fuel being available are not adversely affected by industrial action.

A state of affairs is a good opening phrase in a definition. Another good template is the effectiveness of (whatever) in doing (whatever) in such a way as to achieve (whatever). Having the paragon definition right is vital to the accuracy of dependency modelling. Many models are flawed due to inconsistent definitions, or to the use of the same paragon at several points in the model, each one requiring a slightly different definition.


6-Feb-14

Uncontrollables, Trees and Dynasties The paragons on the extreme right of each branch of the model don't have dependencies, or at least they're not shown in the model. That's just as well otherwise the model would never end. Dependencies can have their own dependencies and so on to any number of layers we like and we stop adding more when we feel we've gone far enough. Any paragon with no dependencies acts as a sort of “given” - a starting point. We can think of it as the point where risk and uncertainty enter the model. We call such paragons uncontrollables to emphasise that we can't do anything about changing their properties. That doesn't mean we cannot reduce the risk they pose. It turns out to be quite simple to reduce such risks, but more on that later. We specify uncontrollables simply with a table with no dependencies. Here is an example for weather OK. State Probabilities for weather OK p0 p1 0.05 0.95 where p0 and p1 are just names for the probabilities that it will be in its bad and good states respectively. Notice that we've written them both in the same row. Think of it is as one row – the only row in this case - of a conditional probability table.

Independence Uncontrollables must be statistically independent. Any statistical interdependence must be explicitly expressed by cross-linking with one or more common dependencies, which may entail introducing extra paragons. For instance, recall that we introduced a correlation between roads open and fuel available by arranging that they shared the same level as the mutual causal uncontrollable industrial harmony.

Depth of Model and Granularity Our model can have few or many layers. It is always possible to convert an uncontrollable into a dependant by merely presenting its dependencies (or obtaining status probabilities directly or from a common dependency in another model) – after all everything has dependencies – and in this way we can increase the depth to embrace more and more, lower level dependencies. We stop this drilling down process when we feel we've gone far enough. The leaves at that point are the end-stops, the givens, the uncontrollables. A model with many layers which starts from a major goal such as the success of a department, and ends with very low-level dependencies such as the effectiveness of a stapling machine, could also be described as having fine granularity. Similarly a model with few layers would have coarse granularity.

Risk Trade-offs Because we are dealing with a model of an enterprise and not the enterprise itself we can easily answer the what-if question: “What would happen if such-and-such were different?” 32 Modelling Basics ds 0.1

6-Feb-14

We don't have to modify the enterprise, just the model. For each variation we can evaluate the risks. There are many ways of measuring risk, such as sensitivity, criticality or failure modes. By making changes to our model we can compare the various different ways of setting-up or modifying our enterprise in terms of the different risks each entails. Each of these variations would have a different cost which an accountant could evaluate. And each would have different values on intangible scales involving subjective judgements of an aesthetic, social or moral nature. But what has been missing from the equation is how the variations compare in terms of risk. However we have seen that Dependency Modelling now brings several way of comparing their risks. This should enable us to make informed decisions involving trade-offs between risk, cost and intangibles. We could for example reduce cost at least increase in risk, or reduce risk at least increase in cost. Or we could increase the aesthetic appeal with the least increase of cost or risk.

10. Case study 4 - West of Scotland Water: Cutting capital expenditure without sacrificing standards

Budget cuts are a fact of life in today’s modern business world, but the real challenge is ensuring that it does not have a detrimental effect on the business. Knowing where you can afford to reduce spending and where an organisation should increase its investment is a growing issue for companies with diverse business needs. West of Scotland Water, now a part of Scottish Water, was required by the water regulator to submit plans for a reduction in its capital investment plan as part of an ongoing restructure of the entire water industry in Scotland. Part of this restructure was the creation of Scottish Water, which amalgamated the then three water companies in Scotland into a single authority that


6-Feb-14

would be better placed to offer customers value for money, whilst providing a cost-effective means to manage the future investment required in the overall business infrastructure. Before the merger took place West of Scotland Water was required to show how and where reductions in its planned investment could be made, without jeopardising any of the business values. Senior management had just eight weeks to put forward new plans that included the proposal for a reduction in costs of £700 million. Alan Scott, Network Planner of (WoSW/SW) takes up the story of how they completed this challenge: “We needed to review the capital investment plan and to assess the relative impact of any potential deferrals or cuts on West of Scotland Water’s business values. To ensure that every possibility was considered we required a method that would enable us to compare the relative effectiveness of each of the different investment lines in mitigating the associated risks. It was also hoped that this would then provide the basis for identifying a hierarchy of risks and mitigations for the potential development of a future company-wide risk register. We did consider FMECA (Failure Modes, Effects and Criticality Analysis), but it was felt that this would not be appropriate due to the short time scales and data available to us. Chris Baker and his colleagues were recommended to us by another utility company and after a brief period of consultation it was decided that dependency modelling would suit our needs.” Dependency modelling allows for the simple representation of complex risk scenarios. By using simple AND and OR dependencies on strategic business values, it is possible to demonstrate clearly the existence, or otherwise, of alternative strategies or contingencies. The risks to these business values are calculated using input from three different areas; the likelihood of risk materialising, the cost of failure should the risk materialise and the effectiveness of implementing controls to mitigate the risks. Once the risks to achieving the values are mapped and understood, a comprehensive set of controls and countermeasures can be implemented to reduce risks. West of Scotland Water undertook an intensive series of risk workshops using iDepend®’s toolkit, which were attended by senior technical staff. These workshops helped to define and develop the business values and their relative importance to the business. This information was then applied to the dependency model where the major failure points and their likelihood of occurrence, were identified. Once this had been completed it was then possible to identify the capital investment required to mitigate the risks in correlation to the failures. “The workshops were helpful in understanding the impact of the Capital Investment Plan in line with our objectives, but the results were dependent on the availability of the expert opinions of our own asset management practitioners and asset operators. As with any analysis of a business’s infrastructure it’s the workers out in the field that have the knowledge of the implications if something was to fail. Though dependency modelling gave us a clearer understanding of the risks and their impact, it was the individual input from West of Scotland Water’s staff that helped the project to be a success. I would recommend to


6-Feb-14

anyone considering dependency modelling to ensure that all relevant senior personnel are included in the initial stages.” The results of the dependency modelling provided West of Scotland Water with an increased efficiency in decision-making, a strategic plan for investment and clarity of the roles and responsibilities of individuals involved. But the ultimate success was the reduction of £700m in planned capital investment and within the time scales set out by the regulator. Alan Scott concludes, “The savings were made in several areas, some of which were obvious but others would not have been considered. In addition, some planned deferments were reviewed as a direct result of the dependency modelling. We now have a clear understanding of the primary purpose of each of our investment lines and its contribution to mitigating risk to the business. This in turn has given us a structure that reflects the proposed optimum investment plan and allows us to demonstrate that all risks have been considered”

11. Tips and Pitfalls in building models When people first start to build models there are a number of common pitfalls that often occur. They all come down to building a model with logical flaws in it. It's worth going through a few.

Failure to rip up and start over – sunk costs People want their model building to be a logical linear process, but that's not how it works. When you first start to build a model it usually happens that you soon realise there's a much better way to do it than your initial choice. This is fine - just rip it up and start over. You learn most from the bad models you discard. The person who rips up nine models and presents only the tenth, has a much better grasp of the risk structure of the enterprise, than the person who only sees the final version. A dogged insistence in not wasting effort already spent is false economy and a failure to learn from experience. It's also incidentally an example of the Sunk Costs Fallacy that fuels gambling addiction.

Failure to define a paragon Many faults really come down to ill-defined paragons. This is an easy mistake to make and comes in many forms. The version we'll look at here is when we hastily create a paragon and use it in several places in the model without noticing that it needs to have slightly different meanings in some places. For example we might make a statement to the effect that all our widgets will turn out perfect 35 Modelling Basics ds 0.1

6-Feb-14

if either there are no manufacturing errors on our widget production line OR our widget quality-control procedures weed them out. We could probably come up with paragon definitions that would render this statement correct. But if we don't, it can be awfully tempting to use say no_manufacturing_errors somewhere else on the model, where it now refers to a different production line, or to use quality control ok in a second context where it refers to something slightly different.

Inappropriate correlation through re-use of paragon Whenever a paragon is re-used in a model it refers to exactly the same instance of the same paragon, not another one just like it. So if we have a paragon called interruption-_free electricity and we use it twice, then it refers to exactly the same electricity supply, right down to the same generator and supply network. Whenever the one fails, so does the other, at exactly the same moment, because they're the same thing. This implies a perfect correlation between events that will make a mockery of any countermeasure exploiting the fact that they should be uncorrelated. If you need two similar paragons that behave independently you must create two, with different names, like supply1 and supply 2.

Definition drift Sometimes as a model is worked on, the meaning of a paragon subtly becomes changed without the designer noticing. For example a paragon might mean the state of affairs whereby a computer performs faultlessly between two given dates which are common to all paragons in the model. Later when the time-frame is altered the designer may forget to readjust the probabilities. Another example is where a paragon refers to the reliability of a particular computer, then to another computer, and later still to the set of all computers in a department

Paragons with just one dependency Since a paragon plus its single dependency may both be replaced by just one paragon, we should be suspicious of single dependencies as they may point to a risk analyst who doesn't understand what he's doing. However there are sometimes good reasons to show single dependencies. One is where several paragons each have the same single dependency, i.e. it is a common cause deliberately introduced to produce correlation. Another reason might be to emphasise a point to a particular audience. But we can’t compute it with iDEPEND!

Mixing different philosophical planes Models can come at various philosophical levels or planes. For example one model might be concerned with the reliability, efficiency etc. of various physical entities and the effectiveness of various procedures. We can think of this as a concrete approach. By contrast there could be a model concerned with the effectiveness of rather ethereal 36 Modelling Basics ds 0.1

6-Feb-14

concepts such as SUCCESS

= INTEGRITY AND CAPABILITY AND COURAGE AND LUCK

This is an extreme example, but I've seen people writing models this way. There might be a problem tying down the meanings of the various paragons, but given suitably creative definitions it could probably be made to work to some extent, especially if it were used tongue-in-cheek to make a point as part of a staff training program, because then one could get away with sloppy reasoning and it wouldn't really matter. But what wouldn't work is trying to mix the two approaches, since it would be hard to find any paragons that could join the two approaches together.

Use of non-paragons Some concepts can't be rendered as paragons. For instance the success of expanding an established business into another country might depend on  the business climate in that country  whether certain pre-requisites are legal in that country  whether staff costs are affordable there  etc The second of these, whether something is legal, is a perfectly sensible consideration, but it isn't a paragon since it can't be given meaningful statistics. It's either true or false. In a given country it’s not true say 80% of the time at random. Such a model may look sensible but it's sloppy reasoning, and anyone looking at it would realise the author didn't understand the modelling process. It comes down to the everyday use of sensible statements in common language, and the need to be precise in a risk model: most sensible statements don't convert to risk models.

Use of ready-made lists Organisations are fond of compiling lists. Beware of adopting them as paragon statements. For example an organisation might have a department hierarchy diagram and it would be tempting to make a top-level statement along the lines

ORGANISATION _OK

= DEPT1_OK AND DEPT2_OK AND DEPT3_OK AND DEPT4_OK …

The reality is this will turn out to be a big mistake unless you pay a great deal of attention to the precise definition of each paragon. You would have to define exactly what is meant by say dept1_OK. It would have to include defined states each with a probability; a reference time-frame for those probabilities; it would have to be spelled out exactly what constitutes success and what failure; it would need a precise list of its dependencies; you could not use the same paragon somewhere else to have a slightly different meaning, and so forth. Getting it right could be a nightmare. Also by implication the success of the organisation is defined as the success – whatever 37 Modelling Basics ds 0.1

6-Feb-14

that means – of each of the departments, which might be not what was intended. Basically don't do it.

Mission statements and respectable goals Catchy statements don't usually make good paragons. Many organizations have a mission statement. It is usually a mistake to adopt it as the enterprise's main goal paragon. For instance a mission statement for a hospital could be along the lines To provide a quality service and value to the health authority while respecting the dignity of our patients and treating our staff with courtesy and respect. However if you ask the senior management in a hospital what risks they fear most they are likely to be 

Closure due to a change of government policy.



Losing budget due to failure to meet government targets



Losing out to competition from the private sector



Giving permanent brain damage to a one-year-old child and being sued.

Another organisation's management might have as its aims To provide value and courtesy to customers, returns to shareholders and respect to employees But in reality their true concerns might be 

The government will declare us a monopoly and we'll have to merge



A change in government policy will wreck our profitability or stop us paying our senior staff big bonuses



Our CEO will lose interest and sell off the business to an organisation we dread



Lower labour costs in the Third World will lead to closures

These don't get a mention in the list of goals. A consciously instrumented risk management policy stands a fair chance of achieving its aims. But these may not be what the management really wants. Unless your overt goal reflects your true concerns you may achieve what you said, not what you meant. Be careful what you wish for.


6-Feb-14

12.

Advanced Users - the Maths

Apart from death and taxes, so the saying goes, nothing in this life is truly certain. Equally no physically possible event can be dismissed as having zero probability. This uncertainty (“on objectives “) is what the latest ISO standard (31000) actually defines as “RISK”. So in real life we have to cope with making decisions under uncertainty, somehow guestimating the “odds” to help us rationalize our “instinctive” preferences. But for important decisions, transparency, governance and integrity, demands a more responsible approach – we need a more rigorous justification of the probabilities (we assumed?). Probability Theory is aimed at addressing this need. There are a few simple rules, like:   

The probability(P), of an event A occurring is P(A), a number between 0 and 1 The probability of the “exhaustive” event P(Ex) is the sum of all possible events in that situation; which in any given case is P(Ex) = ∑P(E1) + P(E2) + ------ =1 The probability of two completely independent events A and B happening is the sum of their respective probabilities P(A)+P(B) The probability that an event B occurs, given that A has already occurred is a CONDITIONAL PROBABILITY P (B I A) = P (A∩ B)/ P (A) = P (A, B)/P (A)

In practice all real life probabilities are shaped by context and background and hence they are “conditional”! So this is often referred to as THE FUNDAMENTAL RULE!

Bayes Theorem follows from the above rule set

This gives us a simple method to calculate these conditional probabilities.


6-Feb-14

P (B I A) = [P (A I B) x P (A)]/ P (A) This can be used to update the probability P (H), of a Hypothesis, H, given the existence of Evidence, E, with a probability of being correct P (E) P (H I E) = [P (E I H) x P (H)]/P (E) This allows us to update a prior (guessed) probability for some unknown event when we see evidence about the event. Again in the real world there will be many unknown events and many different pieces of evidence. We can represent this situation better graphically, representing the uncertainties as nodes, linked together to show their relationships, as a Bayesian network (BN) see definition from Fenton and Neil (see below)

Bayes theorem can then be applied sequentially, correctly updating the prior estimates with evidence systematically through the network.

In iDepend this can be modelled as an ‘AND’ relationship.


6-Feb-14

Thus, the Prior estimated probability of the Home OK is taken as the first Dependency probability - the Nutrition Probability P (N) P (H = True) = 0.9 But if we add another dependency, AND Sanitation, P(S), we need to update this prior estimate, P (H=T), to P (H=TI N, S). Thus based on the input probabilities of the “leaves”, P (N) and now P (S), the model predicts that the probability that the home will be OK, given the evidence (probabilities) of N and S, as P (H=T) = P (H=T|N, S) * P (N) * P (S) Computation of the above equation,(Where T = True and F = False), gives:=(0.9)*09*0.7 = 0.567

These probabilities are shown by the red and green bars in Figure 2

FIG 2 – Calculated Probabilities displayed in iDEPEND

iDEPEND has been developed to be both user friendly and very agile (to handle efficiently very large networks). The algorithms therefore allow a user to build and analyze their models without having to worry about the details of the calculations. So in iDEPEND Basic mode (automatic), the networks are assumed to be pure BOOLEAN logic gates (which can be either AND, or OR T junctions).

This means that: 

Single dependencies (parents) merely transmit the “Leaf” probabilities upwards through the gate to the next node and Mixed Logic Gates (AND’s and OR’s) are modeled using, an additional OR gate to link the two separate groups of dependencies.

Experienced users, however can use all the features of iDEPEND which allow multiple states (not just ON or OFF, True or False) for entities, so one on one now makes sense with NPT’s for different states. For example a rolled dice could be said to have 6 states.


6-Feb-14

It also allows “Custom” conditional probability tables instead of set Boolean AND or OR logic, to determine the relationships at the NODE (or GATE). The purpose of the Conditional Probability Distributions (CPDs) in a BN is to quantify the strength of the relationships that are defined within its structure. The Conditional Probability Distributions can be defined as: (a) (Conditional) Node Probability Table (NPT) – for discrete variables and (b) Conditional Probability Distributions (CPD) –for continuous variables So for our example we can construct and insert a “custom” probability table.

All of this is conventional Bayesian Belief Net methodology, but 

This iDEPEND software now enables its deployment as a simple technique/ process –

Dependency Modelling; and,   



a simple, visual, drag and drop mind mapping tool. It looks and feels intuitive and simple to use. Changes in network logic, structure and input estimates are easily made (tear up and start again!) and this is the last time we need to bother about the mathematics, the computer will take care of it for us!


6-Feb-14

13.

Uncertainty and Risk

How do we measure risk? There are as many definitions as there are authors on the subject. Many people simply use probability of disaster. Assuming that disaster means failure to achieve a particular goal, then Probability of disaster is just the size of the red part of the appropriate band. However we're going to suggest a different measure that better captures the idea of risk. In popular imagination risk is more than just the probability of success or failure to achieve a goal. An ancient illustration, beloved of insurance actuaries makes the point. Traditionally, gunpowder factories were always blowing up. They did so with such frequency and reliability that insurers soon learned the odds and merely fixed the premium so they still made a profit. This is why a high failure probability is not the same as a high risk. And it is almost certainly not what most people mean by risk. Gunpowder factories can be relied on to blow up, whereas risk is more about uncertainty. By contrast suppose you are considering investing in a business-plan written by an enthusiast, for a novel enterprise in an untried market whose success depends on many imponderables, and you suspect unjustified optimism may have crept in. You are probably most worried by the possibility of some unforeseen or ill-understood circumstance that could overturn all the assumptions behind the business model and take you by surprise, causing you to fail to achieve your goals. This sort of uncertainty is how most people think of risk. It would be good to capture risk as the dependency of goal-achievement on uncontrollables those elements of reality that cannot be fully controlled or understood. This definition is intuitively satisfying, and given a more precise formulation, this risk metric may be determined as an actual number. By careful refinement of the definition we can draw together a number of key issues into an elegant measure that addresses risk, uncertainty, statistics and ignorance. To get started, suppose we calculate the probability P that we will achieve some goal Y to be (for the sake of argument) 80%. In making this determination we will have used certain figures in the model. Two important questions that naturally arise are “Where do we get these figures from?” and “What if they are wrong?” Both questions are highly relevant and both figure in the determination of s-risk. Clearly if the numbers are wrong then our calculations will give the wrong answer for P. However that probability was not how we choose to define risk. Recall the gunpowder factory. The probability that the factory would blow up was not the same as the risk to the enterprise. The risk, in everyday understanding, is more about uncertainty that we've got the picture right and the implications of being taken by surprise by something unforeseen. Suppose there is some uncontrollable X (it might for instance be freedom from industrial action) about which we know very little except that it is an uncontrollable in our model. Suppose the a-priori success probability we'd used for X was some number Q. In other words P was calculated using the value of Q. 43 Modelling Basics ds 0.1

6-Feb-14

Clearly if we change Q then P will change. However in reality we will have only a rough idea of the “true” value for Q. So how shall we handle this? One obvious thing we might try is to change the value of Q and see the effect on P. It turns out that if we keep all the other figures constant the relationship between P and Q is linear, as in the figure below. Note the values marked P0 and P1 - the extremes of the range of P as Q varies over its full domain from 0 to 1.

Now we come to the nub of the argument. If ΔP = P1-P0 is small then we don't need to know Q very accurately. If on the other hand ΔP is large then the value of Q has a very important effect on P. Suppose we find that ΔP is large, and we have almost no idea of the correct value of Q, then this is unquestionably a highly risky position to be in. Our goal Y depends crucially (since ΔP is large) on something X, whose statistics Q we know little about. But from our preamble about the uncertain enterprise, this is precisely what risk is about – and ΔP has just measured it for us! So if we define Risk = R = ΔP then R is an obvious candidate for a measure of risk. Notice that because ΔQ = 1, then R = ΔP/ΔQ. Also, because the relationship between P and Q is linear, Risk = R = dP/dQ. This is just the sensitivity of P to the value of Q. R is always less than 1 because both P0 and P1 are less than 1. So Risk R is just the sensitivity of the probability P we're interested in (that we will achieve our goal) to the probability Q of something we can't control, don't fully understand, and whose value we don't know accurately

The sensitivity to uncontrollables - (An Example of Risk?) The diagram below shows the sensitivity of our goal home OK to each the uncontrollables in the dependency model. For example its sensitivity to freezer is 0.152 (it's shown in red numbers in the diagram). If the a-priori success probability of freezer changes from 0 to 1 then the success probability of home OK changes by 0.152. The diagram shows more than this. It actually shows the two probabilities P0 and P1 as the freezer probability changes from 0 to 1. They are about 0.37 and 0.52. 44 Modelling Basics ds 0.1

6-Feb-14

The interface between the red band and the green band is the nominal success probability of home OK when all the uncontrollables have their nominal a-priori settings. The red parts show how much worse the success probability of home OK could be as the probability of each uncontrollable is reduced, and the red parts show how much better it could be as the probability of each uncontrollable is increased. Because it shows these three probabilities for each uncontrollable it's called a 3-point-sensitivity chart.

Reducing the risk Look at the 3-point-sensitivity chart for main TV. The sensitivity, or risk is only about 0.051. However if we introduce the extra bit of information that the spare TV is not functioning this is how the picture changes.

By removing the spare TV the risk represented by the main TV has leaped from 0.051 to 0.512. Or putting it another way, by introducing a spare TV we can reduce the risk from 0.512 to 0.051.


6-Feb-14

14.

Failure Modes

It's worth saying what we mean by a failure mode. If M is a set of one or more uncontrollables and G is a goal, then M is a failure mode of G if and only if  

G inevitably fails if every uncontrollable in M fails, but G doesn't fail unless all the uncontrollables in M fail.

The number of uncontrollables in the smallest failure mode M is an important measure of risk. The phrase "belt and braces" is an everyday expression suggesting a failure mode of size 2. If M consists of just one uncontrollable then M is a single-point failure. This is normally considered a serious risk because a single item going wrong can cause a disaster. The larger the number of uncontrollables in the smallest failure mode, the more things have to go wrong for the goal to fail, and the more resilient and less risky is the enterprise.

Custom types Earlier we mentioned that in addition to AND-type and OR-type relationships we could have CUSTOM-types. The nature of the relationship is specified by a conditional probability table. Here's the one for entertainment on the home OK model. TV no no no no yes yes yes yes

antenna no no yes yes no no yes yes

electricity no yes no yes no yes no yes

prob of state no yes [1] [0] [1] [0] [1] [0] [1] [0] [1] [0] [1] [0] [1] [0] [0] [1]

This is a typical table for an AND-relatioinship. All the elements have just two states called no and yes. The probability of the yes state is zero unless all three of TV, antenna and electricity are all yes. The probabilities in each row must add up to 1.


6-Feb-14

Here is the table for TV, with dependencies main TV and spare TV main TV no no yes yes

spare TV no yes no yes

no [1] [0] [0] [0]

yes [0] [1] [1] [1]

This is a typical OR-relationship. Now there is no reason to limit ourselves to just these two cases. Here is an example where we capture the idea that even when TV, antenna and electricity are all present there is still only a 95% chance that we have entertainment. TV no no no no yes yes yes yes

antenna no no yes yes no no yes yes

electricity no yes no yes no yes no yes

prob of state no yes [1] [0] [1] [0] [1] [0] [1] [0] [1] [0] [1] [0] [1] [0] [0.05] [0.95]

We can model a wide range of relationships by cunningly specifying the conditional probability table of entities.

Take a look at this model.

Two ways of organizing a filling-station forecourt It shows two ways of organizing a garage filling station. Each of the ways is show as a different goal.

The garage has four appliances - two pumps and two pay-points (tills). The first setup is called Islands where the appliances are organized as two islands each with a pump and a till. 47 Modelling Basics ds 0.1

6-Feb-14

Each island as an AND-relationship of a pump and a till, while Islands is an OR-relationship of the two islands. In the second setup, called Functions, the plumbing and wiring is slightly different so that either till works with either pump. It's organized by functionality, and there are two function called Pumps and Tills. Pumps is satisfied if one or more pumps is working, so it's an ORrelationship of the two pumps, and Tills is similar for the tills. However Functions is an AND-relationship of Pumps and Tills. As the diagram shows, Functions is more likely to succeed than Islands, which some people find surprising. A good way to help people understand why this happens is to show the failure modes. These are the four failure modes for Islands:

Four failure modes for Islands


6-Feb-14

The entities with red borders fail. The failure is caused by the uncontrollables failing on the right. For instance the first failure mode is caused by Pump-1 and Pump-2 failing. As a result both islands fail, and hence Islands fails. These are the two failure modes for Functions:

Two failure modes for Functions The obvious difference is that Islands has more failure modes than Functions. Moreover every failure mode of Functions is also a failure mode of Islands and so those occur with the same probability in both models. The fact that Islands has more ways to go wrong is pretty convincing as an explanation, even for someone with a non-technical background.


6-Feb-14

15.

“Bigger” Model Building

If we analyse a large enterprise such as the infrastructure of a country the immediate dependencies might be things like the machinery of government, armed forces, security services, telecommunications systems, transport systems and so forth. If instead we were concerned with the success of running a village fête our top level paragons would involve the safety of the events, the insurance, recruiting stallholders, printing of tickets, reliability of bouncy castles and so forth. If we pick a household our immediate concerns might be weatherproofing, sanitation, cooking, relaxation, security and so on. We quickly see that there is no such thing as a typical example. But whatever we pick similar issues will arise. However if we pick something of serious concern to everyone such as National Security, the reader is likely to focus on whether the model reflects his particular concerns, rather than on the modelling issues themselves. Therefore we will pick something more neutral that most people can understand but few will have issues with. We're going to pick the successful completion of the flight of a passenger-carrying jet. We want to restrict the complexity of the model so we'll limit ourselves merely with the flight from the moment the passengers are seated on the plane to the moment the plane comes to rest at its destination. We ask ourselves what would have to happen for the flight to be deemed successful. In other words what would be the definition of our main goal paragon, flight OK? We might decide that a successful flight would entail the following requirements: no crash

The plane must not crash

no bomb

No bombs must explode on board

no hijack

The plane must not be hijacked

takeoff ontime

The take-off must not be seriously delayed at the last minute

land ontime

The landing must be more or less on time

tranquillity

Passengers must not get into fights or cause disruptions

smoothness

The ride must not be bumpy

no stack

Air traffic control must not require the aircraft to stack for more than 15 minutes

no diversions

The flight mustn't be diverted to another airport

food safe

There must be no food poisoning

food hot

The food must be hot

entertainment ok

The in-flight entertainment electronics must work OK

We could go on, and different analysts would have different lists, but that's enough for this example. Basically we want all those things to happen before we deem the flight successful, so an AND relationship suggests itself. 50 Modelling Basics ds 0.1

6-Feb-14

The consistency rule tells us that the main goal paragon that results from this choice is in effect defined by its dependencies, so by definition a successful flight is one with all these attributes. We have an immediate problem here. The attribute plane must not crash would probably be regarded as much more important than entertainment must work OK. If we lump them together in an AND relationship they will be forced to have equal weight. There are at least two ways we can handle this. Firstly we could let flight OK have three states with meanings something like:   

State 0 = failure State 1 = at-least-there-wasn't-a-crash State 2 = success

A second way to have flight OK have two immediate dependencies representing essentials and desirables; and to let the former cover the life-threatening aspects, while the latter handles the niceties. This gives us two goals rather than one, but if we feel this is aesthetically unsatisfactory we can always tie these two together for the sake of appearances as sub goals of a main goal flight OK, but of course the sub goals are really the only important ones. Like this:

First level There is nothing wrong with either approach, they are just different styles. Here we'll pick the second because if we introduce a 3-level paragon we make failure mode analysis problematic. Let's see if we can partition some of the other paragons together. Under essentials we'd probably group no crash, no bomb and no hijack. Everything else would come under desirables, like this:

Second level


6-Feb-14

We'll lump takeoff, ontime, land ontime and no diversions together under to schedule. Let's fill in some details. When modelling a something like no bomb – meaning that no bomb is brought onto the plane and goes off during the flight - we would like to represent in our model the tension between potential terrorists and the security specialists trying to thwart them. We can do this by introducing two paragons, no bombers and bomb security ok and use an OR relationship type countermeasure that we saw earlier. No bombers is the desirable quality that no bombers make attempts to bomb the plane – an uncontrollable. Bomb security ok is the desirable quality that the security specialists manage to thwart the attempt. We simply arrange these in an OR relationship. That way we can separate out the threat and the countermeasure, and give each a probability. Here's the result:

no bomb paragon We next ascribe the success of the paragon no crash to a combination of the plane mechanical parts being in good shape (mechanicals ok), and the weather (weather ok). In turn mechanicals ok is a paragon with dependencies the effectiveness of flight maintenance (flight maint ok) OR no failures, in an OR relationship. Notice again the use of the OR relationship to represent a threat and a countermeasure with the threat being inverted into a goal in effect as no threat. It's important to note that this is a very simple model that does not go into much detail. We could expand uncontrollables by giving them their own dependencies, thereby drilling down as far as we like. The depth, or number of layers in a model is governed by the purpose for which is developed. For most practical purposes we can regard this as two models, loosely connected for convenience onto one diagram, one each for essentials and desirables. However they're not statistically independent since they share the paragons weather ok, mechanicals ok, flight maint ok and no failures. To make it more interesting we've made the success probabilities of the uncontrollables unrealistically pessimistic. Nothing has a higher success rate than 0.99. At first sight the probability of a successful flight is scarily low, around 60%, but on inspection this is almost entirely due to the success probability of desirables. The success probability of essentials is about 99%. We continue along these lines, and leave the reader to work out the details for himself. We arrive at something like this:


6-Feb-14

Flight ok

Flight model flaws. This model has a number of flaws of varying degree of seriousness. However rather than just fixing them, we'll give you a chance to see how many you can find. (Give up? Then here are some faults in the flight model, we identified earlier!) 

The paragon mechanicals ok is used twice, as a dependency both of no crash and of no diversions and it probably means something slightly different each time. While many mechanical faults might affect both paragons, there would be others that would only affect one.



The paragon weather ok is also used twice, and clearly with different meanings. It is a dependency of no crash and land ontime and clearly different kinds of weather are implied. 53


6-Feb-14



Similarly airport ok is used twice, once as a dependency of no stack and again as a dependency of no diversions. In this case it is possible that the same kind of problem could have these two distinct outcomes. For instance closure of the destination airport could cause both effects. So this is a less glaring error.



Similarly other traffic ok is given as a dependency of no stack, takeoff ontime and no diversions and it doesn't necessarily mean the same thing each time.



Again air traffic contr ok is common to land ontime and takeoff ontime, possibly with different meanings.



The paragons caterers OK and services OK are single dependencies and are either a part of a fuller description of Food Safe and Food Hot or are additional And dependencies of Cuisine (OK?)



When we re-use a paragon as a dependency of two dependants, these dependants become strongly correlated, i.e. when the one fails the other is much more likely to fail. This may be exactly what we intended, but then again it may not.

(We can take precise control of correlations by using different paragons for the different contexts, such as other traffic ok 1 and other traffic ok 2. These two would then be uncorrelated. If we want to introduce a carefully controlled amount of correlation between them we can do so like this

Controlled correlation Here common cause is common to both other traffic ok 1 and other traffic ok 2, but each also has its own independent random component. The relative contribution of these components can be controlled by the probability tables.) These are very common flaws, and easy to fix. .Debug

as you go

Authoring a model in one go before analysing it is like writing a software application without testing any of its modules. It has almost zero chance of succeeding. Here's just one example of the sort of problem that arises if you don't check it out one step at a time. When we first did an analysis for Fidelity Investments their risk department produced a model that suggested very high resilience to attack. Yet history had shown this not to be the case. So something was probably wrong with the model, but what? After a while we tracked it down and here's what we found. The model contained a vault in a secure building surrounded by various perimeters that were supposed to resist penetration. 54 Modelling Basics ds 0.1

6-Feb-14

The model implied that the vault would be safe if either (1) there were no attacks, or (2) the perimeters were effective against attack. The perimeters were things like walls, barriers, barbed wire, passwords, ID checks etc. This statement is illustrated below for just three such perimeters.

Vault 1 model The model tells us that the array of perimeters is effective if at least one is effective, so it's an OR relationship. If we arbitrarily say each perimeter is 99% effective and assign an 80% probability to no attacks these are the state probabilities for the elements:

ELEMENT STATE PROBABILITIES

vault 1 OK Perimeter 1 protects Perimeter 2 protects Perimeter 3 protects No attacks Perimeters effective

Fail

succeed

0.00000020 0.01000000 0.01000000 0.01000000 0.20000000 0.00000100

0.99999980 0.99000000 0.99000000 0.99000000 0.80000000 0.99999900

We see that the probability that vault 1 OK fails is about 2x10-8 which is clearly daft since it was burgled twice in a 5-year period. But Fidelity Investments couldn't find what was wrong with their reasoning. So what's wrong? If someone had already managed to scale a wall and bluffed their way through an ID check are we really saying they are no more likely to get over another hurdle than someone who hadn't managed the two previous feats? Clearly not. Anyone who can clear a couple of barriers obviously has something special going for them. Maybe they have a helicopter, know a password, or the guard is their brother-in-law. It doesn't matter how they did it - the point is these perimeters aren't as independent as the model suggests and something that would penetrate one perimeter might help in penetrating others. There are dozens of ways we can fix this error, but here's a very simple one: 55 Modelling Basics ds 0.1

6-Feb-14

Vault 2 model Here we've introduced a generic no sidestep paragon AND-ed with perimeters effective to represent something that might bypass those misleading perimeters. It might be a forged ID badge, a captured password, blackmail data or a magic broomstick - it doesn't matter what it is - but it's a safe bet that all those perimeters OR-ed together give a misleading sense of security. These are the probabilities for the elements in vault 2 OK

ELEMENT STATE PROBABILITIES vault 2 OK perimeter 1 protects perimeter 2 protects perimeter 3 protects no attacks perimeters effective no sidestep measures effective

fail 0.04000016 0.01000000 0.01000000 0.01000000 0.20000000 0.00000100 0.20000000 0.20000080

succeed 0.95999984 0.99000000 0.99000000 0.99000000 0.80000000 0.99999900 0.80000000 0.79999920

Step by Step It was easy to spot this error because (1) it's a well-know classic, but more importantly because (2) the model isn't cluttered. If the error were any one of dozens of much more subtle pitfalls, and if it were buried in a complex rat's-nest of many other elements, and if all that alerted us was that the model gave suspicious results, we would be hard-pushed to find what's wrong. So the way to avoid these pitfalls is to treat it like a software development project. We develop a model one step at a time, at each stage analysing it to see if it makes sense and is consistent with the earlier, simpler version. That way, if we expand the model by one step and it suddenly stops behaving sensibly we know we just introduced an error, and we know exactly where it is. Imagine if we'd written an entire model with 100 paragons, and unbeknown to us it contained many errors, but by chance some of them offset the effect of others so that the entire model seemed to behave sensibly, we might never even look for an error.


6-Feb-14

16.

Modelling Interdependencies The Dutch Example (from El Metodo)

“Consider a Business unit (functionality) that provides IP-connectivity between various locations.” Its manager defines the “objective “ or goal of this functionality and models what he is in turn dependent on to meet his obligation, for example • IP-connectivity needs to be provided for at least 99.9% of the time. • IP-connectivity services need to comply with the Data Protection Act. • Annual revenue of IP-connectivity Unit should be at least € R and annual costs are at most € C. A manager can only meet his “goal” if he can rely on his dependencies to be fulfilled, either  

within his own scope of control (internal dependencies) or, by some other business unit or functionality (external dependencies)”.

Below is the screen shot of the simple iDEPEND model of this case. We can see that he is dependent on his network infrastructure working, his “boxes” functioning correctly and reliable (maintained and serviceable); which requires as an example, the routers to be available; and an electric power supply connected and working. The System for providing IP connectivity.

Fig. 1: Linking paragons (Obligations) to Goals and Dependencies in unit IP connect


6-Feb-14

In turn, this depends on the glass fibre network providers doing their job (– twice for both an A – net and a B-net.). These units can now be modeled in the same way with the paragons A Net OK and B Net OK being common dependencies of the two separate systems –IP connect and the Glass Fibre provider systems.

Glass Fibre System models A – Net and B-Net Optical transponders A-net work

tical transponders

BBB- Net B - Net

B-net work

Fig. 2: Linking Obligations and Expectations Between Functionalities in the separate systems

In the Dutch paper “Figure 2 illustrates the linking of expectations of the functionality ‘IPconnect’ with obligations of the functionality ‘Glass’. Not only does this provide insight in functional dependencies, it also provides a means to automate risk management, as we shall see in the next section”. iDepend clearly accomplishes these requirements and more.

Results and Discussion In the modelling sequence, the software prompts the user to specify not only the gate logic (And / Or) for the dependencies, but also the probabilities of the dependency paragons to be in the states required (expectations). This is accomplished by typing in a dialogue box or estimating from experience using a “slider”. The availability of A Net and B Net are automatically updated from the other business unit models and the Maintenance performance from the SLA’s. 58 Modelling Basics ds 0.1

6-Feb-14

But the power is an external feed, which can be estimated from experience, but because it is readily available from a website or a direct feed from the supply distribution board, this can be updated in real time as an actual state (on/off) not an estimated probability. The software can then produce three reports.   

Failure Probabilities Failure Modes, and A 3 – point sensitivity presentation of the effect of the critical dependencies

Failure Probabilities This is displayed in the application as a red/green bar chart presentation, overlaid on the respective paragons. (Red means failure, green success). This display is “live” and responds immediately to any change(s) in the linked paragons involved. It will also indicate, quantitatively, the key vulnerabilities of the system(s), by setting the top paragon to fail and observing the failure status of key leaves. The display can then generate a “Modality” report, which can indicate which dependencies have a direct, unattenuated effect on the System performance. In the illustration below, it can be seen that maintenance is a critical Mode 1 dependency.

Failure Mode and Effect Reports

Figure 3 – Failure Modes and Probabilities for IP Connect


6-Feb-14

Sensitivity Report This report is a very helpful display of 3 crucial insights:1. The predicted (or actual?) probability of the system meeting its objective (99.5% availability). 2. The relative impact of the critical leaves on the overall failure, and 3. The extent to which, improving the performance of a key leaf can have on the total system behaviour. This is shown below for our three systems:-

Figure 4 – the Ranking of Importance and Scopes for the key dependencies of IP Connect

The overall probability of meeting the performance targets for this system are shown as 86% with the two most important dependencies as Maintenance and the availability of the hardware, which makes sense. Further the analysis suggests that improving the reliability of maintenance SLA could increase the overall performance of the system to over 90%. On the other hand, the intuitive nervousness of the effect of interruption of Power supply has been shown to be effectively addressed with the provision of the alternative UPS back up. Improving the reliability of the UPS could reduce this anxiety still further. Again it clearly meets the Dutch requirement – “Ideally, Unit managers will communicate the probabilities of their obligations to the units that depend on them”, “For business units within a single company this is feasible because the required transparency is readily achievable. Communication of probabilities between business units in different companies (supply chains) may need more coordination. However, the method still works, although unit managers may need to second guess or assure the probabilities of external expectations themselves. 60 Modelling Basics ds 0.1

6-Feb-14

Note that if all managers in a value chain really cooperate, they could see the effects that their risk treatment decisions have in other areas. Also, risk mitigation can be optimized as risks that are run in a specific unit might well be mitigated by controls implemented in another scope”. But the software will do it for him; implemented as a cloud based portal, it is instantly updated and accessible across the whole organization. Thus, a chain (or better: web) of functionalities can be modelled and corresponding risks computed automatically).

Conclusions This paper has applied a “Dependency Analysis” approach to modelling the interaction and interdependencies of effects on systems in a supply chain, or mix of corporate business units. The example used was a suggestion from a Dutch paper that laid out the required attributes of a model to manage risk within an organization and between organizations. This has been achieved but surpassed in that we can not only do this in real time, but with the ability to import external and global influences, the model is capable of continuously updating the dependencies in real time giving the manager an instantaneous overview of the risk status of his system or system of systems. Furthermore, the out puts are quantitative and enable failure mode diagnostics to highlight critical dependencies, their effects and scope for improvement. As well as real time monitoring the models can be utilized “offline” as “what if” planning or in retrospect as incident analysis tools. Finally, as the implementation is as a cloud based web portal, this status information (top paragons only) (as a first level of interrogation), can be available nationally (CPNI) or trans nationally (ECOSSIAN) for the really challenging application of monitoring critical infrastructure components and interdependencies. This is the first approach that finally makes possible/ practicable the monitoring and management of risks in complex, interacting “Systems of Systems”.

Bibliography Slater, D. et al, S. e. (2010). Managing Risk in Complex Interdependent Systems. London: TSB Fast Track Project BK106A. J. Gordon. (2010). Dependency Modelling - a better way. Intradependency. Cambremnsis. M. Hoeve et al. El Metodo Managing Risks in Value Chains. TNO. Slater, D. (2012, February). Risk Assessment: from the top. The Chemical Engineer , pp. 40-42.


6-Feb-14

17.

More Linked in Models 1. Hospital Admission System Model

•

Consider a simplified dependency model for a hospital admissions system.

•

The model takes into account

 the availability of the buildings and standby sites,  the availability of particular types of staff that are needed to run and support the system, and  some suggested dependencies for the admission system itself. •

Note the dependency of Hospital Admission System OK on Hospital Data Links OK and Cloud Data Service OK.

•

The hospital can do nothing to manage or monitor the success of these dependencies as they are provided by the two external organizations.

•

They are uncontrollables.

•

If the data links fail and/or the cloud service cannot provide data services, the hospital will suffer an impact to its operational continuity.

Hospital Admission System Model

04/03/2013

Obiectivo Rome

38


6-Feb-14

2. Cloud Data Service Provider Model •

Note that within this model, the top-level paragon is called Cloud Data Service OK.

•

This paragon represents the same goal as Cloud Data Service OK in the previous hospital model.

•

Here, they are named the same but in reality they could have different names as different people developed them.

•

However, the point is that they represent the same goal and if we really mean they are the same, we can utilise the same existing entity in all these models and the different models will import that entity with all its dependencies.

•

Notice that Cloud Data Service OK in this model also depends on Comms Links OK to be successful.

•

This can be in yet another linked in model

Cloud Data Service Provider Model

04/03/2013

Obiectivo Rome

40

3. Telecoms Links Provider Model •

Finally, the dependency model for the telecoms links provider.

•

Note that it contains a paragon called Comms Links OK,

•

which represents the same goal as the paragon Comms Links OK in the cloud data service provider’s model.

•

Again, if the paragon name match is for convenience only; it could be named anything

•

But if we mean it is the same we use the same Comms Link OK entity in our list 63


6-Feb-14

The ultimate risk model? Enterprise

Service

Industry

Registry of paragons

Distributed query engine

36 04/03/2013

Source: Gordon, J., Dependency modelling and understanding risks to the infrastructure, 2011/Burnap, P., Hilton, J., Gordon, J. and Slater, D., A goal-oriented approach to determining risk in distributed interdependent systems, 2011 © Intradependency Ltd 2011

Obiectivo Rome


6-Feb-14

18. A “Dependency” model of Fukushima The potential of the tool can be further illustrated by looking at a model of the Fukushima dependencies – How would this have helped the Japanese Operators (TEMPCO) better understand their dependencies on the adequacy of the safeguards in place?

The probability of tsunami has been set to 1 in a million. Predictably the probability of disaster is small as witnessed by the vast expanses of green, and no visible red:

But probe a bit deeper and the flaws show up. Here's what you get, given that a disaster occurred:


6-Feb-14

This is an a-postiori result, showing the most likely causes of a disaster, given that it has happened. Just how independent and individually resilient were their lines of defence? And finally, the coup-de-grace the 3-point-probability chart:

If ever there was a diagram that justified this methodology as a measure of risk, it's this one. We can now build a better model using iDEPEND and get a more detailed report on the sensitivities But now the a priori probabilities show the vulnerability of the design and the tsunami is reflected in at least 10 different leaves. This results in an expectation that Fukushima safe is less than 50%?


6-Feb-14

Figure 5 - Fukushima Reactors Safe – iDEPEND illustrative example


6-Feb-14