uncertainties are dealt with on the form: âdette er .... given that the reconnaissance did not give new and .... http://www.phs.ki.se/csp/pdf/indicator09/indicators_.
Reliability, Risk and Safety – Ale, Papazoglou & Zio (eds) © 2010 Taylor & Francis Group, London, ISBN 978-0-415-60427-7
Risk based thinking is more important than accurate risk modelling: A discussion of risk assessment tools as basis for two categories of decisions O. Njå
University of Stavanger, Stavanger, Norway
E.L. Rake
South Rogaland Fire Department, Rogaland, Norway
R. Aanestad
Municipality of Klepp
G.S. Braut
Norwegian Board of Health Supervision, Norway
ABSTRACT: The British sociologist Anthony Giddens has introduced the concept of fateful moments. Fateful moments are situations when individuals and societies are called upon to make decisions that are important for their ambitions and future conditions. Recognising fateful moments in the real world is not an easy exercise. Unawareness of system complexity, structural and organisational barriers, lack of time and lack of competence make this a difficult task. Tools for risk and vulnerability analysis are generally regarded as powerful means to support critical decisions in order to ensure optimal decisions. The risk analytic approach could be expected to clarify possible fateful moments. This paper discusses the use of risk based thinking in two different settings for decision making, none of which are typical for the current use of risk analyses. Firstly, we deal with temporary activities under the responsibility of a local government (municipality), for example arranging school trips, medical treatment in nursing homes, or organising a sports event. These are all activities that require planning taking adverse effects into considerations. Secondly, we look into crisis management on the scene of an accident. In crises the time frame can be extremely short and the choices of action can turn out to be fatal either for victims or for the rescuers. Based on an analysis of our cases we claim that the probability theoretical concepts are the major obstacles for individuals to employ proven risk assessment tools in such situations. The decision makers feel uncomfortable and are frightened to provide erroneous or bad assignments, disregarding the core aim of the assessment, namely the quality of the decisions to be made. The implications are that risk based decision processes are undermined and critical reflections omitted as part of the decision making. Furthermore, this mental inability lays the ground for “risk experts” to provide the risk assessments and safety considerations become alienated from involved personnel. The paper concludes with a discussion of the transferability of our observations to other decision areas, in which risk based decisions have strong traditions. 1 Introduction Safety could be considered as a system’s ability to prevent damages and losses. The concept of risk is employed to communicate the degree of safety, and risk analyses tools are the measurement methods. However, concluding on how safe a normal activity or system is, requires indirect measurements, because we do not know how many events, fatalities or injured people we have prevented by designing our system, whether it is a workplace,
a residential building or a local society. The same challenge is connected with successful decision making in critical situations. Could we then say that we are able to “measure” the safety level for all kind of systems? We claim that the answer is no, and that the difficulties related to this type of measuring are fundamental and exist on all levels. First of all, the concept of risk is problematic. Risk cannot be directly observed in the real world, such as a person’s weight or height, which is easily measurable. Risk is related to (undesired) events (A),
2169
consequences (C), and related uncertainties (U). Risk is commonly communicated as combinations of the concepts probability (or frequency) and outcome (or consequences). The concept of risk is thoroughly debated by many researchers (Aven, 2010; Kaplan, 1997; Kaplan & Garrick, 1981; Rosa, 1998; Watson, 1994). In general the interpretations of risk could be categorized into the classic positivistic view versus a constructivist’s standpoint. The first comprises the classical relative frequency probability theory and the second comprises the Bayesian predictive approaches. This academic discussion is foundational and unresolvable. One core issue is how to understand and handle uncertainties. Employing risk analysis tools, which can be done on basis of both those fundamentally different risk concepts, implies use of models, assumptions and data more or less accurate and relevant, which further complicates the issue of communicating safety levels. In real life historical data are often used to predict and communicate the level of safety. In Norway the number of deaths from accidents is approximately 1.800 per year, which equals to approximately 4 persons per 10.000 inhabitants. The distribution of types of accidents causing death shows an interesting picture. For the years 2004–2008 the official statistics show this distribution related to the place of the accident. Accidents in the home or in residential areas count for 26%, road traffic accidents 13%, other outdoor arenas 8%, hospitals and nursing homes 3%. Though, it is worth to note that in almost 42% the place of the accident is recorded as unknown. Work place accidents typically stand for between 3%–4% when all types of work (land based and maritime) are included. The majority of fatal accidents occur in homes, on leisure times and workplaces often well outside areas and systems where traditional risk analysis processes come into play. Use of risk based regulations has become general in all sectors in Norway, from area planning to road traffic and social and health services. This presupposes functional requirements instead of prescriptions of detailed solutions. Thus, it is prerequisite that risk and vulnerability analyses play an important part in planning and decision processes, whether the condition is developing new systems (areas, infrastructures, industrial sites etc) or it is the operation of the daily work. In 1994 the Norwegian authorities introduced a guideline on risk and vulnerability analysis for use in the local governments in order to encompass risk based thinking in the public services (DSB, 1994). However, the change into considerations of probabilities and consequences has not been an easy task for employees within the local governments. There are multiple reasons for this, both lack of
incitements to employ the tools, the tools require ability to abstract situations, difficulties to observe the benefit by using such tools and probability theory not easy to comprehend. This has in many municipalities led to misuse, superficially analyses, and a general loss of confidence in risk based decisions. In this paper we compare two different planning and decision contexts to which risk and vulnerability assessments have not been prominent up to now. Our issue is to discuss which parts of the risk management process that promotes confidence and the intended use and which parts that do not. The first case is the local government approach to risks in the daily life and ongoing services provided by the local administrations. The other example is related to response challenges on scene a fire scenario. We give some examples on experiences from practical risk assessments and use these practice based experiences to suggest some recommendations to how and when risk and vulnerability assessments could be suitable as decision support. 2 risk assessment on local level in municipalities The municipality of Klepp in south-western part of Norway is a medium size, rural community with about 15000 inhabitants. The population is mostly concentrated around five areas but also spread around the rest of the municipality. Most of the nonurbanised land is used for agricultural purposes. The municipality is closely connected to and share infrastructure with the urban area around Stavanger. 2.1 Experienced need for a practical tool when dealing with risks The local authorities decided in 1999 to initiate a risk-based, accident prevention programme. This effort led forward to appointment as a safe community in 2002, according to the World Health Organization’s concept (WHO, 1986). Safe Communities are certified local communities that meet a set of 12 criteria (later changed to six indicators) set out by the WHO Collaborating Centre (WHO CC) on Community Safety Promotion at Karolinska Institutet in Stockholm (WHO, 2009). Within the frames of this concept the local authorities have been focusing particularly on the practical use of risk and vulnerability analyses in the accident prevention work, believing that the proactive approach in such methods is suitable for gaining engagement among common people, not only experts. Even before 1999 the local authorities had gained experiences from using risk and vulnerability
2170
analysis for planning emergency preparedness according to the requirements and methods presented by the governmental authorities (DSB, 1994). But this approach was felt cumbersome, too extensive and not very suitable for local community use; characterised as an exercise performed by few persons in the administration and of little use for daily operative efforts. A need for a simplified tool emerged. Initiated by this local experienced need they developed a tool called “DagROS”— DailyRiskAndVulnerability-analysis. This tool is characterised by being self-instructive, easy to use and suitable for use by common people, without any previous experiences or knowledge in risk and vulnerability analysis. DagROS is now being used as a normal part of planning of service provision and for evaluating work place environment among municipal employees as well as among voluntary organisations. Examples of processes where this tool has been used are: • By the adult staff when planning a beach outing for day care children • By the management before reducing staff at night time at a ward in a nursing home • By the local sport organisation when setting up a major arrangement • By municipal technical authorities before temporarily closing down a foot- and bike path used by school children • By school management and scouts’ leaders when the scouting group shall be allowed to use the carpenters’ hall at the school 2.2 Deficiencies with a traditional tool As mentioned above the local authorities were well acquainted with a traditional tool for risk assessment, based on a preliminary hazard analysis (DSB, 1994). But at local level they felt scarce benefit of this approach. This approach was felt bureaucratic, and perceived as a requirement imposed from above. The local authorities performed it just because they had to, not because of the possible local benefits. It was difficult to engage people in participation, and they did not feel an ownership, neither to the process nor to the results. Further, neither the process nor the results were much used in other municipal planning processes than emergency preparedness. It is particularly of interest here to note that the assessment of probabilities was experienced as very difficult. And the level of concern related to risks in the local community was not raised. Nilsen (2007) has studied perceptions related to the use of risk assessments tools in the municipalities of Klepp and Time in Norway.
2.3 Advantages with the new tool The new tool is used by the persons who are going to take part in or be responsible for the analysed activity or system. The involved persons are defining the unwanted incidents as well as suggesting the possible solutions. This implies that the analysis is closely connected not only to experiences with that particular activity or system, but also with the responsibility for the subsequent results of the specific decisions. The DagROS tool consists of six intuitive steps with simple questions expressed in daily wording leading the involved through the process: 1. Which activity is to be assessed? 2. What events are we worried can happen? Nothing to be excluded at this stage. 3. What must and should we do something about? Mark the incidents in item 2 that should be followed up. 4. What should be done to reduce likelihood of the events prioritized? List measures and responsible person. 5. What should be done to reduce the damage if the event happens? List measures and responsible persons. 6. After the activity is carried through—what are the lessons learned? After the practitioners themselves have been introduced to the DAGROS tool they are supposed to act as analysts as well as decision makers, thus in a way challenging the traditional view on the different and separate roles of the analysts and the decision makers. DagROS provides direct connection of uncertainty assessments and acceptability (step 3). The risk mitigation principle applied is that measures related to probability reduction (step 4) are discussed before consequence reduction (step5). The participants immediately perceive the analytic process as useful. They can proceed from the analytic process directly into the decision making and performing phase. The analytic process is closely connected to practical challenges and thus is experienced as highly relevant. Instead of dealing with theoretical probabilities the participants are sorting the important issues from the unimportant factors. It enables reflection out of practical, important issues. In fact probabilities as such are not identified, rather the uncertainties are dealt with on the form: “Dette er vi redde for kan skje” (We are worried that these events can happen). Also Rundmo & Iversen (2004) found that “worry and other emotional reactions” were good predictors on behaviour. 2.4 Restrictions with the new tool The new tool may be suspected to have important limitations especially when facing more complex
2171
challenges. It is inaccurate and gives a very rough risk picture. Therefore the use is restricted to analysis of transparent, limited and preferably well known processes. When extended outside the well known domain one may suspect increasing possibilities for wrong decisions. As it is closely connected to the operational situation, it may give rise to lack of foundation of the decisions among upper managerial staff. It is also a low level procedure with low requirements for documentation. Therefore the subsequent decisions may appear as less traceable than desired. 2.5 Signs of good processes For validating the analysis one may search for signs of a good process. These are identifying that the right persons have been involved, that the process has been felt as positive with extensive discussions on risks, that the solutions emerge as a result of co-operation, and that the responsibilities for following up are clarified. 3 Risk assessment in crises Crises may be considered as “occasions for decisions”. Crises can take many forms, but for the incident combating resources crises are related to social, human, material and environmental values at immediate risk. Crises are described as borderless threats, crises are to be viewed as ongoing processes (compressed decision–making), crises are political and filled with conflicts, and crises are subjected to media roaming free conditions (Rosenthal, Boin & Comfort, 2001; Rosenthal, Charles & Hart 1989). The decision-making situation is characterized by the necessity of critical choices, or as Rosenthal (1986) puts it: “a serious threat to basic structures or the fundamental values and norms of a social system, which—under time pressure and highly uncertain circumstances—necessitates making critical decisions”. It is widely acknowledged that the incident commander’s decision-making on-scene is important for the outcome. It is not our intentions to discuss this hypothesis here. We will discuss how the incident commander may make use for a risk analytic approach in his or her work on scene. When a crisis is emerging, the possible outcomes often remain unclear, and the set of possible outcomes are unknown. The incident commanders, neither in the police nor the fire brigades or the ambulance services have substantial experience from large/major incidents. Such incidents and situations will encompass novelty and different kinds of uncertainties. In minor and daily actions, as use of
CPR (cardiopulmonary resuscitation) in a traffic a ccident, the decision maker recognizes the situation as typical (by use of serial and mental evolution) and select an action he knows will cope with the urgent situation (Klein, 1993). He will initially also follow the predetermined procedures and monitor the situation to reveal discrepancies from the expected expansion of the response effort due to his experiences from similar incidents. To cope with situations which are unexpected, unscheduled, unprecedented and almost unmanageable (Rosenthal, Hart & Charles, 1989) the incident commander must use alternative strategies to make sufficient decisions. By using risk analysis as a decision support during decisions making on—scene, the incident commander may decide what kind of information is needed and possible to gain in the relevant time frames, and then decide which strategies and measures to be applied in real-time (Rake, 2003). There are few empirical evaluations of the actual influences of risk assessment on the decision making of human service professional on-scene. In addition, the complex situation on-scene, with multiple actors, critical values at stake, and need for prompt action make the use of risk concept demanding. An investigation of 22 incidents in real time showed that the use of a risk analysis or similar approaches were uncommon (Rake & Njå, 2008). Effective decision making is a necessity of excellent incident performance. Cannon-Bowers et al. (Cannon-Bowers & Bell, 1997; Cannon-Bowers, Salas & Pruitt, 1996) identified six attributes of effective decision making that are important to the incident commander; Flexible (alternative courses as far as possible), quick, resilient, adaptive (depends on other decisions), risk taking (impossible to avoid all threats and hazards), and accurate. Successful incident commanders reflect, use mental simulations and mental models (defining the situation) to systematic reduce the inherent uncertainty and make the situation “familiar” (Rake & Njå, 2008). The decisions are not made in a vacuum but rather in close cooperation with the subordinates and other actors on scene. A proactive management strategy is imperative to succeed in major accidents. A risk approach, Risk Informed Decision Making (RIDM), has been tested as decision support at demanding and uncertain incidents. The purpose with RIDM is to establish a way of thinking on scene that is general and independent of the actual situation (Njå & Rake, 2003; Rake, 2004). RIDM enforces the decision maker to place the focus on the quantities regarded important, and RIDM require that the decision maker (on-scene management team) evaluates uncertainties systematically. Through active use of the RIDM-model,
2172
we believe that uncertainties will become reduced, decisions made will improve the on-scene performance, and that fewer abnormal accident situations become unsatisfactorily managed. The key principles of our recommended risk approach are the following: • The process is ongoing throughout the scenario, from being alerted until the situation is normalised. • Based on available information, identify the observable quantities involved, including the critical values at stake (lives, environment, and assets). Clarify decision alternatives. • Predict the outcome of the quantities, in a specified time horizon. What can happen? • Uncertainty related to the observable quantities and the predictions is expressed by probabilities (for example percentages) • Decision is made amongst the alternatives: Action is necessary, Can wait, Ignore/Keep under observation. • Feed-back is ensured, both with respect to efficiency of the decisions made and related to general information gathering efforts. The efficiency could be measured directly on changes to the observable quantities. Let us illustrate the model by the following incident. The south-western part of Norway (Rogaland) consists of a variety of mountains, valleys, moors, woods, fjords, farmland etc. A fire alarm is given late in the evening—a bushfire in an area with cliffs and rocks far from the nearest road. What now? Certainly, standard procedures was followed, engines equipped and firemen prepared for action and transportation to the scene. What information was available for the commander? The information from the alarm central was important, maps of the area as well, weather forecasts and status, and people acquainted with the locality. The first step—which quantities to identify? Critical values were: Human beings in the vicinity (including responders), protection worthy nature, buildings and other assets in the area. Observables: The topographic, temperature, wind (speed and direction), fire zone and materials on fire, fire speed and intensity, accessibility to the fire scene, extinguishing equipment, human resources. Decision alternatives: The commander analysed the information and also used his experience and knowledge of the area, and set up three options, (1) command people out to combat the fire, (2) wait and gather more information in order to optimise a strategy for combating, or (3) ignore the fire—just maintaining surveillance until dawn. When the commander arrived at the meeting point, he knew that the nearest building was 3 km from the fire front (uncertain extension), which
was also the distance for the fire engines (nearest road). The terrain was near an old forest, important to protect and the forest was located on a lower level than the fire scene. It was dark and the terrain was difficult to access (few foot paths, many cliffs, depressions and rocks). The temperature was approximately 0° Celsius, and the wind direction away from the forest fire and nearest building. Outcomes: Responders are vulnerable in the darkness with difficult accessibility (falling, trapped in fire, loss of communication, etc). Other people could be in the area, but there existed no such information. The fire could spread to the wood and the buildings nearby. Uncertainties: The commander assigned a high probability (∼70%) that responders would become injured if fire fighting was commenced immediately. He relied on the weather forecasts indicating stable weather conditions through the night, and he assigned a small probability (less than 5%) to that the old forest and building would be caught on fire during the night. The location and extension of the fire were unclear. Decision: The commander decided to carry out a helicopter reconnaissance, and wait with fire extinguishing until dawn the next day, given that the reconnaissance did not give new and vital information. The decision process involved experts, such as meteorologist, helicopter pilot and a fire fighter well acquainted with the area. It also involved the alarm centre to obtain more information for the commander on his way to the fire. The subordinates, the police and the ambulance service were also included in the process. The reconnaissance strengthened the decision. At dawn the fire brigade entered the area and successfully extinguished the fire. None of the responders were injured. The old forest and the buildings were far away from the fire front. 4 Discussion Above we have presented some practical experiences with two different “low-tech”-approaches to analysing risk in connection with decision making. The methods are different as are the decision contexts. Though, both approaches are meant to be used in situations where competent decision makers need a broad basis for their judgements. The subsequent decisions may affect the lives and welfare for a lot of other persons than the decision makers. Thus these instruments are to be used when approaching what Giddens calls fateful moments (Giddens, 1991). In the studied situations the roles of the analyst and the decision maker are combined in the same person(s). From a traditional risk
2173
a nalytic viewpoint this is a situation that often is recommended to be avoided. But in the types of practical environments studied here, there is a close connection both in time and in space between the moment for analysing and the time for decision. There is no room for thorough discussions between a decision maker and an analyst on methods, material, criteria for evaluation etc, neither before, during or after the analysis is performed and before a decision is made. Combining these roles may also make the decision maker aware of the responsibility to secure a broader basis for the decisions than when the roles are separated. A decision maker cannot get away with an excuse saying that the analysts evaluated the risk as low. It is the decision makers themselves that need a valid risk picture for their judgements and decisions. The experiences may signify that the presented tools help the decision makers to open up for an evidence based reflection on the possible risks related to the activities or systems involved. It is not the analysis in itself that is of interest in these situations. The interest is focused at getting reasonable and sound decisions. Both approaches are extremely operational, which enable the involved parties to evaluate the quality of the decisions made and alternate more or less ad hoc the actions and decisions to come. The tools might then be seen as valuable learning means, providing feed-back that can be used in assessments of change, assessments of confirmation and assessments of comprehension (Braut & Njå, 2009). In specific, particular situations it is not the possible distributions of the probabilities for certain outcomes that are of main interest. But the decision maker is searching for possible outcomes of different alternative decisions, and attributing his or her belief on the occurrence of each of them in the specific situation. Therefore one may expect that a very exact risk modelling will not necessarily increase the validity and reliability of the analysis. Of greater importance to the appropriateness of the final decision may be the validity of the risk picture of that specific situation employed by the decision maker(s) there and then. This type of a risk picture diverges a bit from those often used in traditional risk analysis. It does not claim to be generalised. It is just a presentation of the belief of possible outcomes on which an actual decision in made. It is representing the way of thinking of the decision makers. Thus a more relevant name on this phenomenon could be a risk image; the image of possible outcomes that the decision is based upon. Based upon this study it is not possible to more than suggest the possible strong and weak sides with the presented approaches. We suspect that these approaches fit best when dealing with decisions in well known activities and systems that
may give rise to a multitude of outcomes. These are situations where a traditional risk modelling procedure quickly becomes very complicated with a lot of challenges connected to prediction of probabilities. Instead of abandoning a risk based approach for decision making in such situations, we claim that more simplified approaches as described here may give rise to better, risk-informed decisions. Having easy to perform, still somewhat structured approaches to lean upon, can give some sort of standardisation of the procedure guaranteeing a minimum of quality on the process as well as on the results. It may also stimulate practitioners to continuously reflect upon different risks connected with their daily work, thus making risk analysis a working tool more than an academic exercise for the trained specialist. Normally risk management approaches on scenes of accidents are limited to evaluations of the response units’ safety. One of the reasons for the lack of use of a risk analytic approach may be that more than 90% of the incidents are handled by standard procedures and common problem solving strategies and tactics. Therefore it seldom is a prominent need for a deeper analysis using mental resources for simulations etc for getting an acceptable result. It neither seems to be expected by the relevant actors, and the incident commanders perceive it as unnecessary. By reducing uncertainty and highlighting alternative decisions options the performance will improve and risk reduced (Lipshitz & Strauss, 1997; Rake, 2003). 5 Conclusions We suggest that training practitioners in constructing risk images before and during their ordinary decision making may lead to better decisions as they open up for a broader perspective on possible outcomes and force the decision makers to become aware of their believes in the different outcomes. Instead of looking upon this as constructing a risk picture in a more traditional way by ordinary risk analytic approaches, we claim that this approach make use of the intellectual capacity of the professionals to build images of possible outcomes based upon their own experiences, their professional knowledge and discussions with relevant other people. To separate this from the risk picture produced by traditional risk analytic approaches, we here prefer to call this a risk image. This combined analyst and decision maker role certainly will not produce as traceable and documented processes as strived for in ordinary risk analysis, but it may be expected that it enhances
2174
the feeling of responsibility for gaining a good outcome and it couple the analytic activity direct to the practical work. References Aven, T. (2010). Misconceptions of risk. Chichester: Wiley. Braut, G.S. & Njå, O. (2009). Learning from accidents (incidents). Theoretical perspectives on investigation reports as educational tools. In R. Briš, C. Guedes Soares & S. Martorell (Eds.), Reliability, Risk and Safety. Theory and Applications (pp. 9–16). London: Taylor & Francis Group. Cannon-Bowers, J.A. & Bell, H.H. (1997). Training Decision Makers for Complex Environments: Implications of the Naturalistic Decision Making Perspective. In C.E. Zsambok & G. Klein (Eds.), Naturalistic Decision Making (pp. 99–110). Mahwah, New Jersey: Lawrence Erlbaum Ass. Cannon-Bowers, J.A., Salas, E., & Pruitt, J.S. (1996). Establishing the boundaries of a paradigm for decision making research. Human Factors, 38(2): 193–205. DSB (1994). Veileder for kommunale risiko- og sårbarhetsanalyser [Guideline for risk and vulnerability analysis for use in local government]. Oslo, Norway: Direktoratet for sivilt beredskap [Directorate for civil protection]. Giddens, A. (1991). Modernity and self-identity: self and society in the late modern age. Stanford, Calif.: Stanford University Press. Kaplan, S. (1997). The Words of Risk Analysis. Risk Analysis, 17(4): 407–417. Kaplan, S. & Garrick, B.J. (1981). On The Quantitative Definition of Risk. Risk Analysis, 1(1): 11–27. Klein, G. (1993). A Recognition—Primed Decision (RPD) Model of Rapid Decision Making. In G. Klein, J. Orasanu, R. Calderwood & C.E. Zsambok (Eds.), Decision Making in Action: Models and Methods (pp. 138–147). Norwood, N.J.: Ablex Publishing Corporation. Lipshitz, R. & Strauss, O. (1997). Coping with uncertainty: A naturalistic decision-making analysis. Organizational Behavior and Human Decision Processes, 69(2): 149–163. Nilsen, A.S. (2007). Municipal Risk Management. Implications of the use of different risk tools. University of Stavanger, Stavanger. Njå, O. & Rake, E.L. (2003). Risk Based Decision Making on Accident Scenes. In Emergency Management in
a Changing World. Paper presented at the The International Emergency Management Society 10th Annual Conference. Rake, E.L. (2003). Emergency management and decision making on accident scenes: taxonomy, models and future research. International Journal of Emergency Management, 1(4): 397–409. Rake, E.L. (2004). A Risk-Informed Approach to Decision Making in Rescue Operations. Paper presented at the International Conference on Probabilistic Safety Assessment and Management (PSAM7/ESREL04), Berlin. Rake, E.L. & Njå, O. (2008). Perceptions and performances of experienced incident commanders. Journal of Risk Research, 12(5): 665–685. Rosa, E.A. (1998). Metatheorethical foundations for post-normal risk. Journal of Risk Research, 1(1): 15–44. Rosenthal, U. (1986). Crisis Decision Making in The Netherlands. Netherlands’ Journal of Sociology, 22: 103–129. Rosenthal, U., Boin, R.A. & Comfort, L.K. (Eds.). (2001). Managing crises: Threats, dilemmas, opportunities. Springfield, Ill.: Charles C. Thomas. Rosenthal, U., Charles, M.T. & Hart, P.t. (Eds.). (1989). Coping with Crises: The Management of Disasters, Riots, and Terrorism. Springfield, Ill., USA: C.C. Thomas. Rosenthal, U., Hart, P.t. & Charles, M.T. (1989). The World of Crises and Crisis Management. In U. Rosenthal, M.T. Charles & P.t. Hart (Eds.), Coping with Crises: The Management of Disasters, Riots, and Terrorism (pp. 3–33). Springfield, Ill., USA: C.C. Thomas. Rundmo, T. & Iversen, H. (2004). Risk perception and driving behaviour among adolescents in two Norwegian counties before and after a traffic safety campaign Safety Science, 42(1): 1–21. Watson, S.R. (1994). The meaning of probability in probabilistic safety analysis. Reliability Engineering and System Safety, 45: 261–269. WHO (1986). Ottawa Charter for Health Promotion. First International Conference on Health Promotion Ottawa, 21 November 1986 - WHO/HPR/HEP/95.1 Retrieved March 31, 2010, from http://www.who.int/ hpr/NPH/docs/ottawa_charter_hp.pdf WHO (2009). Indicators for International Safe Communities. WHO Collaboration Centre on Community Safety Promotion Retrieved March 31, 2010, from http://www.phs.ki.se/csp/pdf/indicator09/indicators_ international_sc.pdf
2175
