Comparative risk assessment (CRA) has been used as an environmental decision making tool at a range of regulatory levels in the past two decades.
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
TOWARDS USING COMPARATIVE RISK ASSESSMENT TO MANAGE CONTAMINATED SEDIMENTS T. BRIDGES, G. KIKER U.S. Army Engineer Research and Development Center Environmental Laboratory 3909 Halls Ferry Rd, Vicksburg, MS 39180, USA J. CURA Menzie-Cura & Associates, Inc. 8 Winchester Place Winchester, MA 01890, USA D. APUL Department of Civil Engineering The University of Toledo 2801W Bancroft St., Toledo, OH 43606, USA I. LINKOV Cambridge Environmental Inc. 58 Charles Street, Cambridge, MA 02141, USA Abstract Comparative risk assessment (CRA) has been used as an environmental decision making tool at a range of regulatory levels in the past two decades. Contaminated and uncontaminated sediments are currently managed using a range of approaches and technologies; however, a method for conducting a comprehensive, multidimensional assessment of the risks, costs and benefits associated with each option has yet to be developed. The development and application of CRA to sediment management problems will provide for a more comprehensive characterization and analysis of the risks posed by potential management alternatives. The need for a formal CRA framework and the potential benefits and key elements of such a framework are discussed. 1. CRA – Background and Applications Comparative risk analysis has been most commonly applied within the realm of policy analysis, in that it supports tradeoff decisions with broad implications. Andrews et al. (2004) distinguish CRA uses at macro and micro scales.
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
At the macro scale, programmatic CRA has helped to characterize environmental priorities on regional and national levels by comparing the multi-dimensional risks associated with policy alternatives. U.S. government agencies at various levels have logged significant experience with policy-oriented, macro-level CRA. Gutenson (1997) suggests that the starting point was a series of Integrated Environmental Management Projects performed during the 1980s. These took place in Santa Clara, CA, Philadelphia, PA, Baltimore, MD, Denver, CO, and the Kanwha Valley, WV. Their common goal was to improve local environmental decision-making by supporting it with quantitative risk analysis. However, it was the national-level “Unfinished Business” report (US EPA, 1987) that created sustained momentum for the use of CRA. During this time period the USEPA offered grants to encourage U.S. regions, states, and localities to undertake similar projects. From 1988 to 1998, some twenty-four states and more than a dozen localities undertook comparative risk projects. See Andrews et al., 2004, Andrews, 2002, and Jones, 1997 for detailed descriptions of the nature of these projects. International CRA applications are reviewed in Tal and Linkov, 2004 and in Linkov and Ramadan, 2004. At smaller scales, so-called micro studies in CRA have been used to compare interrelated risks involved in a specific policy choice (e.g. drinking water safety: chemical versus microbial disease risks). In these micro applications, the CRAs often had more focused objectives within the general goal of evaluating and comparing possible alternatives and their risks in solving problems. The number and varied nature of CRA applications suggest that there is a measurable degree of acceptance for CRA as a decision-making tool even though no clear or specific guidance exists on how to conduct a CRA or use the information the analysis produces. This paper expands the discussion on the micro applications of CRA by identifying the benefits and key elements of an effective CRA framework that can be used for managing contaminated sediments. 2. CRA and Management of Sediments Several different management alternatives and technologies exist for contaminated and uncontaminated sediments. While the number of management options is constrained, the relevant exposure pathways, receptors at potential risk, and implementation costs and benefits vary broadly. Example sediment management alternatives include unrestricted open-water placement, confined aquatic disposal (i.e., capping), use in constructed wetlands projects, disposal in lined landfills, chemical or physical stabilization technologies, cement manufacture, lightweight aggregate production, topsoil production, and in situ treatment (Seager and Gardner; 2005). Selection of the “best” option is complicated by the varied nature of the risks, costs and benefits associated with the options. A “mind-opening” approach for analyzing the risks, costs and benefits for each potential management alternative will provide for more informed and credible decision making.
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
Cura et al. (2004) reviewed CRA in an attempt to rectify or at least demonstrate the differences among the varied definitions offered in the literature, explain the use of CRA at various regulatory levels, and search for an application of CRA at operational, as opposed to policy, levels. They reviewed the status of CRA within the context of environmental decision-making, evaluated its potential application as a decisionmaking framework for selecting alternative technologies for managing dredged material and made recommendations for implementing such a framework. Cura et al. (2004) emphasize in their review that CRA, however conducted, is an inherently subjective, value-laden process. They found that while there was some objection to this lack of total scientific objectivity (“hard version” of CRA), that the “hard versions” provided little help in suggesting a method that surmounted the psychology of choice in decisionmaking schemes. The application of CRA in the decision making process at dredged material management facilities will involve the use of value-based professional judgments. The literature suggested that the best way to incorporate this subjectivity and still maintain a defensible comparative framework was to develop a method that was logically consistent and addressed this issue of uncertainty by comparing risks on the basis of more than one set of criteria, more than one set of categories, and more than one set of experts. 3. CRA Outcomes and Framework There is no single, precise definition for CRA. In the context of sediment management problems, it can be viewed as a part of a decision making process that relies on estimated relative risks or impacts associated with each management alternative under consideration. These risks can be expressed in terms of their relationship to the natural environment, human health, the legal or regulatory context, and socioeconomics. Estimation and interpretation (possibly weighing or ranking) of relative risks is also an integral part of the CRA process. From this broad definition, it is clear that application of CRA to sediment management problems inherently avoids an overly narrow and isolated consideration of alternatives as CRA begins with the notion that various options are available and need to be evaluated with respect to the differences that exist among them. An effective CRA of sediment management alternatives can provide a range of benefits to the decision-making process. A CRA can be used to promote a structured, fair, and open exchange of ideas among scientists, citizens, and government officials on a broad range of issues related to characterizing the risks, costs and benefits of management options. The comprehensive and comparative nature of a CRA will lead to assumptions being more transparent and reduce the hidden influence of undeclared biases. For these reasons CRA should result in more consistent and reproducible decision making. While the benefits of CRA as a decision tool are well recognized, a formal, accepted procedure for conducting a CRA does not exist. We propose that CRA can provide the necessary data inputs concerning risks, costs and benefits that pertain to identifying the “best” sediment management alternative; however, a guidance framework for applying
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
CRA to sediment problems must be developed in order to realize the benefits of the approach. A process that provides for incorporating specific stakeholder concerns and perspectives about the attributes of the problem at hand will make for a transparent and comprehensive analysis of risks, costs and benefits. To be effective, a CRA framework must guide the analysis and comparison of disparate endpoints (e.g., human health hazard indices and cancer risks, ecological toxicity quotients, management costs, habitat loss/creation etc.) that are expressed in different units and scales. To address the needs of decision makers a CRA should identify the management alternative, or set of alternatives, that maximize risk reduction and minimize the risk of not achieving the risk reduction objective per unit cost (e.g., dollars per m3 of sediment managed or remediated). A comprehensive CRA framework should lead to providing the answers to such questions as: • • • • • • • • • • • •
How will the analysis address the decision drivers How will the sufficiency of the analysis be determined? What are the bases of comparison among the alternatives? What are the analytical components/causal pathways in the analysis? Which alternative maximizes the risk reduction per unit cost? Where is the best place to put this material? Which alternative minimizes risk? Which alternative minimizes cost? Which alternative minimizes uncertainty? What are the sources of uncertainty and variability in the assessment? Which alternative reflects stakeholder preferences? Does the analysis address the decision drivers?
Implementing a CRA framework to achieve specific programmatic or regulatory objectives raises several additional questions including: • • • •
What, if any, programmatic or regulatory constraints exist for using CRA to inform decision making? How will the CRA incorporate stakeholder concerns and preferences? How will uncertainties in the analysis be accounted for in the decision making process? How will the results of the analysis be communicated to stakeholders and decision-makers ?
4. CRA Methods Cura et al. (2004) identified and compared nine important analytical elements in nine different CRAs and found that most or all of the studies categorized elements to be compared, developed criteria as the basis of comparison, and scored the categories. Many of the studies did not incorporate public advice and only a few convened a
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
Delphi panel, employed iteration, weighed the criteria, and defined the sources of uncertainty and quantified it. Their detailed analysis of these various assessments resulted in eight recommendations (Table 1) regarding the development and application of a CRA method for the management of dredged material . Expanding on the work of Cura et al. (2004), the key elements of an ideal framework are presented in Figure 1. The outer circle represents the logical implementation of activities starting with determining the alternatives and progressing towards making a decision. A circular presentation of activities was selected to convey the iterative nature of an ideal framework. The steps in the process will be revisited and refinements made as information and data are collected and judgments are made regarding the sufficiency of the information for decision making. Other activities and essential elements of an effective framework are included in the center of the figure . An explicit understanding of the linkage among these elements of the process has yet to be developed by risk practitioners; however, these particular activities cut across many of the activities that are central to assessing and comparing risks . Their inclusion is intended to provide some direction for future development of CRA as a practical and operational tool. 4.1 A PROPOSED LOGICAL PROGRESSION OF ACTIVITIES IN CRA Step A. Determine Alternatives: The purpose of a CRA is to compare the risks, costs and benefits associated with more than one alternative management option. Comparing a number of alternatives requires that a list of alternatives be prepared prior to beginning the CRA. Depending on the context for the CRA the amount effort and potential contention associated with selecting the alternatives to be compared could vary considerably among sites. A list of alternatives could be developed through a formal regulatory process (e.g., a feasibility study) or through close coordination with the stakeholder community at the site. Ensuring that the product of the CRA is responsive to the needs of the decision-making process requires that the list of alternatives must be sufficiently complete to represent the full suite of feasible options. The list of alternatives will form the basis for the Problem Formulation of the CRA, including the development of conceptual models for each of the alternatives. Step B. Identify Categories of Comparison We propose five broad categories of risks for evaluation at a sediment site: human health, ecological, socioeconomic, legal/regulatory, and engineering reliability. The extent to which risks within these broad categories would be evaluated for a specific site will depend on the scope of decision making and objectives defined by the regulatory context Widely accessible guidance for assessing risks to human health and the environment are available for application to the problem of contaminated sediment and for relevant subcategories for these risks. Example subcategories that may be identified for purposes of making comparisons among alternatives include risks from various exposure pathways, risks from specific contaminants, or cancer and non-cancer risks. Framing the risk questions
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
Table 1 Recommendations for development and application of a CRA framework (adapted from Cura et al., 2004) Recommendation
Rationale
Implementation
The approach for dredged material management should reflect and use realistically estimated risks Develop and specify the CRA model and its specific elements in advance of the analysis
Use realistic estimates as much as possible because the management decision that uses the comparison results in tangible actions and effect. There is no general model for CRA especially at the operational level. Therefore the USACE will have to develop a CRA model and its elements to assure that there is an even application of CRA.
Identify and convene a Delphi panel that represents the goals and interests of the USACE to define the categories of risks for ranking and the attributes against which to judge them
The literature recognizes the importance of such a panel in a system that is so value laden.
The tiered approach for conducting environmental evaluations of dredged material (USEPA/USACE 1992) results in a robust site or technology specific database for those sites that reach Tier IV in the process. Recommend that the USACE incorporate Lash’s (1996) suggestion that a CRA should involve at least five areas: defining what is meant by risk, selecting endpoints to consider; categorizing the risks for comparison; selecting a time frame for evaluating adverse effects; and gauging the seriousness of the consequences. The panel should represent the diverse nation-wide interests of the USAE. This will avoid the potential problem that Lash (1996) recognizes, narrow questions that focus on narrow standards (technology, cancer risk, cost) in the decision making process can be so simple as to produce that answers that are wrong. Morgan et al. (2000) recognize that categorization of risks will define the course of the analysis, and they provide explicit guidance for their development including; grouping risks into a manageable number of categories; choosing a set of attributes to evaluate each category, and developing descriptions of each category in terms of the attributes. Specific criteria should be either numeric (e.g. human cancer endpoint_ or narrative (e.g. comparison of ecological properties to a reference area). Obviously, the criteria will have to match the categories. Implementation of this recommendation requires that the engineering, business, regulatory, and societal concerns be represented in the planning of the risk assessment.
Define the risk categories
Develop criteria for ranking the categories Apply the categories and criteria in the planning and problem formulation stage of the ecological and human health risk assessment Score the categories against the criteria Identify sources of uncertainty and present them probabilistically where possible
All assessments develop or assume categories for ranking and criteria for judging them. A clear definition of risk categories for comparison will assure a more even application of a comparative risk. The criteria provide the bases for comparing specific categories among different alternatives or technologies. Assures that the output of the assessments will be compatible with the CRA framework and consistent with recent SAB recommendations (USEPA, 2000). This should avoid the need for second order assumptions or “retrofitting”. If the CRA is to move beyond the “comparison of risks” it must integrate or rank the categories. Some method of scoring or ranking is an obvious solution. This broadens the field of knowledge open to the decision maker and informs the decision in a quantitative manner.
The literature indicates that one way to generate some confidence in the systems is to have more than one set of categories against which to make the comparisons or have more than one panel of experts making the scores. The USACE has already proceeded to identify sources of uncertainty and even prioritize them for a subset of categories (Vorhees et al., 2002)
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
Develop comparison criteria
Determine issues and broad categories of comparison
Determine alternatives
Address spatial scales
Incorporate public advice Calculate or estimate the risks
Uncertainty quantification
Convene Delphi panel
Uncertainty characterization
Address temporal scales
Interpret relative risks
Make the decision Figure 1 Schematic of relevant CRA activities.
or hypotheses for the other categories of risk will be more complicated given the lack of specific, accepted guidance for evaluating these risks. For example, in the case of engineering reliability, evaluating the risks that alternatives will be unable to attain the desired risk reduction over a specific time horizon will involve use of standardized environmental risk assessment methods as well as engineering methods associated with conducting a failure analysis for designed structures. Assessing the socioeconomic risks associated with a range of alternative management options poses particular challenges given the broad array of subcategories of risks involved, including cultural impacts, lost use of natural resources, issues related to environmental justice, and the difficult in describing the uncertainties associated with those risks. The degree to which the categories and subcategories of risks are aggregated or disaggregated will have significant consequences for how the risk information is interpreted and used in decision making. For example, the aggregated risk to receptors in contact with surface water may be estimated as high; however, when disaggregated into contaminant classes actual risks may vary to a considerable degree over a range of difference temporal and spatial scales. Step C. Develop Criteria for Basis of Comparison Once the relevant risk categories have been identified, comparison criteria and sub-criteria can be developed. In this step, endpoints, or criteria, are selected for each category and subcategory of risk that
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
will form the basis for comparison among the selected alternatives. The criteria may be a combination of general and site-specific features. Factors such as spatial-scale, temporal variation or uncertainty can be incorporated as specific criteria, if desired. Criteria should be assessed and modified to achieve an exhaustive and consistent set of measures for further analysis (Roy, 1985). The development and acceptance of the criteria/sub-criteria set by decision-makers and stakeholders is a significant step before costs and effort are expended in assessing criteria values for the various alternatives. While some criteria modifications or clarification may be necessary in later stages, it is beneficial to achieve a basic level of consensus on the overall criteria set for clarity and direction in the assessment stage. Step D. Calculate or Estimate the Risk Based on Criteria At this stage of the CRA, information and data are collected for populating the selected criteria. In most cases it will be necessary to use both quantitative and qualitative information to describe the relevant risks. The criteria/sub-criteria set defined in the previous stage guides the assessment effort. Step E. Interpret Relative Risks and Potential Tradeoffs Among Alternatives At this stage, the CRA process can be efficiently linked with a multi-criteria decision analysis (MCDA) process so that risk-based criteria values can be interpreted along with other non-risk decision criteria. MCDA methods are most useful in integrating the various criteria to show potential tradeoffs between alternatives in terms of the various decision criteria. Sensitivity of the alternative ranking to variations in criteria values through spatial and temporal variation or uncertainty can be explored at this stage. In addition, MCDA allows decision-makers to explore the variation of alternative rankings to criteria weights and stakeholder values. Within some MCDA processes, risk neutral, risk averse and risk tolerant perspectives can be explored with respect to the decision criteria. The analytical methods used for integrating the criteria are described in more detail in the MCDA review chapter. 4.2 SUMMARY CRA has provided a useful framework for organizing the various risks inherent in environmental decisions into a logical and transparent format for decision-makers and stakeholders. Its cross-cutting perspective provides a valuable complement to indicators and benchmarking efforts, and issue-specific regulatory programs. Its broadly scoped analysis approach complements the narrower, more detailed risk assessments performed in support of specific regulatory proposals. In private industry, many product and service design choices involve environmental tradeoffs, and comparative risk analysis helps inform those tradeoffs. The role of CRA in systematically gathering risk-related information links efficiently with frameworks such as MCDA that can integrate a broader scope of risk and non-risk-related criteria into structured and transparent decisions.
Levner, E., Linkov, I., Proth, J.M., eds. “Strategic Management of Marine Ecosystems,” Kluwer, Amsterdam 2004 (in press).
5. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.
16.
Andrew, C.J., Apul, D.S., and Linkov, I. (2004) Comparative risk assessment: past experience, current trends and future directions. In Linkov, I. And Ramadan, A. eds. “Comparative Risk Assessment and Environmental Decision Making” Kluwer, 2004 Andrews, C.J. (2002) Humble Analysis: The Practice of Joint Fact-Finding,Praeger, Westport, CT. Beierle, T.C. (2002). The quality of stakeholder-based decisions. Risk Analysis 22(4):739-749 Cura, J.J., Bridges, T.S., McAdrle, M.E. (2004) Comparative risk assessment methods and their applicability to dredged material management decision-making. Human and Ecological Risk Assessment, 10:485-503 Gutenson, D. (1997) "Comparative risk: what makes a successful project?" Duke Environmental Law and Policy Forum, Vol VII, No1,pp 69-80. Jones, K. (1997) Can comparative risk be used to develop better environmental decisions? Duek Environmental Law and Policy Forum, Vol VII, No1, pp 33-46 Lash, J. (1996) Integrating science, values, and democracy through comparative risk assessment. In Finkel, A.M., and Golding, D. (eds), Worst Things First? The Debate Over Risk-Based National Environmental Priorities, pp. 69-86, Resources for the Future, Washington, DC Linkov, I., Ramadan, A., eds “Comparative Risk Assessment and Environmental Decision Making,” Kluwer, Amsterdam 2004. Morgan, M.G., Florig, H.K., DeKay, M.L. (2000) Categorizing risks for risk ranking. Risk Analysis 20(1):49-58 Roy, B. (1985, translation 1996). Multicriteria methodology for decision aiding. Kluwer, Boston. Seager, T., and Gardner, K.H. (2005) Barriers to adoption of novel environmental technologies: contaminated sediment. In J.M. Proth, E. Levner, I. Linkov, eds. “Strategic Management of Marine Ecosystems” Kluewer (in press) Tal, A., Linkov, I. (2004, in press). The Role of Comparative Risk Assessment in Addressing Environmental Security in the Middle East. Risk Analysis. U.S. Environmental Protection Agency (USEPA), Office of Policy Analysis, Office of Policy, Planning, and Evaluation (1987) Unfinished Business: A Comparative Assessment of Environmental Problems, Overview report and technical appendices, USEPA, Washington, DC. USEPA (U.S. Environmental Protection Agency)/USACE (US Army Corpts of Engineers) (1992) Evaluating environmental effects of dredged material management alternatives – A technical framework. EPA 842-B-92-008. Department of the Army, Washington, DC. USEPA (U.S. Environmental Protection Agency). (2000). Toward integrated environmental decision-making. EPA-SAB-EC-00-011. Science advisory board, Washington, DC. Vorhees, D.J., Driscoll, S.B., von Stackelberg, K. (2002) An evaluation of sources of uncertainty in a dredged material assessment. Human Ecological Risk Assessment. 8(2):369-89
