The user, not the tool: perceptions of credibility and relevance affect the uptake of prioritization
Article accepted in the journal “Environmental Management”, ISSN 0364-152X, DOI 10.1007/s00267015-0653-3 The final publication is available at http://link.springer.com/article/10.1007/s00267-015-0653-3 Authors' complete mailing addresses: Milena Kiatkoski Kim (corresponding author):
[email protected]. School of Earth and Environmental Sciences, James Cook University, Douglas, Queensland 4811, Australia. Fax: +61 7 4781 5581 Louisa Evans: ARC Centre of Excellence for Coral Reef Studies, James Cook University Douglas, Queensland 4811, Australia. Current address: College of Life and Environmental Sciences, University of Exeter, Rennes Drive, Exeter, EX4 4RJ, United Kingdom. Lea M. Scherl: School of Earth and Environmental Sciences, James Cook University, Douglas, Queensland 4811, Australia. Helene Marsh: School of Earth and Environmental Sciences, James Cook University, Douglas, Queensland 4811, Australia. Acknowledgements This research was funded by Graduate Research School, the School of Earth and Environmental Sciences, Professor Helene Marsh’s Services Fund (James Cook University) and the Skyrail Foundation. M.K.K. was supported by a JCU Postgraduate Research Scholarship and a stipend scholarship from Professor Helene Marsh’s Services Fund. The authors acknowledge the time and knowledge shared by interviewees and the support from the Threatened Species Unit from the Queensland Government. We also appreciate the comments and suggestions from three anonymous reviewers. Abstract
1
Prioritisation methods have been used in conservation planning for over 20 years. The scientific literature focuses on the technical aspects of prioritisation, providing limited information on factors affecting the uptake of priorities. We focused on the Back on Track species prioritization program in Queensland, Australia, used to prioritise species conservation efforts across Queensland from 2005. The program had low uptake by intended users. Our study aimed to identify the perceived limitations in the technical-scientific quality of this speciesbased prioritisation process and its outcomes in terms of credibility (scientific adequacy of the technical evidence) and relevance (of information to the needs of decision makers). These criteria have been used to understand the uptake of scientific information in policy. We interviewed 73 key informants. Perceptions of credibility were affected by concerns related to the use of expert judgement (rather than empirical evidence) to assess species, impressions that key experts were not included in the planning process and the lack of confidence in the information supporting prioritisation. We identified several trade-offs and synergies between the credibility and relevance of priorities to potential users. The relevance of the output plans was negatively affected by the lack of clarity about who were potential users and implementers of the priorities identified. We conclude with recommendations to enhance the credibility and relevance of such initiatives. Keywords: prioritization; credibility; relevance; conservation planning; users; uptake INTRODUCTION Much of the scientific knowledge relevant to the solution of environmental problems fails to inform decisionmaking (McNie 2007). Biodiversity conservation science and planning are a case in point (Knight et al. 2008; Fox et al. 2012; Jacobson et al. 2013). Decision-makers are faced with difficult choices of how and where to invest limited resources. Conservation planning attempts to design methods and instruments to influence how decision-makers allocate resources to minimise the loss of biodiversity and other valued aspects of the natural world (Pressey & Bottrill 2009). Although conservation planning initiatives have been used to determine and assess priorities, such initiatives are rarely evaluated (Bottrill & Pressey 2012). There is limited information about whether potential users utilise the prioritization outputs to inform their investments – a process referred to here as the “uptake” of priorities – and, especially, what factors affect the uptake of conservation priorities (Kim 2014). Species-based prioritization is one type of conservation planning used to assist investment decisions in species conservation and recovery. Over the last two decades, many species prioritization methods have been adopted by organisations to inform their conservation investment decisions (e.g. Marsh et al. 2007; Joseph et al. 2008,
2
Cawardine et al. 2012). However, few studies discuss conservation managers’ perceptions of the technicalscientific aspects of species prioritization methods (e.g. Baldi et al. 2010), and there have been no studies on the uptake of priorities identified via species prioritization methods in decision-making. The growing body of literature focusing on the gap between science and policy can contribute to the understanding of factors affecting the uptake of conservation priorities. The ‘science–policy gap’ literature points out that the use of scientific information in policy is mediated by potential users’ perceptions (Contandriopoulos et al. 2010; Jacobson et al. 2013; Fazey et al. 2014). Perceptions typically reflect subjective judgement (Bauler 2012), rather than the quality of scientific information. Cash et al. (2003) provide compelling evidence that potential users’ perceptions of three attributes affect the uptake of scientific information: credibility, relevance and legitimacy (Cash et al. 2003). These attributes have been adopted by several studies to understand the adoption of environmental science in policy (Bauler 2012; Hegger et al. 2012; Sarkki et al. 2014). Potential trade-offs and, to a lesser extent, synergies between these attributes have been acknowledged (Cash et al. 2003, Bauler 2012). Understanding such trade-offs can provide guidance in the concrete operation and design alternatives for science-policy interfaces (SPIs) (Box 1). Nonetheless, there are few empirical studies addressing the practical meaning of trade-offs and synergies between these attributes (e.g. Sarkki et al. 2014). Box 1: Definitions of key terms utilized in this article DEFINITIONS Science-Policy Interfaces (SPIs): “social processes which encompass relations between scientists and other actors in the policy process, and which allow for exchanges, co-evolution, and joint construction of knowledge with the aim of enriching decision-making” (van den Hove 2007: 815) Credibility: “involves the scientific adequacy of the technical evidence and arguments” (Cash et al. 2003: 8086) Relevance: “the relevance of the [information] to the needs of decision makers” (Cash et al. 2003: 8086) Legitimacy: “the production of information and technology [is] respectful of stakeholders’ divergent values and beliefs, unbiased in its conduct, and fair in its treatment of opposing views and interests.” (Cash et al. 2003: 8086) Species Prioritisation: The identification of priorities for species conservation and recovery (priority species, threats and/or actions). Decision-making tools such as species prioritisation methods (in our case, the method used by the BoTp) can inform this process of identification of priorities. Output: The output produced by the science-policy interface. In our specific case, the Actions for Biodiversity Plans (available at https://www.ehp.qld.gov.au/wildlife/prioritisation-framework/bot-biodiversitydocuments.html) Outcome: The ability of science-policy interfaces to influence the behaviour of intended audiences. In our specific case, the main intended outcome would be that decision makers use the Actions for Biodiversity Plans to decide where and how they invest in species conservation in Queensland.
3
SPIs include conservation planning initiatives in which scientists interact with other policy actors with the aim of enhancing decision-making on priorities for environmental conservation and management. In such cases, credibility, relevance and legitimacy can be used to understand the uptake of conservation plans by decisionmakers. This paper explores users’ perceptions of a species-based prioritization process and its outcomes, using a case study that had limited uptake by potential users (Table 1) – the Back on Track species prioritization framework. The Back on Track program (BoTp) was initiated by a state government agency and included expert elicitation to generate plans (described below). It can thus be considered as an example of a SPI. While credibility is generally associated with scientific rigour, legitimacy is related to decision-making and is thus, perceived as political. However, the term ‘scientific legitimacy’ can also be used in reference to a rigorous process of hypotheses testing, empirical validation and peer review (Heink et al. 2015). Potential users and participants of the BoTp were interviewed regarding their perceptions of the program (Methodology section, below). Their perceptions included comments on the technical-scientific features of the BoTp, and normative and ethical concerns, including political legitimacy. We thus restricted the scope of this study to technicalscientific attributes – credibility and relevance – while the political legitimacy of the BoTp was analysed in another publication (Kim et al. in review). We acknowledge conceptual overlaps between credibility and legitimacy as highlighted above by Heink et al. (2015). Therefore, ‘scientific legitimacy’ is implicitly included in the credibility concept we adopted in this study (Box 1). Our study aims to inform the design and implementation of SPIs in general, and conservation planning in particular, by: 1) Exploring perceptions of the technical-scientific quality, in terms of credibility and relevance, of a species prioritisation initiative; 2) Identifying synergies and trade-offs between credibility and relevance in the context of this case study. Case study: the Back on Track program The Back on Track program (BoTp) is a prioritization framework used by the Queensland state government (Australia), adapted from a method developed by Marsh et al. (2007). The BoTp was the first species-based prioritization framework to be implemented in Australia, and has been developed and implemented since 2005 (Supplementary Material). It prioritizes species (including those not legally listed as threatened), threats and actions using several criteria, including species’ probability of extinction, consequences of extinction and the
4
potential for successful recovery. The BoTp was implemented in five stages (Box 2) and included the participation of scientists (in expert panels) and non-scientists to identify priority species, threats and actions. It encompassed a vast and largely data deficient geographic area, involved several potential adopters and implementers, and was developed by a government agency with very limited human and financial resources available to environmental conservation. For example, at the time of our interviews the state Threatened Species Unit (responsible for the BoTp) had three full-time field staff, with an annual budget of less than US$5,000 to cover the whole state, which is almost three times larger than Texas. These problems are often faced by conservation planning initiatives.
Box 2 – Stages of the Back on Track program The framework was developed in five stages: STAGE 1 – EXPERT PANELS: consisted of expert panels assessing Queensland native species divided by taxonomic groups against seven criteria (Supplementary Material). The taxonomic groups assessed included all species listed under the national and the Queensland legislations, in addition to all mammal, bird, reptile, frog, freshwater fish, cartilaginous fish and butterfly taxa. Experts comprised community members, environmental consultants and experts from universities, museums, state and Commonwealth government agencies, local government and natural resource management (NRM) groups. During 1–2 day workshops, experts reached consensus in scoring species from 1 to 4 against each criteria. They also provided a score 1–4 reflecting their level of confidence on the assessment provided in each criteria. STAGE 2 – MANAGERS’ WEIGTHING: The method assumed that important factors that are usually implicitly considered by politicians and managers when allocating funds for conservation. The reliance only on threatened species lists or experts’ opinion based on ecological criteria for prioritizing species for conservation risks excluded such ‘implicit considerations’. Therefore, the method included both the social value of the species and management aspects. The framework incorporated such factors by asking managers to distribute 100 points between the seven criteria, by consensus, according to what they considered more important when making a resource allocation decision. Senior managers from the Queensland department of environment, and members of the boards and/or senior officers from NRM bodies weighted the criteria in 12 separate exercises. Each NRM group had the option of choosing between the weightings of either the Queensland department of environment or the NRM body managers to create their list of regional priority species. STAGE 3 – LISTS OF PRIORITY SPECIES: The experts’ scores were multiplied by the managers' weighting to determine the total score for each species. This exercise was repeated in each NRM region and at state level, given the differences in managers’ weights. Based on their total score, species were ranked as Critical (>299), High (250–299), Medium (200–249) or Low (
