Experiences and Attitudes Towards ...

Zoonoses and Public Health

ORIGINAL ARTICLE

Experiences and Attitudes Towards Evidence-Informed Policy-Making Among Research and Policy Stakeholders in the Canadian Agri-Food Public Health Sector I. Young1,2, K. Gropp1, K. Pintar3, L. Waddell1,2, B. Marshall4, K. Thomas4, S.A. McEwen2 and A. Rajic2,5 1 2 3 4 5

Laboratory for Foodborne Zoonoses, Public Health Agency of Canada, Guelph, ON, Canada Department of Population Medicine, University of Guelph, Guelph, ON, Canada Laboratory for Foodborne Zoonoses, Public Health Agency of Canada, Ottawa, ON, Canada Centre for Food-borne, Environmental and Zoonotic Infectious Diseases, Public Health Agency of Canada, Guelph, ON, Canada Nutrition and Consumer Protection Division, Food and Agriculture Organization, Roma, Italy

Impacts

• Through a series of focus groups and qualitative interviews, research and •

•

policy stakeholders in Ontario, Canada, identified six key principles necessary to enhance the use of scientific evidence to inform policy-making in the agri-food public health sector Critical factors included establishing clear policy objectives and context, identifying credible evidence and integrating it with other diverse policy inputs, and enhancing communication, networks, relationships, organizational capacity and individual skills among research and policy communities Ongoing and planned efforts, a supportive culture, and additional education and training in these areas are needed to facilitate evidence-informed policy-making in this sector

Keywords: Agri-food public health; evidence-informed policy-making; knowledge transfer and exchange; qualitative research Correspondence: I. Young. Laboratory for Foodborne Zoonoses, Public Health Agency of Canada, 160 Research Lane, Suite, 206 Guelph, ON N1G 5B2, Canada. Tel.: +1 519 826 2098; Fax: +1 519 826 2255; E-mail: [email protected] Received for publication July 18, 2013 doi: 10.1111/zph.12108

© 2014 Blackwell Verlag GmbH

Summary Policy-makers working at the interface of agri-food and public health often deal with complex and cross-cutting issues that have broad health impacts and socioeconomic implications. They have a responsibility to ensure that policy-making based on these issues is accountable and informed by the best available scientific evidence. We conducted a qualitative descriptive study of agri-food public health policy-makers and research and policy analysts in Ontario, Canada, to understand their perspectives on how the policy-making process is currently informed by scientific evidence and how to facilitate this process. Five focus groups of 3–7 participants and five-one-to-one interviews were held in 2012 with participants from federal and provincial government departments and industry organizations in the agri-food public health sector. We conducted a thematic analysis of the focus group and interview transcripts to identify overarching themes. Participants indicated that the following six key principles are necessary to enable and demonstrate evidence-informed policy-making (EIPM) in this sector: (i) establish and clarify the policy objectives and context; (ii) support policy-making with credible scientific evidence from different sources; (iii) integrate scientific evidence with other diverse policy inputs (e.g. economics, local applicability and stakeholder interests); (iv) ensure that scientific evidence is communicated by research and policy stakeholders in relevant and user-friendly formats; (V) create and foster interdisciplinary relationships and networks across research and policy communities; and (VI) enhance organizational capacity and individual skills for EIPM. Ongoing and planned efforts in these areas, a supportive culture, and additional education and training in both research and policy realms are important to 1

Evidence-Informed Policy-Making in Agri-Food Public Health

I. Young et al.

facilitate evidence-informed policy-making in this sector. Future research should explore these findings further in other countries and contexts. Introduction There is growing international momentum and commitment to ensure that credible and reliable scientific evidence is used to inform policy and practice (World Health Organization, 2004). In the agri-food public health sector, which we define as the interface of interdisciplinary fields such as food safety, veterinary public health, and ‘One Health’ (Rajic et al., 2013), policy-makers must deal with complex, cross-cutting and multi-disciplinary issues. These issues require consideration of a range of contextual factors (e.g. politics, economics and stakeholder perceptions) in addition to scientific evidence (Fazil et al., 2008; Rajic et al., 2013). Given their broad implications and the accountability that policy-makers must demonstrate, there is need to ensure that the policy-making process is informed by the best available scientific evidence. Evidence-informed policy-making (EIPM) is defined as the structured and transparent identification, assessment and integration of scientific evidence as an input into government and organizational policy-making (Lavis et al., 2009a). Evidence-informed policy-making is related to the broader field of knowledge transfer and exchange (KTE), which is an iterative process that includes the synthesis, dissemination, exchange and application of scientific evidence to improve its uptake and utilization among end-users (Graham et al., 2006; Straus et al., 2009; Canadian Institutes of Health Research, 2013; Rajic et al., 2013). KTE have been extensively developed and applied in many contexts during the past several years, particularly in the healthcare and health services sectors (Graham et al., 2006; Straus et al., 2009; Rajic et al., 2013). KTE also have a long history of application in agri-food public health in the form of agricultural extension services, which involves the exchange of evidence between researchers and practitioners in the field (e.g. veterinarians and producers) (Klerkx and Leeuwis, 2009). Evidence-informed policy-making shares many of the same key principles and concepts as KTE, but it focuses specifically on improving the use of scientific evidence in the policy-making context (Lavis et al., 2009a; Lomas and Brown, 2009; Rajic et al., 2013). Several studies have been conducted in the health sector to investigate factors affecting and facilitating the use of scientific evidence to inform policy-making (Innvaer et al., 2002; Mitton and Patten, 2004; Lavis et al., 2005; Bowen et al., 2009; Peirson et al., 2012). However, to the best of our knowledge, no previous studies have investigated these issues in the agri-food public health sector, and this information is needed to identify key factors necessary to

2

enhance EIPM in this sector. We conducted a qualitative descriptive study to investigate the perspectives of agri-food public health policy-makers and research and policy analysts in Ontario, Canada, about how scientific evidence is currently used to inform policy-making and how to facilitate the process (Sandelowski, 2000). Throughout this article, we refer to the term ‘scientific evidence’ as encompassing all potential sources of scientific knowledge about a subject (e.g. peer-reviewed journal articles, government and organizational reports, and unpublished surveillance data). Materials and Methods Participant selection The study population was agri-food public health policymakers and policy and research analysts from federal and provincial government departments and private industry organizations in Ontario, Canada. A list of potential participants was compiled through publicly available government directories, industry websites and key informants. Participants were purposively selected from this list to ensure a diverse representation of perspectives (i.e. policy and research, agri-food and public health) in each focus group (Patton, 2002). In qualitative research, purposive sampling refers to the selection of participants that can best inform the study objectives based on their key characteristics (Patton, 2002). Selected participants were emailed a recruitment notice, and those who did not respond within one to 2 weeks were contacted via telephone. If no response was received, or if participants declined, a new participant was recruited. If participants were interested in the study but could not attend one of the focus groups, we arranged to conduct one-to-one semistructured telephone interviews with them instead. An honorarium of $50 or a charity donation of equal value was offered as an incentive to participants. Data collection We collected data through a combination of focus groups and semistructured interviews (Patton, 2002; Krueger and Casey, 2009). Five focus groups of three to seven participants were conducted in Ontario, Canada, from March 2012 to May 2012. Two groups included participants from different federal departments (i.e. public health, food safety and agriculture), while one included only participants from the Ontario Ministry of Agriculture and Food and Ministry of Rural Affairs, one included only industry participants


I. Young et al.

and the other included individuals from federal and provincial governments and industry. Five semistructured interviews were also conducted with federal government participants from public health (n = 3) and food safety (n = 2) departments. The focus groups lasted between 1–2 h, and interviews lasted from 20 to 40 min. Focus groups were conducted using a structured guide that consisted of seven questions (Table S1–S2). The questions asked participants about their experiences and attitudes towards the use of scientific evidence to inform policy-making, methods to transfer or disseminate this evidence, the extent of stakeholder engagement in the process, and factors influencing these activities and decisions. A pilot focus group was held with eight policy and research analysts from the Public Health Agency of Canada in Guelph to pre-test and revise the questions before use. Focus groups were moderated by I.Y. with note-taking assistance from one additional team member. One-to-one interviews were also conducted by I.Y. and followed the same question guide. Focus groups and interviews were audio-recorded for accuracy. This study was approved by the University of Guelph Research Ethics Board (protocol #12JA026). Data analysis Focus group and interview audio-recordings were transcribed and validated by a research assistant. The transcriptions were imported into NVIVO 9 (QSR International Pty Ltd., Doncaster, Australia) for analysis. We conducted a thematic analysis of the focus group and interview transcripts to identify the key concepts and themes related to participants’ perceptions about EIPM and how to facilitate this process. Thematic analysis is a method of analysing textual or narrative data to extract and summarize the major recurrent and substantive themes (Braun and Clarke, 2006). Two analysts (I.Y. and K.G.) independently reviewed all of the transcriptions and wrote memos highlighting the key concepts and issues. Findings were discussed, and the analysts inductively developed a code list (Table S3). The code list was independently applied on all documents by both analysts, who met periodically to compare and discuss their findings. Coding results from both analysts were aggregated, and overall themes were developed and agreed upon by grouping and consolidating codes that represented similar concepts. Data validation and credibility were ensured through multiple processes (Creswell and Miller, 2000; Patton, 2002). First, the purposive sampling strategy ensured that participants with a diversity and range of characteristics were included in the study. We also used triangulation of data collection methods (i.e. focus groups and interviews) © 2014 Blackwell Verlag GmbH


and data analysts (Creswell and Miller, 2000; Patton, 2002). Finally, we conducted member-checking, whereby participants were sent a summary of the results via email and were asked to indicate whether our interpretation made sense to them and accurately reflected their perspectives (Creswell and Miller, 2000). Participant comments were integrated into the final narrative and interpretation. Results Participant characteristics A total of 93 stakeholders were invited to participate in this study, of which 33 (35.5%) participated (n = 28 in five focus groups and n = 5 in one-to-one interviews). Of those who did not participate, 35 indicated that they could not attend the focus groups due to various reasons (e.g. too busy or prior engagements), 26 did not respond or were away from the office, six indicated that they were not the best person to participate, two were not interested, and one did not show up. The demographic characteristics of participants are shown in Table 1. Thematic analysis results We identified six themes related to participants’ perceptions about EIPM and key principles necessary to facilitate this process in the agri-food public health sector. Both analysts agreed that data saturation was reached during analysis (i.e. all key insights and concepts were identified). Eleven participants (33.3%) provided feedback on the preliminary results during member-checking, and these comments were incorporated into the final summary and interpretation, while the other participants did not respond after two contact attempts. The following is a comprehensive description of the six themes. Illustrative quotes from participants to support each theme are shown in Table 2. Identify the policy objectives and context Several participants emphasized the importance of having clear policy objectives and the need to differentiate the type of policy being addressed (e.g. regulatory, strategic or programme-level) and stage of input (e.g. early or late in the process). Participants indicated that these factors could influence the extent, types and sources of evidence considered; the degree of stakeholder engagement; timelines; and procedural requirements. The reactive (e.g. urgent) versus proactive (e.g. ongoing) nature of the policy issue was also considered as a key factor affecting the EIPM process. For example, the process of identifying and assessing evidence was described as less systematic and in-depth when addressing urgent issues (e.g. in response to a disease 3


I. Young et al.

Table 1. Demographic characteristics of 33 focus group and interview participants Question

No.

%

Gender Female 24 72.7 Male 9 27.3 Current position best described as Policy analyst or advisor 18 54.5 Researcher or research analyst 8 24.2 Policy-maker, manager or director 7 21.2 Organization Federal government 18 54.5 Provincial government 11 33.3 Industry 4 12.1 Years of experience in current position 1–5 years 16 48.5 6–10 years 7 21.2 >10 years 10 30.3 Primary work activitiesa Policy analysis and advice 24 72.7 Policy development 16 48.5 Programme management and administration 16 48.5 Research 13 39.4 KTE 12 36.4 Risk analysis 5 15.2 Other (e.g. outbreak investigation, surveillance) 10 30.3 Areas of expertisea Public health 17 51.5 Food safety 13 39.4 Agriculture 9 27.3 Zoonotic diseases 7 21.2 Animal health 6 18.2 Microbiology/biotechnology 6 18.2 Healthcare 5 15.2 Environmental health/management 3 9.1 Years of experience in policy-making or informing the policy-making process 1–5 years 10 33.3 6–10 years 3 10.0 >10 years 17 56.7 a

More than one selection was possible.

outbreak) compared with a long-term situation (e.g. formulating new legislation). Some participants noted that enhanced planning and foresight activities could facilitate a more systematic approach to EIPM. Support policy with credible and trustworthy scientific evidence Participants consistently agreed that credible and reliable scientific evidence is needed to support policy-making. They noted a preference for peer-reviewed sources, but many also noted the importance of other trustworthy sources such as organizational and government reports, 4

surveillance and field data, and expert opinion. Some participants expressed frustration about how scientific uncertainty is explained and interpreted in the policy-making process and in knowing ‘how much evidence is enough’ to have an impact on policy. However, they also recognized that uncertainty is inherent in research and that policy decisions often have to be made regardless of the state of evidence. Integrate scientific evidence with other diverse policy inputs Most participants recognized that scientific evidence is only one ‘lens’ used by policy-makers to support their decisions. They discussed that other factors, including practicality and local applicability of options, economics, political considerations and timing, informal knowledge and experience, public acceptability, and stakeholder or lobbyist influences, are also considered in the policy-making process. For each policy issue, the factors are balanced or weighted in support of decision-making and greater weight may be given to different factors depending on the specific issue. Participants also noted that while it can be challenging to integrate the multiple mandates of different stakeholders for some agrifood public health issues, engagement and integration of these various perspectives are essential to achieve more robust and practical solutions. Communicate scientific evidence in relevant and user-friendly formats Participants indicated a need for more effective and targeted communication of evidence to improve its uptake and to reduce the chance of its misinterpretation among policy-makers and those who inform them. They discussed the importance of providing succinct ‘sound bites’ and the need to answer the ‘so what?’ question – of ensuring that messages convey the key implications and why policy-makers should care. Participants noted several formats that they use to inform policy (e.g. government briefing templates, posters and presentations), and they highlighted the value of formal and informal in-person conversations, demonstrations and field visits with policy-makers and analysts to communicate key research messages. There was also consensus that the agri-food public health sector could explore more opportunities to use social media and other innovative technologies in support of EIPM. Create and maintain relationships between science and policy stakeholders Participants noted the importance of relying on prior and ongoing trusted relationships as a first choice for expert © 2014 Blackwell Verlag GmbH

I. Young et al.


Table 2. Themes and illustrative quotes from focus group and interview participants Themes

Illustrative quotes

Identify the policy objectives and context

In developing things like clinical guidelines or prevention and control measures for infectious diseases, we would look at the evidence quite strongly. But then as you sort of move out of that practice milieu and more into strategic policy, in my experience you don’t have a lot of time to delve into the real depth of the evidence so it’s given kind of a cursory review. There are now areas that have continued to develop background work, research and analysis in the hope and anticipation of that ‘window of opportunity’ opening up at some point and to be ready for that to happen. You have to be absolutely sure about the source. The source is credible, reliable, well known. I think there’s a lot of research that industry does that might not be at the peer-reviewed level but it still should be there to guide. It really becomes a question of bringing pieces of information to a narrative as opposed to having a conclusive, scientific iron-clad case. The reality is, when you’re making policy decisions you always have to take into consideration elements that are less tangible, be they societal, political, economic. Based on my experience, stakeholders are very happy to take a place around the table. They really like to be involved right from the beginning. It’s something that we are looking for to make sure that they will receive good buy-in to the process. I think the real point is being as succinct as you can in getting at the real grain of truth of what your issue is. Really boil it down to what are the key points and describe them in a way that people who are not familiar with the subject matter can understand. I don’t think our industry has moved to accept online things as easily. Cell phone apps, Facebook, Twitter – it’s just not a widely accepted thing in organizations and we don’t understand how we would move into that role yet either. I think that’s the point that is missing in communications. It’s a segregation between policy and science. A lot of times things get lost in translation because there’s no close collaboration. What it comes down to maybe is relationships between certain scientists that have a capacity to convey the message. And a lot of policy people have science backgrounds and you want to kind of isolate them and get them talking to those broader-picture research scientists that have credibility and merit. I think we need a conscious effort put in our organizations to actually create that knowledge translation piece to inform policy and for science to understand how it can influence policy. If we keep asking the ‘so what’ questions, after a while you’re going to slowly change the culture. I think it [requires] other kinds of skills as a communicator to send the proper message in plain language. As a scientist, it’s not what we are used to. . . We haven’t been educated to do that. You also need to have the internal capacity to interpret the evidence, to have some understanding of what the evidence says and what it means in order to be accountable to inform the decision.

Support policy with credible and trustworthy scientific evidence

Integrate scientific evidence with other diverse policy inputs

Communicate scientific evidence in relevant and user-friendly formats

Create and maintain relationships between science and policy stakeholders

Enhance organizational capacity and individual skills

consultations and stakeholder engagement. However, this was also discussed as a limitation because of the possibility of excluding the perspectives of other key groups. Participants agreed that there is a need to connect to new networks and create linkages with a wider range of experts and stakeholders. They also emphasized the need to achieve stronger integration between science and policy communities and different disciplines in agri-food public health. They discussed a variety of methods that are used to facilitate these linkages, including communities of practice, working groups, seminars and workshops. Intermediaries (e.g. knowledge brokers) and funding agencies were discussed as having a key role to promote and foster these relationships.


Enhance organizational capacity and individual skills Participants recognized challenges in the practice of EIPM, including frequent changes in organizational priorities, staff turnover, fiscal limitations, lack of consistent documentation of the use of evidence in policy decisions and communication issues. However, they also expressed an eagerness to improve capacity development for EIPM in this sector. Participants also noted the importance of providing additional training for researchers and policy stakeholders to be able to engage in EIPM activities. For example, it was noted that many researchers do not have sufficient training to communicate research messages to end-users effectively. Participants also indicated that policy-makers and their

5


advisors often do not have the essential skills to identify, evaluate, synthesize and interpret scientific evidence to inform policy. Discussion Participants discussed six key principles to enable and facilitate EIPM in the agri-food public health sector. The first related to the importance of recognizing the policy objectives and context, which is consistent with previous research in the health sector (Mitton and Patten, 2004; Bowen et al., 2009; Lomas and Brown, 2009). Participants indicated that it is a challenge to comprehensively identify and assess scientific evidence in emergency management and outbreak response situations, which have tight timelines and require advance preparation. One example of a preparatory tool in these situations is Health Canada’s weight-of-evidence framework, which outlines recommended steps and criteria in identifying and evaluating evidence to guide actions and decisions during foodborne illness outbreak investigations (Health Canada, 2011). In conjunction with these anticipatory efforts, rapid risk assessment methods and structured rapid reviews of the scientific evidence can be used to support urgent decisionmaking in situations where an emerging agri-food public health threat is identified. International guidelines on the former have recently been published (World Health Organization, 2012); however, there is currently no consensus for how best to conduct structured rapid reviews to support decision-making in different contexts and settings (Ganann et al., 2010; Rajic et al., 2013), and further research is required in this area. Participants also discussed the importance of supporting opportunities for more proactive EIPM in this sector. Formal foresight and horizon scanning methods could offer opportunities to facilitate this process (Smith, 2007; Vanderstichel et al., 2010). These methods allow organizations to anticipate and plan for multiple and diverse future scenarios in order to inform present-day decision-making (Smith, 2007; Vanderstichel et al., 2010). For example, the Canadian Food Inspection Agency is currently using this approach as part of a multistakeholder initiative called the Foresight for Canadian Animal Health Project (Fore-CAN) to enhance animal health emergency management in Canada (Vanderstichel et al., 2010). Enhanced use and integration of these approaches across government agencies and departments is warranted. Participants highlighted a need to support policy with credible and trustworthy scientific evidence. Knowledge syntheses (e.g. systematic reviews) are widely considered an effective way to support EIPM because they are more transparent, systematic and have a lower risk of bias and greater contextual applicability compared with individual research 6

I. Young et al.

studies, expert opinion and traditional narrative reviews (Lavis et al., 2005; Waddell et al., 2009; Rajic et al., 2013). The European Food Safety Authority (EFSA) recently adopted these methods to support and inform their food and feed safety risk assessments, which are used to provide scientific opinions and advice for European Union policymaking on these issues (EFSA, 2010). There is a need to increase end-user awareness of knowledge synthesis methods and other approaches that can be used to systematically and transparently examine multiple sources of evidence in support of EIPM in this sector. Participants discussed the importance of integrating scientific evidence with other diverse policy inputs, a finding that has also been highlighted in previous studies of health policy-makers (Mitton and Patten, 2004; Bowen et al., 2009). This process can be made more transparent and accountable through the use of formal methods such as multi-criteria decision analysis (MCDA). MCDA can be used to combine multiple criteria (e.g. weight of evidence, efficacy, costs, practicality) to help inform decision-making about food safety interventions and other policy-relevant issues in this sector (Fazil et al., 2008; Ruzante et al., 2010). For example, Ruzante et al. (2010) developed a multifactorial framework that uses MCDA to incorporate measures of public health impact, market impact, consumer perceptions and acceptance of risk, and social sensitivities to support risk prioritization decision-making of foodborne pathogen and food product combinations in Canada. A unique finding of this study was the identified challenge of addressing complex agri-food public health policy issues due to the differing mandates of the various stakeholders. Stakeholder dialogues are a potential method that could be used to facilitate policy-making in these situations (Lavis et al., 2009b). They are interactive, facilitated forums that bring together diverse stakeholders to discuss and help guide policy decisions about high-priority and controversial issues (Lavis et al., 2009b). One example is a stakeholder dialogue that was conducted with government, science, business and community stakeholders in New Zealand to support and help to achieve sustainable decision-making about the potential impacts of future food technologies (Cronin et al., 2012). Participants highlighted the importance of appropriately communicating scientific evidence to facilitate EIPM, a finding that has been noted in several previous studies and reviews in the literature (Innvaer et al., 2002; Lavis et al., 2005; Mitton et al., 2007; Rajic et al., 2013). Many different frameworks, toolkits, guidelines and summary formats have been developed to support more effective dissemination and communication of research to end-users (Lavis et al., 2009a; Canadian Foundation for Healthcare Improvement [CFHI], 2010; Wilson et al., 2010; Bennett and Jessani, 2011; Food and Agriculture Organization [FAO], 2011). © 2014 Blackwell Verlag GmbH

I. Young et al.

One recommended approach to summarizing research for end-users is to use a layered or graded-entry format. For example, the CFHI has published guidelines for developing evidence summaries using a 1 : 3 : 25 approach, which includes a one-page summary of the key ‘take home’ messages and a three-page, plain language executive summary to accompany the 25-page full report (CFHI, 2010). Participants also identified an interest in exploring the potential use and effectiveness of social media tools to facilitate EIPM activities. These tools can be used to increase the rapid and timely dissemination of evidence, to engage with new and diverse networks and to monitor the uptake and impact of KTE and EIPM activities (Eysenbach, 2009; FAO, 2011). However, these tools also have some limitations, including the need to filter through background noise and misinformation and to ensure that target audiences can be effectively reached (Eysenbach, 2009). Participants in our study discussed the importance of developing new linkages and relationships between research and policy communities as key strategies to facilitate EIPM. These strategies have been previously described as potentially the most critical to the success of EIPM and KTE efforts (Lavis et al., 2006; Mitton et al., 2007). In a recent survey of Canadian food safety researchers, Wolfe et al. (2012) found that there was a lack of adoption of integrated KTE approaches (i.e. engagement of end-users from the beginning and throughout the study). Enhanced linkages and increased adoption of integrated KTE approaches are needed in this sector to support EIPM. One strategy discussed among participants is the use of knowledge brokers, which are intermediary individuals, groups or agencies that facilitate KTE and EIPM through a variety of interrelated functions and roles, including: promoting linkages, building networks, and mediating relationships and dialogues between research and policy communities; supporting enduser access to, interpretation, and application of evidence; and strengthening organizational capacity and systems-level processes for EIPM (Ward et al., 2009). Knowledge brokers should be trustworthy, politically neutral, credible and knowledgeable about the issue they are facilitating (Ward et al., 2009). However, many knowledge exchange strategies – including knowledge brokering – have yet to be comprehensively evaluated in different contexts (Ward et al., 2009; LaRocca et al., 2012), and further research is needed on their effectiveness. Participants discussed the importance of sufficient organizational capacity and training to support EIPM in this sector, a finding that is consistent with previous research in the health sector (Mitton and Patten, 2004; Bowen et al., 2009; Peirson et al., 2012). The establishment of a structured and practical organizational framework for EIPM, dedicated science-to-policy or KTE units and roles, and increased resources and recognition © 2014 Blackwell Verlag GmbH


for these activities could help to facilitate this capacity. For example, many funding agencies now require researchers to pre-specify a KTE plan and how the results will be used by end-users as part of grant applications. Similar tools could be used to support improved documentation of how scientific evidence is identified, assessed and applied in policy decisions. Enhancing skills through training was seen as a valuable way to support EIPM for both researchers and policy-makers. Researcher ability in this area could be enhanced although future academic or continuing education programmes and workshops and through the provision of supportive resources and toolkits. Training and support are also needed for policy-makers and those who support them to conduct EIPM activities such as formulating appropriate questions, identifying and critically appraising evidence, and assessing the applicability of evidence within the local context. Several web-based resources are available in this area in the health sector (Lavis et al., 2009a; Dobbins et al., 2010). In recognition of the need for similar resources specific to the agri-food public health context, a comprehensive KTE and EIPM handbook was recently developed for science-to-policy professionals in this sector to guide and facilitate their conduct of these activities (Rajic and Young, 2013). To ensure credibility of our results, we used purposive sampling to identify a diverse range of participants with different characteristics and perspectives on the issue (Creswell and Miller, 2000; Patton, 2002). In addition, we used multiple data collection methods and analysts and member-checking to help ensure that the identified themes were robust and accurately represented participants’ views and experiences. One limitation of this study is the relatively small number of participants (n = 33). However, we are confident that collection of additional data would not have changed the overall conclusions, because both analysts agreed that data saturation of the thematic categories was reached in the analysis and the findings were confirmed and validated among a diverse group of participants. Another limitation is that due to the heterogeneity of participants in each group, we were not able to make explicit and formal comparisons among themes by participant characteristics such as public health versus agri-food perspectives. Using credible evidence and integrating it with diverse policy inputs, enhancing research and policy communication and relationships, and developing organizational capacity and individual skills are all key components of the successful adoption and establishment of EIPM in this sector; however, the applicability of each of these factors also depends on the specific policy objectives and context, including the level of policy (e.g. regulatory, strategic, or programme-based) and nature of the issue (e.g. proactive 7


versus reactive) to be addressed. Ongoing and planned efforts in these areas, a supportive culture, and enhanced education and training in both research and policy realms will be necessary to facilitate EIPM in this sector. Future research is needed to investigate and validate these findings in other countries and settings and among larger, representative samples of stakeholders, while also considering participant characteristics such as area of expertise, years of experience and other potential modifying factors. Additional research could also include the application and evaluation of different methods, strategies and tools to support various EIPM and KTE activities in this sector. Acknowledgements We greatly acknowledge all of the individuals that participated in the focus groups and interviews. We also thank Kathy Kimmerly for audio transcription, Mia Cikovic for transcription validation and Mai Pham for focus group moderation assistance. This study was funded by the University of Guelph and Ontario Ministry of Agriculture and Food/Ministry of Rural Affairs’ Agri-Food and Rural Link ‘Knowledge Translation and Transfer Funding Program’ and the Laboratory for Foodborne Zoonoses, Public Health Agency of Canada. References Bennett, G., and N. Jessani, 2011: The Knowledge Translation Toolkit. SAGE Publications India Pvt Ltd., New Delhi, India. Bowen, S., T. Erickson, P. J. Martens, and S. Crockett, 2009: More than “using research”: the real challenges in promoting evidence-informed decision-making. Healthc. Policy 4, 87– 102. Braun, V., and V. Clarke, 2006: Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101. Canadian Foundation for Healthcare Improvement, 2010: Communication notes: Reader-friendly writing – 1:3:25. Available at: http://www.cfhi-fcass.ca/publicationsandresources/ resourcesandtools/communicationnotes/10-06-01/ d497a465-5398-4ec8-addf-d7cbf86b1e43.aspx (accessed on 1 November 2013). Canadian Institutes of Health Research, 2013: About knowledge translation. Available at: http://www.cihr-irsc.gc.ca/e/29418. html (accessed on 1 November 2013). Creswell, J. W., and D. L. Miller, 2000: Determining validity in qualitative inquiry. Theory Pract. 39, 124–130. Cronin, K., V., Baker, J., Foote, M., Hepi, J., Hutchings, G., Nicholas, and K., Taupo, 2012: Stakeholder dialogue on future food technologies: Report on phase III of the ‘coming to the table’ research project. Available at: http://www.esr.cri. nz/SiteCollectionDocuments/ESR/PDF/SocialScience/FutureFoodsFieldworkReport29082012.pdf (accessed on 1 November 2013).

8

I. Young et al.

Dobbins, M., K. DeCorby, P. Robeson, H. Husson, D. Tirilis, and L. Greco, 2010: A knowledge management tool for public health: health-evidence.ca. BMC Public Health 10, 496. European Food Safety Authority, 2010: Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA J. 8, 1637. Eysenbach, G., 2009: Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J. Med. Internet Res. 11, e11. Fazil, A., A. Rajic, J. Sanchez, and S. McEwen, 2008: Choices, choices: the application of multi-criteria decision analysis to a food safety decision-making problem. J. Food Prot. 71, 2323– 2333. Food and Agriculture Organization, 2011: Food security communications toolkit. Available at: http://www.fao.org/docrep/ 014/i2195e/i2195e00.htm (accessed on 1 November 2013). Ganann, R., D. Ciliska, and H. Thomas, 2010: Expediting systematic reviews: methods and implications of rapid reviews. Implement. Sci. 5, 56. Graham, I. D., J. Logan, M. B. Harrison, S. E. Straus, J. Tetroe, W. Caswell, and N. Robinson, 2006: Lost in knowledge translation: time for a map? J. Contin. Educ. Health Prof. 26, 13–24. Health Canada, 2011: Weight of evidence: Factors to consider for appropriate and timely action in a foodborne illness outbreak investigation. Available at: http://www.hc-sc.gc.ca/ fn-an/pubs/securit/2011-food-illness-outbreak-eclosionmalad-ailments/index-eng.php (accessed on 1 November 2013). Innvaer, S., G. Vist, M. Trommald, and A. Oxman, 2002: Health policy-makers’ perceptions of their use of evidence: a systematic review. J. Health Serv. Res. Policy 7, 239–244. Klerkx, L., and C. Leeuwis, 2009: Establishment and embedding of innovation brokers at different innovation system levels: insights from the Dutch agricultural sector. Technol. Forecast. Soc. Change 76, 849–860. Krueger, R. A., and M. A. Casey, 2009: Focus Groups: A Practical Guide for Applied Research. SAGE Publications, Thousand Oaks, CA. LaRocca, R., J. Yost, M. Dobbins, D. Ciliska, and M. Butt, 2012: The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health 12, 751. Lavis, J. N., H. Davies, A. Oxman, J. L. Denis, K. Golden-Biddle, and E. Ferlie, 2005: Towards systematic reviews that inform health care management and policy-making. J. Health Serv. Res. Policy 10(Suppl 1), 35–48. Lavis, J. N., J. Lomas, M. Hamid, and N. K. Sewankambo, 2006: Assessing country-level efforts to link research to action. Bull. World Health Organ. 84, 620–628. Lavis, J. N., A. D. Oxman, S. Lewin, and A. Fretheim, 2009a: SUPPORT tools for evidence-informed health policymaking (STP). Health Res. Policy Syst. 7(Suppl 1), I1. Lavis, J. N., J. A. Boyko, A. D. Oxman, S. Lewin, and A. Fretheim, 2009b: SUPPORT tools for evidence-informed health policymaking (STP) 14: organising and using policy dialogues


I. Young et al.

to support evidence-informed policymaking. Health Res. Policy Syst. 7(Suppl 1), S14. Lomas, J., and A. D. Brown, 2009: Research and advice giving: a functional view of evidence-informed policy advice in a Canadian ministry of health. Milbank Q. 87, 903–926. Mitton, C., and S. Patten, 2004: Evidence-based priority-setting: what do the decision-makers think? J. Health Serv. Res. Policy 9, 146–152. Mitton, C., C. E. Adair, E. McKenzie, S. B. Patten, and B. Waye Perry, 2007: Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 85, 729–768. Patton, M. Q., 2002: Qualitative Research and Evaluation Methods. SAGE Publications, Thousand Oaks, CA. Peirson, L., D. Ciliska, M. Dobbins, and D. Mowat, 2012: Building capacity for evidence informed decision making in public health: a case study of organizational change. BMC Public Health 12, 137. Rajic, A., and I., Young, 2013: Knowledge synthesis, transfer and exchange in agri-food public health: A handbook for scienceto-policy professionals. Available at: http://atrium.lib.uoguelph.ca/xmlui/handle/10214/7293 (accessed on 1 November 2013). Rajic, A., I. Young, and S. A. McEwen, 2013: Improving the utilization of research knowledge in agri-food public health: a mixed-method review of knowledge translation and transfer. Foodborne Pathog. Dis. 10, 397–412. Ruzante, J. M., V. J. Davidson, J. Caswell, A. Fazil, J. A. L. Cranfield, S. J. Henson, S. M. Anders, C. Schmidt, and J. M. Farber, 2010: A multifactorial risk prioritization framework for foodborne pathogens. Risk Anal. 30, 724–742. Sandelowski, M., 2000: Whatever happened to qualitative description? Res. Nurs. Health 23, 334–340. Smith, J. E., 2007: Science and technology foresight: a provocative tool for contending with future challenges in food safety and public veterinary medicine. Vet. Ital. 43, 237–246. Straus, S. E., J. Tetroe, and I. Graham, 2009: Defining knowledge translation. Can. Med. Assoc. J. 181, 165–168.



Vanderstichel, R., I. Van der Linden, S. Renwick, and M. Dubuc, 2010: Foresight: an innovative approach for animal health emergency preparedness. Can. Vet. J. 51, 372–374. Waddell, L., A. Rajic, J. Sargeant, S. Parker, A. Deckert, and S. McEwen, 2009: The methodological soundness of literature reviews addressing three potential zoonotic public health issues. Zoonoses Public Health 56, 477–489. Ward, V., A. House, and S. Hamer, 2009: Knowledge brokering: the missing link in the evidence to action chain? Evid. Policy 5, 267–279. Wilson, P. M., M. Petticrew, M. W. Calnan, and I. Nazareth, 2010: Disseminating research findings: what should researchers do? A systematic scoping review of conceptual frameworks. Implement. Sci. 5, 91. Wolfe, D. M., J. Sargeant, M. Dobbins, and S. A. McEwen, 2012: Knowledge translation and exchange in the Canadian microbial food safety system: a quantitative assessment of researcher awareness, attitude, and activities with government policymakers. Food Policy 37, 589–599. World Health Organization, 2004: World report on knowledge for better health: Strengthening health systems. Available at: http://www.who.int/rpc/meetings/world_report_on_knowledge_for_better_health.pdf (accessed on 1 November 2013). World Health Organization, 2012: Rapid risk assessment of acute public health events. Available at: http://whqlibdoc.who. int/hq/2012/WHO_HSE_GAR_ARO_2012.1_eng.pdf (accessed on 1 November 2013).

Supporting Information Additional Supporting Information may be found in the online version of this article: Table S1. Focus Group Question Guide. Table S2. Semi-Structured Interview Question Guide. Table S3. Qualitative Thematic Analysis Code List.

9