Multi-context versus context-specific qualitative ...

J

Journal Code R S M

Article ID Dispatch: 27.01.12 5 5 No. of Pages: 8

CE: Matugas, Ma. Theresa ME:

Original Article Received 23 September 2011,

Revised 2 January 2012,

Accepted 17 January 2012

Published online in Wiley Online Library

(wileyonlinelibrary.com) DOI: 10.1002/jrsm.55

Multi-context versus context-specific qualitative evidence syntheses: combining the best of both Karin Hannesa*† and Angela Hardenb There is an increasing interest in the conduct of qualitative evidence syntheses (QES), particularly in the field of health care. Approaches to QES vary in the way they conduct a search, a critical appraisal or the data-analysis. To date, the use of multi-context versus context-specific QES has not yet been fully considered. In a multi-context, QES exhaustive searches are used that retrieve studies from a broad variety of geographical, socio-cultural, political, historical, economical, health care, linguistic, or other context relevant to the review. Authors of a context-specific QES would generally have a particular end user in mind, therefore, using a selective search strategy with a focus on one particular context in order to provide lines of actions or theories that are sensitive to a local setting. We used the insights from a recently conducted, context-specific QES to map out potential strengths and weaknesses of these two approaches and make recommendations regarding the future conduct of QES. We propose two ways of combining the best of both: the production of umbrella reviews of context-specific syntheses and/or the trans-cultural modification and trans-contextual adaptation of findings from multi-context syntheses. Copyright © 2012 John Wiley & Sons, Ltd. Keywords:

systematic reviews; qualitative evidence synthesis

Introduction One of the main characteristics of systematic reviews of effectiveness conducted and disseminated by international collaborations such as Cochrane, Campbell, and Joanna Briggs is comprehensiveness. This is generally achieved by conducting a sensitive search to retrieve all relevant published and unpublished literature to prevent publication bias entering the review process. The retrieved studies tend to cover a broad variety of different contexts: countries, regions, systems, and so on. There are few authors of effectiveness reviews that have engaged in discussions on how to account for contextual factors when advising practitioners and policy makers on what works or does not work. A recent systematic review from Zief et al. (2006) evaluating the impact of after-school programs on student outcomes opted for a limitation to the North American region because school contexts worldwide would differ significantly. However, most review authors do not consider process and implementation issues in their systematic reviews of effectiveness. This has recently been highlighted in a preliminary study from Cargo and colleagues, who screened a subset of reviews produced by the Campbell Collaboration (2011). Effectiveness reviews, like basic experimental designs, generally aim to control for potential confounders and attempt to limit the effect of contextual factors to provide results that are generalizable for the population under study. There has recently been some progress made in terms of recognizing limitations inherent in controlling variables for complex interventions, with a shift from the question ‘what works’ to ‘what works, for whom in what circumstances’. This is an emerging field where qualitative evidence synthesis has an important role to play.

a

KU Leuven, Leuven, Belgium University of East London, London, UK *Correspondence to: Karin Hannes, KU Leuven, Centre for Methodology of Educational Research, Andreas Vesaliusstraat 2, 3000 Leuven, Belgium † E-mail: [email protected] b

Authors’ Note Portions of this article were presented as a poster at the first biennial Global Implementation Conference, August 16, 2010, Washington DC, USA

Copyright © 2012 John Wiley & Sons, Ltd.

Res. Syn. Meth. 2012

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

K. HANNES AND A. HARDEN

2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Qualitative evidence synthesis (QES) or meta-synthesis of findings from qualitative research is a process of combining evidence from original qualitative studies to create new understanding by comparing and analyzing concepts and findings from different sources of evidence with a focus on the same topic of interest (Noyes et al., 2008). The popularity of this type of synthesis is increasing, particularly in the field of health care. A recent update of a review on published QES from Dixon-Woods et al. (2007) revealed some notable changes in the conduct of QES the last 4 years. Not only has their popularity increased, more recently published QES also present more systematic and sensitive searches. They tend to provide more details on the databases searched and better describe their search strings and keywords. More authors seem to be convinced of the added value of a critical appraisal of methodological quality. There is some ambivalence regarding the use of the outcome of these appraisal exercises. In several cases an appraisal exercise was performed, however, not used to include or exclude studies based on their methodological quality. Overall, it was noticed that the methods in recently conducted QES have become more transparent (Hannes and Macaitis, 2012). The international debate on techniques and criteria for searching and critically appraising qualitative studies to inform, enhance, extent or supplement systematic reviews of effectiveness has most likely influenced many authors in the field of QES (Cochrane Qualitative Research Methods Group, 2011). It was also noticed that many Q1 QES authors opt for a multi-context review by using an exhaustive search strategy, with little or no country limitations. Examples of this type of QES include the ones on barriers and facilitators to accessing and complying with tuberculosis treatment (Noyes and Popay, 2006), aspects that determine success and failure in school feeding Q2 programs (Greenhalgh et al., 2007), factors that shape young people’s sexual behavior (Marston and King, 2006), experiences of living with an HIV infection (Barroso and Powell-Cope, 2000), and parents’ experiences and perceptions of parenting programs (Kane et al., 2007). The first two examples listed explain or complement results from systematic reviews of effectiveness, whereas the other examples are stand alone approaches. What all these QES seem to have in common is that they summarize findings in a broader, international perspective. They also explicitly state that it is their aim to inform policy and/or practice. However, it is not always clear which particular practitioners or policy makers they are targeting. Policy and healthcare systems may vary substantially between countries (Vos et al., 2000). Countries may vary in aspects such as organizational context, including type of leadership, resources, cultural sensitivity, and support. Healthcare populations and practitioners may have different characteristics on the level of training, schooling, socio-economical status, and so on. Also, the ecological context or settings that are provided as a means to promote health care may vary (Chen, 2005). It follows that implications for policy and practice reported in multi-context QES may become far too general (or close to common sense). It is, therefore, most important to explore the relevance of context-specific QES that articulate the features of a particular health care, social welfare, or social-cultural system. Only few published QES have intentionally focused on one particular country or national system of, for example, health and education. We used the insights from recently conducted context-specific syntheses on barriers to implementing Evidence-based Practice to define and discuss both multi-context and context-specific approaches to QES—their potential strengths and weaknesses—and make recommendations regarding the future conduct of QES. What is meant by multi-context and context-specific QES? A lot of authors of QES have already complied with the general principle of being transparent about the methods and procedure they have used to conduct their synthesis. Where and when relevant, this includes a clear description of the questions and criteria that will guide the review, the search strategy, inclusion and exclusion criteria, standards of methodological adequacy, and the analytical approach followed. Authors of QES may vary substantially in their approach to searching and sampling papers (comprehensive or purposeful) and the signalto-noise ratio accepted in their decisions on whether or not to include findings from methodological flawed papers with highly relevant findings in their synthesis (Booth, 2001). There are a number of different approaches to analyzing findings from original studies. Noyes et al. (2008), in the Cochrane Handbook of Reviews of Effectiveness, suggest that syntheses of evidence from qualitative research: explore questions such as how do people experience illness, why does an intervention work (or not), for whom and in what circumstances. . . what are the barriers and facilitators to accessing health care, or what impact do specific barriers and facilitators have on people, their experiences and behavior?. . .[QES can] aid understanding of the way in which an intervention is experienced by all of those involved in developing, delivering or receiving it; what aspect of the intervention they value, or not; and why this is so. . ..[QES] can provide insight into factors that are external to an intervention including, for example, the impact of other policy developments, factors which facilitate or hinder successful implementation of a program, service or treatment and how a particular intervention may need to be adapted for large-scale roll-out. This suggests that authors may also vary in the way context is addressed. In a multi-context, QES exhaustive searches are used that retrieve studies (preferable published and unpublished) from every potential geographical, socio-cultural, political, historical, economical, health care, linguistic, or other context relevant to the review, with the aim to either aggregate or interpret findings from these studies to provide a broad international overview of Copyright © 2012 John Wiley & Sons, Ltd.



the phenomenon of interest practitioners, and policy makers could draw from for supporting their decisionmaking processes. For some qualitative researchers this may appear to be a very unnatural strategy, because of their particular focus on local context as part of their qualitative enquiry. A context-specific qualitative evidence synthesis would allow researchers to focus on one particular region, setting, socio-economic, political, or healthcare system. The key features of this particular type of QES, then, are: the use of a selective search with a focus on studies (both published and unpublished) that address a similar geographical, socio-cultural, political, historical, economical, health care, linguistic, or other context relevant to the review and as such provide lines of actions or theories that are sensitive to a particular setting. They may or may not take a national or local perspective and provide advice and suggestions that are highly relevant to some stakeholders, however, most likely less transferable to other settings. Some notable differences between a multi-context and a context-specific QES approach are described in Table 1. The main advantages of using a context-specific type of synthesis include a better access to local T1 sources of information and the well-defined niche of end users, which may increase the relevance of the findings for policy and practice and may induce a higher level of acceptance in end users. It also allows reviewers to fully consider particular features of a local system. Professionals in different countries may differ in ethics or perception, which generally results in different lines of action for practice, even if they are based on the same research evidence. The use of electric shocks to punish juveniles may have been proven to be efficient. It would, however, not be an acceptable strategy to many European stakeholders and would, therefore, not appear in their set of suggested strategies. Such discussions are quite common on a policy level, or even on a meso-level: scientific institutes, consumer organizations, enterprises, universities, small-scale formal and informal communication networks, and so on. Studies on the use of dummies, for example, have indicated that it decreases the risk of the sudden death syndrome. At the same time, it may have a negative impact on breast feeding (Ullah and Griffiths, 2003; Callaghan et al., 2005; Hauck et al., 2005). Conclusions to Q3 be generated from this evidence will most likely vary, depending on whether or not a particular country or its healthcare practitioners are strong proponents of breast feeding or not. It is, therefore, difficult to impose particular recommendations, advice, or implications for practice in a QES, without having a particular end user in mind. This proves to be an advantage for context-specific QES. In addition, they would most likely succeed in maintaining the integrity of the individual studies included. Context-specific review on obstacles to implementing evidence-based practice in the Belgian healthcare system The title of this review reflects the importance of considering a local context when trying to promote evidencebased practice (EBP). It was our belief that in order to provide policy-relevant advice to a particular national context, a context-specific synthesis, focusing on what is going on in the Belgian healthcare system, would be Table 1. Strengths and weaknesses of multi-context and context-specific approaches. Multi-context Qualitative Evidence Synthesis Key Features

• Exhaustive search • Little access to or knowledge of local

Strengths

• Wide-ranging in scope • Ability to cross-compare different

databases and experts

settings

• Works very well for topics where little • Weaknesses

• •

heterogeneity between different settings is expected Findings more likely transferable to a broad range of settings Targets a broad audience, but no one in particular Findings may be too general

• Risk of downplaying important local • •

Context-specific Qualitative Evidence Synthesis

• Selective search • Access to and knowledge of local databases and experts

• Targeted audience • Highly relevant practice and policy advice • Maintains integrity with context reported in original studies

• Findings may induce a higher level of acceptance in the end users

• Only relevant to the ‘happy few’ • Findings less likely transferable to other settings

characteristics by focusing on the search for commonalities Context may get lost in the overall picture Potential low level of acceptance in end users


3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65



4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

the better option. The purpose of this QES was to identify obstacles experienced or perceived by different groups of healthcare practitioners: general practitioners, dentists, psychiatrists, physiotherapists, and nurses professionally active in the Belgian healthcare system. These findings could then be used by Belgian policy makers to develop targeted lines of action or strategies to bridge the identified obstacles. A specific and focused search of the literature was conducted after the synthesis had been completed, which revealed information on how other countries had dealt with similar or related problems. This is a generally accepted strategy in some of the more inductive qualitative research designs. We deliberately adopted the ‘ignorance is bliss’ attitude as promoted by the founders of grounded theory (Corbin and Strauss, 1998) that enabled us to look at our data with an Q4 open mind. Search strategy and inclusion criteria. To ensure that all local (read: articles addressing the implementation of EBP in the Belgian healthcare context) would be included in the context-specific synthesis, we searched major databases such as Medline, Cinahl, Psychinfo, Embase, Social Sciences abstracts and ERIC (1990–May 2008), as well as the ‘Federal research actions’—database from the Belgian governmental Department of Science. In addition, we consulted Belgian experts in qualitative research methods and/or EBP to check on any other published papers, manuscripts published ahead of print, or articles in press. The study type we searched for were qualitative, empirical research papers. Opinion pieces and descriptive articles were excluded. We used keywords associated with the broad topic of ‘Evidence-Based Practice’ and combined them with the geographical notion ‘Belgian’, ‘Flemish/Flanders’ or ‘Walloon’. In addition, and where possible, we used a methodological filter on qualitative research. The QES eventually included eight articles reporting on perspectives from different target groups working in the Belgian health care and political context (Van Driel et al. 2003; Hannes et al., 2005; Heymans et al., 2006; Autrique et al., 2007; Hannes et al., 2007; Hannes et al., 2008; Hannes et al., 2009; Hannes et al., 2010). Critical appraisal of studies and study findings. The QES did not include a systematic assessment of the methodological quality of the original studies. All studies that met the inclusion criteria were included. We did, however, conduct a quality assessment on the level of the findings reported in each of the included studies, by using three ‘levels of evidence’ adapted from the Joanna Briggs Institute—approach to QES (Pearson et al., 2004). During the data extraction phase, we actively searched for interview excerpts or citations that could support statements made by authors. Statements that were supported with directly reported evidence (read: interview excerpts or citations) beyond reasonable doubt were labeled as ‘unequivocal’ evidence. Statements backed up with citations that were open to challenge or interpretation were displayed as ‘credible’ evidence. Statements without any supporting information from the raw data obtained the label ‘unsupported’ evidence. All statements were included in the QES, however, it was indicated that ‘unsupported’ statements should be read and interpreted with caution. Qualitative data analysis. We opted for the Joanna Briggs meta-aggregative approach to QES. This synthesis technique is designed to model the Cochrane Collaboration’s process of systematic reviews summarizing results of quantitative studies, while being sensitive to the nature of qualitative research and its traditions (Pearson et al., 2004). The procedure of meta-aggregation involves assembling the findings of studies (variously reported as themes, metaphors, and categories) and pooling them through further aggregation based on similarity in meaning to arrive at a set of synthesized statements presented as lines of action for practice and policy. The methodology is particularly interesting because it is ‘the only one that has explicitly aligned itself with the philosophy of pragmatism, making a case for the production of lines of actions (close to recommendations) that should be considered by end users confronted with a particular problematic situation’ (Hannes and Lockwood, 2011). It directly led to the production of guidance on how to potentially deal with the barriers inventoried. Data-analysis was performed using the ‘Qualitative assessment and review instrument’ software. One researcher extracted and analyzed the findings. The authors of the original studies included in the synthesis were then contacted to establish their response to the researcher’s interpretation of the papers. All lead authors responded with constructive comments to enhance the QES: codes, themes, or paragraphs that could be rephrased to better match the interpretation of the individual authors or codes that were not picked up by the reviewer and might be important to consider. It is not our aim to present the results of the analysis, because these have been published elsewhere (Hannes et al., 2012). Rather, we will outline what we have learned from this review and how it has strengthened us in the idea that context-specific QES are a worthwhile alternative to the multi-context type of reviews. Lessons learned from the conduct of a context-specific QES After having conducted the synthesis, we felt that our interpretation of the findings could be improved by discussing them in a broader international context. In matching the findings of our finalized QES with existing international literature, it was noticed that some of the obstacles mentioned by Belgian practitioners were universal and had been mentioned by practitioners in other countries as well, including time constraints and inadequate facilities such as limited access to information resources, lack of relevant evidence, and contradictions Copyright © 2012 John Wiley & Sons, Ltd.



in scientific literature (Khoja and Al-Ansary, 2007; Grimmer-Somers et al., 2007; Rabe et al., 2007; Zaidi et al., 2007; Parrilla-Castella et al., 2008; Sharek et al., 2008). Some obstacles related to the unique identity of certain disciplines. Q5 Q6 Q7 Dentists, for example, mentioned the technical skills in applying certain evidence-based techniques that influence patients’ outcomes (Hannes et al., 2008). Medical doctors stressed the lack of incentives to act evidence-based, mainly because acts such as searching, appraising, and translating evidence to particular patients were not financially rewarded (Hannes et al., 2005). Nurses reported on their lack of autonomy to decide on best treatments for patients (Hannes et al., 2007), and psychiatrists emphasized the impact of the therapeutic relationship on the outcome of a patient (Hannes et al., 2010). However, we also seemed to have identified barriers that related to typical features of the Belgian healthcare system. These features included, for example, the Belgian fee-for-service system and the dominant position of medical doctors in the Belgian healthcare system. In the Belgian fee-for-service system, practitioners are financed for clinical acts included in the Belgian nomenclature: a list of (para)medical interventions established by law in a Royal Decree, based on which a practitioner’s fee, the medical reimbursement to the patient, and the nonrefundable part of medical expenses are defined. Our synthesis revealed that the often time-intensive face-to-face explanation of diagnoses, treatments or procedures to patients, or the searches for evidence-based information are not included as a clinical act. As a consequence, they are often neglected because they do not lead to a financial compensation for the practitioner (Van Driel et al., 2003; Hannes et al., 2005). The blended approach promoted by Denmark (American college of physicians, 2008), in which practitioners are paid to provide care coordination and case management— including advice through devices such as telephones and e-mail gave us something to think about in trying to bridge this particular obstacle and evaluate whether putting a blended approach into practice would lead to better patient outcomes. It might also shift the focus from EBP seen as a time-consuming activity to a more rewarding activity. Secondly, it was stressed that the hierarchical position of physicians in the Belgian healthcare system hinders allied health professionals to put EBP into practice. A striking example is the so-called ‘old-fashioned’ referral system from medicals to which physiotherapists are bound in the treatment of their patient (Hannes et al., 2009). Although the evaluation of direct access systems to physiotherapists has been promising (Mitchell and De Lissovoy, 1997; Holdsworth and Webster, 2004), Belgian politicians have not yet considered it as a valuable option for Belgian practice. Several allied healthcare disciplines have mentioned the dominance from physicians in important advisory boards, protecting their own interests and preventing others from having their evidencebased opinion heard (Hannes et al., 2008; Hannes et al., 2009). Again, it would be worthwhile for Belgian policy makers to explore these direct access systems. Putting such a system in place would most likely increase the autonomy of allied health professionals and the cooperation between practitioners with different domains of expertise. There were several advantages in matching perspectives from healthcare practitioners working in the Belgian healthcare system with those from practitioners working in other healthcare systems. It enabled us to look at solutions proposed by other countries to solve particular problems, and it highlighted pitfalls in the Belgian healthcare system that local policy makers could focus on to bridge the obstacles perceived by practitioners. We doubt that a QES with an international focus would, for example, have recommended a consideration of direct access systems to physiotherapy, because it has already been put in place in the majority of Western countries. Also, physiotherapists from other countries would most likely not refer to ‘old fashioned’ doctor-referrals, because they have the autonomy to decide on the best possible treatment for their patients themselves. Unless the reviewers of a multi-context synthesis would have been very sensitive to deviant cases in their analysis, this aspect would easily have been ‘overlooked’ or put aside as irrelevant for the final report, because most of these reports address commonalities rather than differences between individual studies. What needs to be changed in Belgium most certainly differs from what needs to be changed elsewhere. Despite their context-specific nature, many of the findings of our study are likely transferable to other healthcare disciplines, for example, specialists and pharmacists working in the Belgian healthcare context. Obstacles that do not particularly link to the specific characteristics of the Belgian healthcare system, such as the time and access issues, probably hold true for practitioners in other countries as well. However, transferability of context-specific findings should be considered with caution. The transferability of results from, for example, guidelines or quality indicators conceptualized elsewhere has already been questioned by other authors (Marshall et al., 2003; Heymans et al., 2006). Important lessons were also drawn from the search strategy. One of the eight papers included in the synthesis was found as a result of contacts with local experts, and two were retrieved from databases owned by local government or professional organizations. These three publications would probably not have been found by a non-native reviewer. All three were in Dutch and/or French and potentially difficult to interpret by a non-native speaker. We have not systematically evaluated whether adding these three studies would have significantly changed our findings. However, they have strongly confirmed some of the problems related to specific characteristics of the Belgian healthcare system. It is the sensitivity to local databases and experts that makes a context-specific synthesis the more comprehensive one, because the more broadly scoped multi-context syntheses risk to miss out on many important studies or insights, therefore unintentionally stripping out the contextual richness of local, individual studies that may be relevant (Estabrooks et al., 1994). There is, however, good value in this particular variant as well. Multi-context QES can be an excellent choice for topics of interest Copyright © 2012 John Wiley & Sons, Ltd.


5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65


6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

where little heterogeneity between different settings is expected. They also assist in building a cumulative knowledge base and thereby provide support for policy and decision-making (Pope et al., 2007). There is reason to believe that the best of both approaches could be combined.

Integrating the best of both synthesis approaches Context-specific QES that highlight particular features of a well targeted setting and provide suggestions or recommendations to local practitioners and policy makers serve well as a basis for an umbrella review with a more international focus, in which insights generated in different settings or countries could be summarized. The concept of umbrella reviews has been introduced by the Cochrane Collaboration to summarize insights of multiple Cochrane Intervention reviews addressing the effects of two or more potential interventions for a single condition or health problem (Becker and Oxman, 2008). Russell and Kiddoo (2006) conducted one of the first umbrella reviews, summarizing insights generated from the comparison of different interventions for the treatment of nocturnal enuresis. The idea of introducing umbrella reviews or ‘overviews of reviews’ would be transferable to the field of QES. However, their focus would be slightly different. QES umbrella reviews would not be targeted toward evaluating different options in answering a particular question. Rather, they would summarize findings from context-specific QES on a particular phenomenon of interest and serve as a basis for international collaboration in the solution of key problems that cut across the different QES. Unlike many multicontext reviews, these umbrella reviews would still be sensitive to social–cultural and political features of local settings, or at least would have the full local ingredients to draw conclusions from. They would enable reviewers to include information on practice circumstances, values, preferences, and trade-offs. Like their quantitative variant, they could direct the reader to individual reviews of interest for additional detail. A second line of thought on ways forward with QES is the development of guidance on how to efficiently translate insights from a multi-context QES to suggestions and lines of action that are acceptable, feasible, and applicable in a given context or a particular local environment. The level of adaptation to a local context may vary in intensity, depending on, for example, available resources, services and interventions, acceptability to the public, differences in cultural and organizational features between the context of the articles included in the synthesis, and the targeted context. Most likely, it will involve trans-cultural modification (translation of language) and trans-contextual adaptation (Fervers et al., 2006). The latter is meant to modify suggestions in such a way that they become more relevant to the context in which they will be implemented. Transcontextual adaptation will make sure that suggestions generated from a QES and imposed on the reader will reply to the needs and policies of the targeted settings. Even when evolving from a multi-context to a more context-specific translation of a QES, any recommendation, advice, line of action, or implication for practice should be based on evidence directly applicable to a setting. In extrapolating findings from multi-context reviews, reviewers need to be aware of unique cultural factors, provider, or organizational factors that characterize a particular local context or setting. A detailed audit trail of the factors considered, the analyses performed and the procedure followed to evolve from findings to conclusions, allows the reader to decide whether he or she shares the same values and would come to the same conclusion for his or her environment. We see value in both pathways. Qualitative umbrella reviews would do a good job in keeping the context in to increase relevance for end users. For projects where the main aim is to build conceptual models or to generate theory to be used to underpin of provide a solid scientific base for certain local programs or interventions, the translation of a multi-context type of synthesis would work very well.

Conclusion The context-specific QES we conducted has led to the important conclusion that in order to be relevant, implications for practice, advice or lines of action generated from qualitative evidence should be tailored to a particular local context or setting, in our case to specific features of the Belgian healthcare system. Advantages of context-specific QES compared with multi-context QES include the easy access to local networks, maintaining integrity with contextual factors, and improving the acceptability and applicability of suggestions in a particular local setting. Two strategies to combine the best of both approaches are suggested, including the development of umbrella reviews covering QES conducted in different settings and the trans-contextual translation of multicontext QES in a product that is relevant to a local setting. Both strategies are defendable and in need of a set of guidelines on how to conduct them.

Conflict of Interest The authors declare that they have no conflicts of interest. Copyright © 2012 John Wiley & Sons, Ltd.



REFERENCES American College of Physicians. 2008. Achieving a high-performance health care system with universal access: what the United States can learn from other countries. Annals of Internal Medicine 148: 55–75. Autrique M, Vanderplasschen W, Thierry HP, Broekaert E, Sabbe B. 2007. Evidence-Based werken in de verslavingszorg: een stand van zaken. Academia Press, Ghent. Barroso J, Powell-Cope GM. 2000. Metasynthesis of qualitative research on living with HIV infection. Qualitative Health Research 10: 340–354. Becker LA, Oxman AD. 2008. Chapter 22: Overviews of reviews. In: Higgins JPT, Green S (eds), Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 The Cochrane Collaboration, Retrieved from www.cochranehandbook.org [Accessed January 8 2011]. Booth A. 2001. Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research? Qualitative Evidence-Based Practice Conference, Taking a critical stance (May 14–16). Coventry University, UK. Callaghan A, Kendall G, Lock C, Mahony A, Payne J, Varrier L. 2005. Association between pacifier use and breastfeeding, sudden infant death syndrome, infection and dental malocclusion. International Journal of Evidence-Based Healthcare 3: 147–167. Chen HT. 2005. Practical Program Evaluation: Assessing and Improving Planning, Implementation and Effectiveness. Sage Publications Inc, Thousand Oaks. Cochrane Qualitative Research Methods Group. 2009. Tools to assist systematic reviewers. Retrieved from http://www.joannabriggs.edu.au/cqrmg/tools.html [Accessed December 12 2010]. Dixon-Woods M, Booth A, Sutton AJ. 2007. Synthesizing qualitative research: a review of published reports. Qualitative Research 7: pp. 375. Retrieved from http://qrj.sagepub.com/cgi/reprint/7/3/375.pdf [Accessed 2 May 2010] . Estabrooks CA, Field PA, Morse JM. 1994. Aggregating qualitative findings: an approach to theory development. Qualitative Health Research 4: 503–511. Fervers B, Burgers JS, Haugh MC, Latreille J, Mlika-Cabanne N, Paquet L, Coulombe M, Poirier M, Burnand B. 2006. Adaptation of clinical guidelines: literature review and proposition for a framework and procedure. International Journal for Quality in Health Care 18: 167–176. Greenhalgh T, Kristjansson E, Robinson V. 2007. Realist review to understand the efficacy of school feeding programmes. British Medical Journal 27: 858–861. Grimmer-Somers K, Lekkas P, Nyland L, Young A, Kumar S. 2007. Perspectives on research evidence and clinical practice: a survey of Australian physiotherapists. Physiotherapy Research International 12: 147–161. Hannes K, Aertgeerts B, Goedhuys J. 2012. Obstacles to implementing Evidence-Based Practice in Belgium: a context-specific qualitative evidence synthesis including findings from different health care disciplines. Acta Clinica Belgica: In Press. Hannes K, Leys M, Buntinx F, Vermeire E, Aertgeerts B, Depoorter AM. 2005. Implementing Evidence-Based Medicine in general practice: a focus group based study. BMC Family Practice 6: 1–13. Hannes K, Lockwood C. 2011. Pragmatism as the philosophical underpinning of the Joanna Briggs metaaggregative approach to qualitative evidence synthesis. Journal of Advanced Nursing 67: 1632–1642. Hannes K, Macaitis K. 2012. A move to more transparent and systematic approaches of qualitative evidence synthesis: update of a review on published papers. Qualitative Research: In Press. Hannes K, Norré D, Goedhuys J, Naert I, Aertgeerts B. 2008. Barriers to the implementation of Evidence-Based Dentistry: a focus group based study. Journal of Dental Education 72: 736–744. Hannes K, Pieters G, Simons W, Hermans G, Goedhuys J, Aertgeerts B. 2010. Implementing Evidence-Based Practice in psychiatry: a focus group based study. Community Mental Health Journal 46: 423–432. Hannes K, Staes F, Vangesselen S, Goedhuys J, Aertgeerts B. 2009. Implementing Evidence-Based Physiotherapy: a focus group based study. Physiotherapy Theory and Practice 25: 476–488. Hannes K, Vandersmissen J, De Blaeser L, Goedhuys J, Peeters G, Schepers R, Aertgeerts B. 2007. Barriers to Evidence-Based Nursing: a focus group study. Journal of Advanced Nursing: 162–171. Hauck FR, Omojokun OO, Siadaty MS. 2005. Do pacifiers reduce the risk of sudden infant death syndrome? A metaanalysis. Pediatrics 116: 716–723. Heymans I, Van Linden A, Mambourg F, Leys M. 2006. Feedback: onderzoek naar de impact en barrières bij implementatie – Onderzoeksrapport: deel II. Federal Knowledge Centre for Health Care reports 32A: 49–61. Holdsworth LK, Webster VS. 2004. Direct access to physiotherapy in primary care: now?-and into the future? Physiotherapy 90: 64–72. Kane GA, Wood VA, Barlow J. 2007. Parenting programmes: a systematic review and synthesis of qualitative research. Child: Care, Health and Development 33: 784–793. Khoja TA, Al-Ansary LA. 2007. Attitudes to evidence-based medicine of primary care physicians in Asir region, Saudi Arabia. East Mediterranean Health Journal 13: 408–419. Marshall MN, Shekelle PG, McGlynn EA, Campbell S, Brook RH, Roland MO. 2003. Can health quality indicators be transferred between countries? Quality & Safety in Health Care 12: 8–12. Marston C, King E. 2006. Factors that shape young people’s sexual behaviour: a systematic review. Lancet 4: 1581–1586. Copyright © 2012 John Wiley & Sons, Ltd.


Q8

Q9

Q10

Q11

7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65


8

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

Mitchell JM, De Lissovoy G. 1997. A comparison of resource use and cost in direct access versus physician referral episodes of physical therapy. Physical Therapy 77: 11–18. Noyes J, Popay J. 2007. Directly observed therapy and tuberculosis: how can a systematic review of qualitative Q12 research contribute to improving services? A qualitative meta-synthesis. Journal of Advanced Nursing 57: 227–243. Noyes J, Popay J, Pearson A, Hannes K, Booth A, on behalf of the Cochrane Qualitative Research Methods Group. 2008. Chapter 20: qualitative research and Cochrane reviews. In: Higgins JPT, Green S (eds.), Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 The Cochrane Collaboration, Retrieved from www.cochrane-handbook.org [Accessed January 8 2011]. Parrilla-Castella ER, Almeyda R, Nogales E, Vélez M, Ramos M, Rivera JE, Dá Vila B, Torres V, Capriles J, Adamsons K. 2008. Evidence-based medicine as a tool for clinical decision-making in Puerto Rico. Puerto Rico Health Science Journal 27: 135–140. Pearson A, Laschinger H, Porritt K, Jordan Z, Tucker D, Long L. 2004. A comprehensive systematic review of evidence on developing and sustaining nursing leadership that fosters a healthy work environment in health care. Health care Reports 2: 146–208. Pope C, Mays N, Popey J. 2007. Synthesizing qualitative and quantitative health evidence: a guide to methods. McGraw-Hill Education, Berkshire. Rabe P, Holmén A, Sjögren P. 2007. Attitudes, awareness and perceptions on evidence based dentistry and scientific publications among dental professionals in the county of Halland, Sweden: a questionnaire survey. Swedish Dental Journal 31: 113–120. Russell K, Kiddoo D. 2006. The Cochrane library and nocturnal enuresis; an umbrella review. Evidence-Based Child Health 1: 5–8. Sharek PJ, Mullican C, Lavanderos A, Palmer C, Snow V, Kmetik K, Antman M, Knutson D, Dembry LM. 2007. Best Q13 practice implementation: lessons learned from 20 partnerships. Joint Commission Journal on Quality and Patient Safety 33: 16–26. Corbin JM, Strauss AL. 1998. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage Publications Inc, Thousand Oaks. Ullah S, Griffiths P. 2003. Does the use of pacifiers shorten breastfeeding duration in infants? British Journal of Community Nursing 8: 458–463. Van Driel M, Provoost S, Van Paepegem T, De Meyere M. 2003. Op wetenschappelijke evidentie gebaseerde zorg: van Q14 theorie naar praktijk: een tweevoudige strategie. Academia Press, Ghent. Vos R, Houtepen R, Horstman K. 2000. Evidence-based medicine and power shifts in health care systems. Health Care Analysis 10: 319–328. Zaidi Z, Hashim J, Iqbal M, Quadri KM. 2007. Paving the way for evidence-based medicine in Pakistan. Journal of the Pakistanian Medical Association 57: 556–560. Zief SG, Lauver S, Maynard RA. 2006. Impacts of after-school programs on student outcomes. Campbell Systematic Reviews 3. doi: 10.4073/csr.2006.3



Author Query Form Journal: Research Synthesis Methods Article: jrsm_55 Dear Author, During the copyediting of your paper, the following queries arose. Please respond to these by annotating your proofs with the necessary changes/additions. • If you intend to annotate your proof electronically, please refer to the E-annotation guidelines. • If you intend to annotate your proof by means of hard-copy mark-up, please refer to the proof mark-up symbols guidelines. If manually writing corrections on your proof and returning it by fax, do not write too close to the edge of the paper. Please remember that illegible mark-ups may delay publication. Whether you opt for hard-copy or electronic annotation of your proofs, we recommend that you provide additional clarification of answers to queries by entering your answers on the query sheet, in addition to the text mark-up.

Query No.

Query

Q1

AUTHOR: “Cochrane Qualitative Research Methods Group, 2011” is cited in text but not given in the reference list. Please provide details in the list or delete the citation from the text.

Q2

AUTHOR: “Noyes and Popay, 2006” is cited in text but not given in the reference list. Please provide details in the list or delete the citation from the text.

Q3

AUTHOR: The citation “Haugh et al., 2005” (original) has been changed to “Hauck et al., 2005”. Please check if appropriate.

Q4

AUTHOR: The citation “Straus and Corbin, 1998” (original) has been changed to “Corbin and Strauss, 1998”. Please check if appropriate.

Q5

AUTHOR: The citation “Parrilla-Castellar et al., 2008” (original) has been changed to “Parrilla-Castella et al., 2008”. Please check if appropriate.

Q6

AUTHOR: “Sharek et al., 2008” is cited in text but not given in the reference list. Please provide details in the list or delete the citation from the text.

Q7

AUTHOR: The meaning of this sentence ‘Some obstacles related to the unique identity of certain disciplines.’ is not clear; please rewrite or confirm that the sentence iscorrect.

Q8

AUTHOR: “Cochrane Qualitative Research Methods Group, 2009” is not cited in the text. Please indicate where it should be cited; or delete from the reference list.

Q9

AUTHOR: If this reference has now been published online, please add relevant year/DOI information. If this reference has now been published in print, please add relevant volume/issue/page/year information.

Remark

Query No.

Query

Q10


Q11


Q12

AUTHOR: “Noyes and Popay, 2007” is not cited in the text. Please indicate where it should be cited; or delete from the reference list.

Q13

AUTHOR: “Sharek et al., 2007” is not cited in the text. Please indicate where it should be cited; or delete from the reference list.

Q14

AUTHOR: Please provide the year of publication for Reference Van Driel.

Remark

USING e-ANNOTATION TOOLS FOR ELECTRONIC PROOF CORRECTION Required software to e-Annotate PDFs: Adobe Acrobat Professional or Adobe Reader (version 8.0 or above). (Note that this document uses screenshots from Adobe Reader X) The latest version of Acrobat Reader can be downloaded for free at: http://get.adobe.com/reader/ Once you have Acrobat Reader open on your computer, click on the Comment tab at the right of the toolbar:

This will open up a panel down the right side of the document. The majority of tools you will use for annotating your proof will be in the Annotations section, pictured opposite. We’ve picked out some of these tools below:

1. Replace (Ins) Tool – for replacing text.

2. Strikethrough (Del) Tool – for deleting text.

Strikes a line through text and opens up a text box where replacement text can be entered. How to use it

Strikes a red line through text that is to be deleted. How to use it



Highlight a word or sentence.



Highlight a word or sentence.



Click on the Replace (Ins) icon in the Annotations section.



Click on the Strikethrough (Del) icon in the Annotations section.



Type the replacement text into the blue box that appears.

3. Add note to text Tool – for highlighting a section to be changed to bold or italic.

4. Add sticky note Tool – for making notes at specific points in the text.

Highlights text in yellow and opens up a text box where comments can be entered. How to use it

Marks a point in the proof where a comment needs to be highlighted. How to use it



Highlight the relevant section of text.





Click on the Add note to text icon in the Annotations section.

Click on the Add sticky note icon in the Annotations section.



Click at the point in the proof where the comment should be inserted.



Type the comment into the yellow box that appears.



Type instruction on what should be changed regarding the text into the yellow box that appears.

USING e-ANNOTATION TOOLS FOR ELECTRONIC PROOF CORRECTION

5. Attach File Tool – for inserting large amounts of text or replacement figures.

6. Add stamp Tool – for approving a proof if no corrections are required.

Inserts an icon linking to the attached file in the appropriate pace in the text. How to use it

Inserts a selected stamp onto an appropriate place in the proof. How to use it



Click on the Attach File icon in the Annotations section.



Click on the Add stamp icon in the Annotations section.



Click on the proof to where you’d like the attached file to be linked.





Select the file to be attached from your computer or network.

Select the stamp you want to use. (The Approved stamp is usually available directly in the menu that appears).



Click on the proof where you’d like the stamp to appear. (Where a proof is to be approved as it is, this would normally be on the first page).



Select the colour and type of icon that will appear in the proof. Click OK.

7. Drawing Markups Tools – for drawing shapes, lines and freeform annotations on proofs and commenting on these marks. Allows shapes, lines and freeform annotations to be drawn on proofs and for comment to be made on these marks..

How to use it 

Click on one of the shapes in the Drawing Markups section.



Click on the proof at the relevant point and draw the selected shape with the cursor.



To add a comment to the drawn shape, move the cursor over the shape until an arrowhead appears.



Double click on the shape and type any text in the red box that appears.

For further information on how to annotate proofs, click on the Help menu to reveal a list of further options:

WILEY AUTHOR DISCOUNT CLUB We would like to show our appreciation to you, a highly valued contributor to Wiley’s publications, by offering a unique 25% discount off the published price of any of our books*.

O

O

The Database Group (Author Club) John Wiley & Sons Ltd The Atrium Southern Gate Chichester PO19 8SQ UK Alternatively, you can register online at www.wileyeurope.com/go/authordiscount Please pass on details of this offer to any co-authors or fellow contributors.

FS

All you need to do is apply for the Wiley Author Discount Card by completing the attached form and returning it to us at the following address:

PR

After registering you will receive your Wiley Author Discount Card with a special promotion code, which you will need to quote whenever you order books direct from us. The quickest way to order your books from us is via our European website at:

EC

TE

Key benefits to using the site and ordering online include: Real-time SECURE on-line ordering Easy catalogue browsing Dedicated Author resource centre Opportunity to sign up for subject-orientated e-mail alerts

D

http://www.wileyeurope.com

Alternatively, you can order direct through Customer Services at: [email protected], or call +44 (0)1243 843294, fax +44 (0)1243 843303

O R

R

So take advantage of this great offer and return your completed form today.

N

C

Yours sincerely,

U

Verity Leaver Group Marketing Manager [email protected]

*TERMS AND CONDITIONS This offer is exclusive to Wiley Authors, Editors, Contributors and Editorial Board Members in acquiring books for their personal use. There must be no resale through any channel. The offer is subject to stock availability and cannot be applied retrospectively. This entitlement cannot be used in conjunction with any other special offer. Wiley reserves the right to amend the terms of the offer at any time.

REGISTRATION FORM

For Wiley Author Club Discount Card

Accounting Public Corporate

[] [] []

Architecture

[]

Business/Management

[]

Chemistry Analytical Industrial/Safety Organic Inorganic Polymer Spectroscopy

[ [ [ [ [ [ [

] ] ] ] ] ] ]

Computer Science Database/Data Warehouse Internet Business Networking Programming/Software Development Object Technology

[ [ [ [ [

Encyclopedia/Reference Business/Finance Life Sciences Medical Sciences Physical Sciences Technology

[ [ [ [ [ [

] ] ] ] ] ]

Engineering Civil Communications Technology Electronic Environmental Industrial Mechanical

[ [ [ [ [ [ [

] ] ] ] ] ] ]

Earth & Environmental Science

[]

Hospitality

[]

Finance/Investing Economics Institutional Personal Finance

[ [ [ [

] ] ] ]

Genetics Bioinformatics/ Computational Biology Proteomics Genomics Gene Mapping Clinical Genetics

[] []

Life Science

[]

U

Non-Profit

] ] ] ] ] ] ] ] ]

[]

O

[]

O

PR

D

TE

EC R

] ] ] ]

O R [ [ [ [ [ [ [ [ [

C

N

Medical Science Cardiovascular Diabetes Endocrinology Imaging Obstetrics/Gynaecology Oncology Pharmacology Psychiatry

[ [ [ [

] ] ] ] ]

FS

To enjoy your 25% discount, tell us your areas of interest and you will receive relevant catalogues or leaflets from which to select your books. Please indicate your specific subject areas below.

Landscape Architecture

[]

Mathematics Statistics

[] []

Manufacturing [] Materials Science Psychology Clinical Forensic Social & Personality Health & Sport Cognitive Organizational Developmental & Special Ed Child Welfare Self-Help

[ [ [ [ [ [ [ [ [ [ [

] ] ] ] ] ] ] ] ] ] ]

Physics/Physical Science

[]

Please complete the next page /

I confirm that I am (*delete where not applicable): a Wiley Book Author/Editor/Contributor* of the following book(s): ISBN: ISBN: a Wiley Journal Editor/Contributor/Editorial Board Member* of the following journal(s):

SIGNATURE: ……………………………………………………………………………………

Date: ………………………………………

PLEASE COMPLETE THE FOLLOWING DETAILS IN BLOCK CAPITALS: TITLE: (e.g. Mr, Mrs, Dr) …………………… FULL NAME: …………………………………………………………………………….… ..…………………………………………………………………………………………………………………

FS

JOB TITLE (or Occupation):

DEPARTMENT: ……………………………………………………………………………………………………………………………………………..

O

COMPANY/INSTITUTION: ……………………………………………………………………………………………………………………………

O

ADDRESS: ……………………………………………………………………………………………………………………………………………………

PR

……………………………………………………………………………………………………………………………………………………………………… TOWN/CITY: ………………………………………………………………………………………………………………………………………………… COUNTY/STATE: ………………………………………………………………………………………………………………………………………….

D

COUNTRY: …………………………………………………………………………………………………………………………………………………….

TE

POSTCODE/ZIP CODE: ………………………………………………………………………………………………………………………………… DAYTIME TEL: ………………………………………………………………………………………………………………………………………………

EC

FAX: ………………………………………………………………………………………………………………………………………………………………

R

E-MAIL: …………………………………………………………………………………………………………………………………………………………

O R

YOUR PERSONAL DATA We, John Wiley & Sons Ltd, will use the information you have provided to fulfil your request. In addition, we would like to: Use your information to keep you informed by post of titles and offers of interest to you and available from us or other Wiley Group companies worldwide, and may supply your details to members of the Wiley Group for this purpose. [ ] Please tick the box if you do NOT wish to receive this information

2.

Share your information with other carefully selected companies so that they may contact you by post with details of titles and offers that may be of interest to you. [ ] Please tick the box if you do NOT wish to receive this information.

N

C

1.

U

E-MAIL ALERTING SERVICE We also offer an alerting service to our author base via e-mail, with regular special offers and competitions. If you DO wish to receive these, please opt in by ticking the box [ ]. If, at any time, you wish to stop receiving information, please contact the Database Group ([email protected]) at John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, PO19 8SQ, UK.

TERMS & CONDITIONS This offer is exclusive to Wiley Authors, Editors, Contributors and Editorial Board Members in acquiring books for their personal use. There should be no resale through any channel. The offer is subject to stock availability and may not be applied retrospectively. This entitlement cannot be used in conjunction with any other special offer. Wiley reserves the right to vary the terms of the offer at any time.

PLEASE RETURN THIS FORM TO: Database Group (Author Club), John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, PO19 8SQ, UK [email protected] Fax: +44 (0)1243 770154