Systematic reviews of health promotion and public health interventions

17 downloads 0 Views 5MB Size Report
Cochrane Health Promotion & Public Health Field. Overview. ○ Overview of systematic reviews. ○ Outline of The Cochrane Collaboration. ○ Role of the HPPH ...
Systematic reviews of health promotion and public health interventions

Rebecca Armstrong Elizabeth Waters Cochrane Health Promotion & Public Health Field

Overview  Overview of systematic reviews  Outline of The Cochrane Collaboration  Role of the HPPH Field  Function of systematic reviews in informing policy and practice  Key elements of systematic reviews Asking answerable questions Searching for evidence Assessing quality Synthesising results Applicability and transferability

1

Types of reviews Reviews (narrative/literature/ traditional)

Systematic reviews

Meta-analysis

Narrative reviews  Usually written by experts in the field  Use informal and subjective methods to collect and interpret information  Usually narrative summaries of the evidence

Read: Klassen et al. Guides for Reading and Interpreting Systematic Reviews. Arch Pediatr Adolesc Med 1998;152:700-704.

2

What is a systematic review? A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review*

*Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

Key elements of a systematic review Structured, systematic process involving several steps : 1. 2. 3. 4. 5. 6. 7.

Formulate the question Plan the review Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps described explicitly in the review

3

Systematic

vs.

 Scientific approach to a review article  Criteria determined at outset  Comprehensive search for relevant articles  Explicit methods of appraisal and synthesis  Meta-analysis may be used to combine data

Narrative reviews  Depend on authors’ inclination (bias)  Author gets to pick any criteria  Search any databases  Methods not usually specified  Vote count or narrative summary  Can’t replicate review

Advantages of systematic reviews  Reduce bias  Replicability  Resolve controversy between conflicting studies  Identify gaps in current research  Provide reliable basis for decision making

4

Limitations of systematic reviews specific to health promotion  Results may still be inconclusive  There may be no trials/evidence  The trials may be of poor quality  The intervention may be too complex to be tested by a trial  Practice does not change just because you have the evidence of effect/effectiveness

Consider these interventions… Interventions to promote smoke alarm ownership and function School-based driver education for the prevention of traffic crashes Helmets for preventing head and facial injuries in bicyclists Do you think the results identified in SRs will be good, promising or absent (and potentially harmful)?

5

Results from systematic reviews  Helmets reduce bicycle-related head and facial injuries for bicyclists of all ages involved in all types of crashes including those involving motor vehicles.  The results provide no evidence that drive education reduces road crash involvement, and suggest that it may lead to a modest but potentially important increase in the proportion of teenagers involved in traffic crashes.  Results from this review suggest that area-wide traffic calming in towns and cities may be a promising intervention for reducing the number of road traffic injuries and deaths. However, further rigorous evaluations of this intervention are needed.

The Cochrane Collaboration

International non-profit organisation that prepares, maintains, and disseminates systematic up-to-date reviews of health care interventions

6

Cochrane Collaboration Named in honour of Archie Cochrane, a British researcher In 1979: “It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials”

The Cochrane Library  Cochrane Systematic reviews : Cochrane reviews and protocols  Database of Reviews of Effects: Other systematic reviews appraised by the Centre for Reviews and Dissemination.  Cochrane Central Register of Controlled Trials: Bibliography of controlled trials (some not indexed in MEDLINE).  Health Technology Assessment Database: HTA reports  NHS Economic evaluation database: Economic evaluations of health care interventions.

7

The Cochrane Library www.thecochranelibrary.com

8

Cochrane HPPH Field  Represent the needs and interests of those in health promotion and public health in Cochrane matters  Represent Cochrane in health promotion and public health forums  In transition from Field to Review Group Will edit PH reviews for the Cochrane Library

Cochrane Collaboration Structure Steering Group Review Groups Centres Consumer Network

Fields Methods Groups

9

Collaborative Review Groups (50)  Produce systematic reviews relevant to a particular disease or health issue for inclusion in the Cochrane Library  Examples Airways Group Drug and Alcohol Group Heart Group Injuries Group Skin Group Pregnancy and Childbirth Group Stroke Group Breast Cancer Group

Methods Groups (12)  Provide advice and support in the development of the methods of systematic reviews  Examples Non-Randomised Studies Screening and Diagnostic Tests Empirical Methodological Studies Qualitative Methods

10

Cochrane Centres (14)  Work to assist all Cochrane entities within a specific geographical area  Examples Australasian Cochrane Centre (at Monash) South African Cochrane Centre Italian Cochrane Centre Chinese Cochrane Centre

Cochrane Fields/Networks (9)  Represent an area of interest which spans a number of health problems - and hence a number of Review Groups  Examples Health Promotion and Public Health Field Primary Health Care Field Cancer Network Child Health Field

11

Cochrane HPPH Field  Cochrane Fields represent a population group, or type of care that overlaps multiple Review Group areas  HPPH Field Registered in 1996 Administered from Melbourne Funded by VicHealth Over 400 members on contact database across >30 countries

Staff Elizabeth Waters (Director) Jodie Doyle (Coordinator) Rebecca Armstrong (Senior Research Fellow) Naomi Priest (Research Fellow)

12

Asking an answerable question

Questions of interest Effectiveness:  Does the intervention work/not work?  Who does it work/not work for? Other important questions:  How does the intervention work?  Is the intervention appropriate?  Is the intervention feasible?  Is the intervention and comparison relevant?

13

Answerable questions EFFECTIVENESS A description of the populations

P

An identified intervention

I

An explicit comparison

C

Relevant outcomes

O

A PICO question Time-consuming question: What is the best strategy to prevent smoking in young people?

14

An answerable question

Q. Are mass media (or school-based or community-based) interventions effective in preventing smoking in young people?

The PICO(T) chart Problem, population

Intervention

Comparison

Outcome

Types of studies

Young people under 25 years of age

a) Television b) Radio c) Newspapers d) Bill boards e) Posters f) Leaflets g) Booklets

a) School-based interventions b) No intervention

a) objective measures of smoking (saliva thiocyanate levels, alveolar CO) b) self-reported smoking behaviour c) Intermediate measures (intentions, attitude, knowledge, skills) d) Media reach

a) RCT b) Controlled before and after studies c) Time series designs

15

Finding the evidence

16

Systematic review process 1. 2. 3. 4. 5. 6.

Well formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Interpretation of results

A good search  Clear research question  Comprehensive search All domains, no language restriction, unpublished and published literature, up-to-date

 Document the search (replicability)

17

Components of electronic searching 1. Describe each PICO component 2. Start with primary concept 3. Find synonyms a) Identify MeSH / descriptors / subject headings b) Add textwords

4. Add other components of PICO question to

narrow citations (may use study filter) 5. Examine abstracts 6. Use search strategy in other databases (may need adapting)

So you want to do a ‘quick & dirty’?  DARE  CENTRAL  PubMed (clinical queries, related records)  CDC  NICE  Organisations who do work in your area  …google

18

The Cochrane Library: www.thecochranelibrary.com

Cochrane HPPH Field

19

Health-evidence.ca

The Guide to Community Preventive Services http://www.thecommunityguide.org/

20

National Institute for Health and Clinical Excellence http://www.publichealth.nice.org.uk/page.aspx?o=home

Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre)

http://eppi.ioe.ac.uk

21

Effective Public Health Practice Project (EPHPP) http://www.myhamilton.ca/myhamilton/CityandGo vernment/HealthandSocialServices/Research/E PHPP/

Centre for Reviews and Dissemination http://www.york.ac.uk/inst/crd

22

Other sources of primary research

Searching  www.google.com.au  The order of terms will effect the results so start with the obvious or key concept  No need for ‘and’  Google will ignore common words If they are important use + (e.g. policy + 3)

 Phrase searching is useful eg “suicide prevention”  Google searches for variations on words eg diet, dietary

23

Searching  Where terms have multiple meanings you can direct google to remove sites you want to avoid (e.g. bass –music)  Keep your search strings brief Mental health promotion initiatives to prevent suicide in young people

 Compartmentalise your search strings “mental health promotion” suicide “Suicide prevention” “young people” Prevention and suicide and youth

Searching  http://www.ncbi.nlm.nih.gov/entrez/query.fcgi  Use the same principles for google – keep it short and sweet.  Key features Journals Database MeSH Database Single Citation Matcher Clinical Queries

24

Searching

Searching

25

Searching

Select the Limits tab – just under the search string

Searching

26

 These next few slides show you how to search MeSH terms in PubMed. Useful if you don’t have access to electronic databases. You combine this method with the one for text words outlined above. The process for combining text words and MeSH terms is outlines below.

27

28

Example Mass media interventions to prevent smoking in young people P= Young people STEP ONE: Find MeSH and textwords to describe young people

Example Mass media interventions to prevent smoking in young people P= Young people MeSH: Adolescent Child Minors

29

Example Mass media interventions to prevent smoking in young people P= Young people Textwords: Adolescent Child Juvenile Young people Student

Girl Boy Teenager Young adult Youth

Textwords Truncation $: To pick up various forms of a word Teen$.tw Smok$.tw Teenage Smoke Teenager Smoking Teenagers Smokes Teens Smoker Teen Smokers

30

Textwords Wild cards ? and #: To pick up different spellings Colo?r.tw (? Can be substituted for one or no characters) Colour Color Wom#n.tw (# Substitutes for one character) Woman Women

Textwords Adjacent ADJn:  retrieves two or more query terms within n words of each other, and in any order  Great when you are not sure of phraseology Eg sport adj1 policy Sport policy Policy for sport Eg mental adj2 health Mental health Mental and physical health

31

Example continued Mass media interventions to prevent smoking in young people I = Mass media interventions

STEP TWO: Find MeSH and textwords to describe mass media interventions

Example continued  MeSH Mass media Audiovisual aids Television Motion pictures Radio Telecommunications Newspapers Videotape recording Advertising

32

Example continued Mass media interventions to prevent smoking in young people O = Prevention of smoking

STEP THREE: Find MeSH and textwords to describe prevention of smoking

Example of search P = YOUNG PEOPLE MeSH

Textwords

………………………. ………………………. ………………………. ………………………. ………………………. ……………………….

………………………. ………………………. ………………………. ………………………. ………………………. ……………………….

OR

P AND

I = MASS MEDIA MeSH ………………………. ………………………. ………………………. ………………………. ………………………. ……………………….

Textwords

OR

………………………. ………………………. ………………………. ………………………. ………………………. ……………………….

I AND C

C = (if required) O = PREVENTION OF SMOKING MeSH ………………………. ………………………. ………………………. ………………………. ………………………. ……………………….

OR

Textwords ………………………. ………………………. ………………………. ………………………. ………………………. ……………………….

AND O

33

Different bibliographic databases  Databases use different types of controlled vocabulary Same citations indexed differently on different databases Medline and EMBASE use a different indexing system for study type PsycINFO and ERIC do not have specific terms to identify study types Need to develop search strategy for each database

Study design filters  RCTs  See Cochrane Reviewer’s Handbook

 Non-RCTs  Not yet developed, research in progress

 Qualitative research  Specific subject headings used in CINAHL, ‘qualitative research’ used in Medline  CINAHL Filter: Edward Miner Library http://www.urmc.rochester.edu/hslt/miner/digital_library/tip_sheets/ Cinahl_eb_filters.pdf

 Systematic reviews/meta-analyses  CINAHL: as above  Medline http://www.urmc.rochester.edu/hslt/miner/digital_library/tip_sheets/ OVID_eb_filters.pdf  Medline and Embase http://www.sign.ac.uk/methodology/filters.html  PubMed

34

2. Unpublished literature  Not all known published trials are identifiable in Medline (depending on topic)  Only 25% of all medical journals in Medline  Non-English language articles are underrepresented in Medline (and developing countries)  Publication bias – tendency for investigators to submit manuscripts and of editors to accept them, based on strength and direction of results (Olsen 2001)

2. Unpublished literature  Hand searching of key journals and conference proceedings  Scanning bibliographies/reference lists of primary studies and reviews  Contacting individuals/agencies/ academic institutions Neglecting certain sources may result in reviews being biased

35

Librarians are your friends!

Principles of critical appraisal

36

Critical appraisal The process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision. Alison Hill, Critical Appraisal Skills Programme, Institute of Health Sciences, Oxford http://www.evidence-based-medicine.co.uk

Critical appraisal I: Quantitative studies

37

Why appraise validity?  Not all published and unpublished literature is of satisfactory methodological rigour Just because it is in a journal does not mean it is sound! Onus is on you to assess validity!

 Quality may be used as an explanation for differences in study results  Guide the interpretation of findings and aid in determining the strength of inferences

Bias – quality assessment tool 1. 2. 3. 4. 5. 6. 7. 8.

Selection bias Allocation bias Confounding Blinding (detection bias) Data collection methods Withdrawals and drop-outs Statistical analysis Intervention integrity

38

Selection bias

Recruit participants

Allocation of concealment Intervention

Allocation

Control

Confounding Exposed to intervention

Integrity of intervention

Not exposed to intervention

Follow-up

Intention-to-treat

Follow-up

Withdrawals Outcome

Analysis

Blinding of outcome assessment

Outcome

Data collection methods Statistical analysis

Analysis

Critical appraisal tools  RCTs The Quality Assessment Tool for Quantitative Studies (http://www.city.hamilton.on.ca/PHCS/EPHPP/).

 Non-RCTs Cochrane Effective Practice and Organisation of Care Group (http://www.epoc.uottawa.ca/). The Berkeley Systematic Reviews Group (http://www.medepi.net/meta/)

39

Critical appraisal II: Qualitative studies

Qualitative research  … explores the subjective world. It attempts to understand why people behave the way they do and what meaning experiences have for people.  Qualitative studies of experience  Process evaluation Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

40

CASP appraisal checklist 1. Clear aims of research (goals, why it is important, relevance) 2. Appropriate methodology (what, how, why) 3. Sampling strategy 4. Data collection 5. Relationship between researcher and participants 6. Ethical issues 7. Data analysis 8. Findings 9. Value of research (context dependent)

Other qualitative checklist  Quality framework Government Chief Social Researcher’s Office, UK http://www.strategy.gov.uk/files/pdf/Quality_framew ork.pdf

 19 question checklist for process evaluations (EPPI-Centre)

41

Appraisal of a systematic review  10 questions 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Clearly-focused question The right type of study included Identifying all relevant studies Assessment of quality of studies Reasonable to combine studies What were the results Preciseness of results Application of results to local population Consideration of all outcomes Policy or practice change as a result of evidence CASP

Interpretation of results

42

Factors influencing effectiveness  Type III error (integrity of intervention)  Theoretical framework of intervention  Context  For whom did the intervention work, why, in what circumstances, at what cost

Difficulties addressing inequalities  Studies rarely present information on differential effects of interventions  Cannot locate studies addressing inequalities  May need original data from authors  Low power to detect subgroup differences

43

Integration of evidence into practice

Assessing the applicability and transferability of interventions  Applicability – whether the intervention process could be implemented in the local setting, no matter what the outcome is. Is it possible to run this intervention in this local setting? Eg. provision of condoms in area where they are not acceptable for religious reasons

Wang et al 2005

44

Assessing the applicability and transferability of interventions  Transferability – if the intervention were to be implemented in the local setting, would the effectiveness of the program be similar to the level detected in the study setting? E.g. if the interventionists lack experience and have few skills in delivering the intervention then its effectiveness in the local setting may be lower than that demonstrated in the study setting Wang et al 2005

Review of questions proposed by Wang et al

45

Contact details  Rebecca Armstrong [email protected] 03 9667 1336 If I can’t help you I might be able to point you in the right direction. If you are interested in training and support for conducting SRs or increasing uptake within your organisation, region, state please let me know

46