Systematic Searching for Systematic Reviews: an ...

1 downloads 0 Views 5MB Size Report
Feb 7, 2018 - latrobe.edu.au. La Trobe University CRICOS Provider Code Number 00115M. Systematic Searching for Systematic Reviews: an introduction.
latrobe.edu.au

Systematic Searching for Systematic Reviews: an introduction Steven Chang – [email protected] La Trobe University Library

Wednesday 7th February 2018

La Trobe University CRICOS Provider Code Number 00115M

Overview  Difference between a systematic review and a traditional literature review  Transforming your research question into a rigorous search strategy  Choosing the right databases / sources to search for your topic area  Developing a robust and comprehensive search strategy  Translating searches across multiple databases and platforms  Becoming aware of key software to support your systematic review (and meta-analysis, if you’re doing one)

La Trobe University

2

In brief:

What’s your background?

Why are you interested in learning to do a systematic review?

La Trobe University

3

What is a systematic review?

“A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias…” - Cochrane Library

Cochrane Library (2017). About Cochrane Systematic Reviews and Protocols [online]. Available at: http://www.cochranelibrary.com/about/about-cochranesystematic-reviews.html [accessed 21/04/17].

La Trobe University

4

What is a systematic review?

https://www.youtube.com/watch?v=egJlW4vkb1Y La Trobe University

5

Study 1

Study 2

Study 3

Systematic review and meta-analysis La Trobe University

6

Case study – post-traumatic psychological interventions

 For traumatic events, psychological debriefing was standard practice until 2002  Cochrane Review in 2002 identified nine studies to evaluate the effectiveness of early single-session psychological debriefing for preventing post-traumatic stress disorder (PTSD)  Shows how systematic reviews can shine a light on the whole picture to change

standard practice

Rose, S. C., Bisson, J., Churchill, R., & Wessely, S. (2002). Psychological debriefing for preventing post traumatic stress disorder (PTSD). The Cochrane Library. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD000560/full

Rose, S., Bisson, J., & Wessely, S. (2003). A systematic review of single-session psychological

interventions (‘debriefing’) following trauma. Psychotherapy and psychosomatics, 72(4), 176184. http://www.karger.com/article/abstract/70781

La Trobe University

7

Why do we need systematic reviews?

 Managing information overload

 Building a comprehensive, trustworthy picture of a topic  Answering what is known and what is unknown  Enable informed decision-making by clinicians, policymakers, and researchers  Minimise the risk of duplicating research efforts and wasting research resources

La Trobe University

8

The hierarchy of evidence Systematic reviews, meta-analyses

RCTs

Quality

Cohort studies

Case-control studies

Case reports La Trobe University

9

The key stages Formulate a focused, answerable research question

Define scope and inclusion/exclusion criteria to be used

Systematically search for all relevant research (both published AND unpublished) Assess the quality and risk of bias for each study included in your review Synthesise included studies via meta-analysis (if possible) La Trobe University

10

How is a systematic review different from a traditional narrative review? Traditional/narrative review

Systematic review

Search strategy not explicitly stated

Literature gathered using explicit search methodology (must be replicable)

Studies may be “cherry-picked” by author

Explicit inclusion & exclusion criteria

Prone to selection bias

Minimises selection bias

No standard consensus on minimum items

Standard protocols i.e. PRISMA guidelines

May describe a broad overview of a topic

Focuses on a specific, structured question

Based on author’s subjectivity and expertise

Based on structured, scientific methodology

Traditional narrative reviews may be at high risk of bias. Systematic reviews aim to minimise this bias. La Trobe University

11

Systematic review vs “systematic literature review” 

A systematic review is a very specific type of study ̶

Uses particular methodology which must be explicitly reported/documented ̶

Follows standardised guidelines (PRISMA, PRISMA-P, etc) ̶

Included studies usually undergo assessment of quality and risk of bias ̶

It is NOT simply a literature search that is more systematic than usual



Are you doing an actual systematic review?



Or are you doing a "normal" literature review that you want to be systematic with?



These are two different things

La Trobe University

12

Consider other types of reviews 

Scoping review



Rapid review



Umbrella review/overview of reviews (systematic reviews of systematic reviews!)



And many more…!

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108.

La Trobe University

13

What you need to get started

 Time!  A team ̶

Subject experts for developing research question and screening records

̶

Librarian / information specialist for performing a systematic search ̶

Statistician / methodologist for undertaking meta-analysis

La Trobe University

15

What you need to get started

WARNING: Slow and deliberate science

 Systematic reviews are a hefty investment of time, labour, and effort! ̶

Average of five authors ̶

Mean of 67.3 weeks from start to finish (to be published) ̶

Average yield of relevant items per search: 2.94% (e.g. only 1 relevant study per 30 search results!)

Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ open, 7(2), e012545.

La Trobe University

16

Before you start – is a review actually needed?  There is a huge amount of duplication and research waste being done  Solution – check PROSPERO before you start your systematic review!  Don’t forget to register YOUR protocol when it’s completed

Moher, D., Booth, A., & Stewart, L. (2014). How to reduce unnecessary duplication: use PROSPERO. BJOG: An International Journal of Obstetrics & Gynaecology, 121(7), 784-786. Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012). The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Systematic Reviews, 1(1), 2.

La Trobe University

17

HANDS ON: searching PROSPERO (3 minutes)

Can you find existing systematic reviews on your topic, or related to your topic?

 Search PROSPERO Tip: Don’t overcomplicate it – just chuck in a few keywords https://www.crd.york.ac.uk/PROSPERO/

La Trobe University

18

Before you start – develop a plan (protocol): PRISMA-P! PRISMA-P checklist http://www.prisma-statement.org/documents/PRISMA-P-checklist.pdf

La Trobe University

19

Before you start – develop a plan (protocol): PRISMA-P!  PRISMA-P elaboration and explanation (this is your step-by-step guide!) ̶

http://www.bmj.com/content/349/bmj.g7647

La Trobe University

20

Formulating your research question

La Trobe University

21

Formulating your research question

 Use mnemonics to design a specific and answerable research question

 Quantitative research question? Go with PICO(S)

̶

C (comparison intervention) and O (outcomes) often omitted from search strategy, but included for developing inclusion/exclusion criteria

 Qualitative? Try SPICE, SPIDER, or others.

La Trobe University

22

Formulating your research question: PICO(S)

P

Patient / population

Patients with heart failure

I

Intervention

Telemedicine

Comparison intervention

Conventional clinical assessment and treatment

O

Outcome(s)

Functional outcome, as measured by modified Rankin Score (mRS)

S

Study type

Randomised controlled trials

C

How effective and safe is telemedicine for patients with heart failure, compared to conventional clinical assessment and treatment? La Trobe University

23

Formulating your research question: SPIDER (qualitative) S

Sample group

Pre-school, primary, and secondary students

P I

Phenomenon of Interest

Mindfulness programs

Design (specific methodology)

Quasi-experimental design (QED)

D

E

Evaluation

Socio-emotional outcomes Behavioural outcomes Academic outcomes

R

Research type (e.g. mixed methods, quant, or qual)

Mixed methods

Do mindfulness programs improve the academic, behavioural, and socioemotional functioning of primary and secondary students? La Trobe University

24

Formulating your research question: SPICE (qualitative) S

Setting (where?)

n/a (global – all countries)

Perspective (for who or what?)

Legumes

I

Intervention (phenomenon of interest)

Elevated CO2

C

Comparison

No elevation in CO2

E

Evaluation (outcome)

Seed mass Germination Seed vigour

P

What is the effect of climate change on the seed quality of legume crops? La Trobe University

25

Formulating your research question: PICo (qualitative)

La Trobe University

26

HANDS ON: Formulating your research question (3 minutes) 1. Describe your research question of interest in one or two sentences. 2. Break down your research question into PICO/SPIDER format



No hard and fast rules



It’s just a guide to help you conceptually



You can leave some parts blank! (usually the C and O of PICO…)

La Trobe University

27

Selecting sources to search

La Trobe University

28

Selecting sources to search – health sciences databases

 MEDLINE

 Cochrane Library

 CINAHL

 PsycINFO

 EMBASE

La Trobe University

29

Selecting sources to search – databases outside health sciences  Education: ERIC, AEI Informit (Australian)

 Sociology: Sociological Abstracts, Anthropology Plus, Informit/FAMILY (Australian), ProQuest

 Legal: AGIS Informit (Australian), CINCH, HeinOnline, WestLaw

 Biosciences/life sciences: Biological Abstracts (BIOSIS), CAB Abstracts, ScienceDirect

 Multidisciplinary: Scopus, Web of Science, ProQuest, ScienceDirect, Informit

 Humanities: JSTOR, ProQuest Central, and many others! (ask our Humanities librarians)

La Trobe University

30

QUICK TEST: Selecting sources to search Which scholarly databases would you search for the topic below?

What do we know about the experiences of patients with depression undergoing cognitive-behavioural therapy?

La Trobe University

31

QUICK TEST: Selecting sources to search Which scholarly databases would you search for the topic below?

What do we know about the experiences of patients with depression undergoing cognitive-behavioural therapy?



MEDLINE



PsycINFO



EMBASE



CINAHL



Cochrane Library



Scopus / Web of Science

La Trobe University

32

How many databases do I need to search?



Broadly speaking, 3-4 minimum



Depends on your discipline area and topic – how long is a piece of string?

La Trobe University

33

Designing your search strategy

La Trobe University

34

Designing your search strategy – logic grid CONCEPT 1 Depression

CONCEPT 2 Cognitive-behavioural therapy

CONCEPT 3 Qualitative studies

What do we know about the experiences of patients with depression undergoing cognitive-behavioural therapy?

La Trobe University

35

How is searching different in systematic reviews?



Highly comprehensive, highly sensitive



Searches both published AND unpublished literature



Uses a combination of subject headings (e.g. MeSH) AND keywords



Considers all synonyms and alternative search terms



Careful translation of search strategy across databases



Accurately documented search strategy



Search must be replicable

La Trobe University

36

Sensitivity vs precision

Sensitivity: comprehensiveness of a search • Higher sensitivity often lowers precision

Precision: relevancy of search results (or, accuracy) • Higher precision often lowers sensitivity

How to build a comprehensive search strategy that strikes a balance between sensitivity and precision?

La Trobe University

37

Sensitivity vs precision An example… 

High sensitivity, low precision



1 study included per 382 abstracts screened

Northey JM, Cherbuin N, Pumpa KL, et al Exercise interventions for cognitive function in adults older than 50: a systematic review with meta-analysis Br J Sports Med Published Online First: 24 April 2017. doi: 10.1136/bjsports-2016-096587

La Trobe University

38

Identifying your search terms

It's all about synonyms (and antonyms!)

Substance-related disorders Substance abuse

 How might others describe/articulate your concepts?

Substance addiction

 What terminology is used internationally? (e.g. Australia vs US)

Substance dependence

 Different disciplines, different words

Substance-induced organic mental disorders

 Has the terminology changed over time?

Drug habituation Drug use disorder Drug abuse

La Trobe University

39

Identifying your search terms

 Consult your fellow subject experts and colleagues

 Browse existing relevant articles – scan title, abstract, and subject headings to extract candidate search terms

 Find Cochrane Reviews similar to your topic – examine full detailed search strategies (documented in Appendix, usually)

 Text mining and word frequency analysis – use PubMed ReMiner (Google it!)

La Trobe University

40

Identifying your search terms Use “proxy terms” that implicitly speak to concepts you are interested in, despite not formally being synonyms.

One good tactic is to search for the *absence* of what you're looking for.

 e.g. for patient advocacy… ̶

Unsupported ̶

Vulnerable ̶

Empowerment ̶

Protection of rights ̶

Autonomy ̶

Lack of support ̶

Decisional capacity ̶

Legal rights, human rights

La Trobe University

41

Identifying your search terms – subject headings

Golden rule: use both keywords AND subject headings What’s the difference?



Keywords: words and phrases found in titles and abstracts ̶



Scattergun approach. Highly sensitive. Low precision.

Subject headings: human-curated terms, organised into a hierarchical thesaurus of categories

̶

La Trobe University

More precision. BUT – not all records have subject headings, and human curation is imperfect.

42

La Trobe University

43

La Trobe University

44

Developing your search strategy – subject headings

La Trobe University

45

Developing your search strategy – subject headings

La Trobe University

46

Developing your search strategy – subject headings

La Trobe University

47

Developing your search strategy – subject headings

La Trobe University

48

Developing your search strategy – subject headings

La Trobe University

49

Developing your search strategy – subject headings

La Trobe University

50

Developing your search strategy – subject headings

La Trobe University

51

Designing your search – adding subject headings to your logic grid CONCEPT 1 Depression

CONCEPT 2 Cognitive-behavioural therapy

CONCEPT 3 Qualitative studies

exp Depression/

exp Cognitive Therapy/

exp Qualitative Research/

exp Mood Disorders/

exp Behavioral Therapy/

exp Interview/

La Trobe University

52

HANDS ON: using subject headings (4 minutes)

1. Go to www.latrobe.edu.au/library -> Databases tab -> M -> MEDLINE 2. Pick ONE concept from your topic. Find TWO subject heading terms to use.



You may not find any subject headings for your concept at all – that’s fine!



Use the scope note to find out more and explore other candidate search terms



Click on the subject heading itself to explore the thesaurus tree



Would you ‘explode’ your subject heading, or not? Why or why not?

La Trobe University

53

Developing your search strategy – keywords

 Acronyms: search both the full term and acronyms ̶

e.g. for Intensive Care Unit: ICU, Intensive Care Unit, PICU, NICU

 However! Consider whether your acronym may accidentally retrieve unrelated sets of records. ̶

e.g. ED as emergency department may retrieve records featuring ED as erectile dysfunction

La Trobe University

54

Developing your search strategy – keywords

 Truncations: find variants and alternate spellings ̶

e.g. physiotherap* retrieves physiotherapy, physiotherapist, physiotherapists ̶

e.g. teen* retrieves teen, teens, teenage, teenager, teenagers

 Watch out! If you truncate too early, it might open the floodgates… ̶

e.g. car* retrieves carrying, careful, cars, carotid arteries ̶

e.g. literat* (as in literate) will retrieve literature

La Trobe University

55

Developing your search strategy – keywords

 Wildcards: replace a letter inside a word with a wildcard symbol to search for variants ̶

e.g. organi*ation retrieves organisation OR organization

La Trobe University

56

Developing your search strategy – keywords

 Proximity: find a word within a specified range of other words

̶

̶

e.g. in MEDLINE, goal* adj5 set* will retrieve o

‘setting of goals’

o

‘goal setting’

o

‘setting realistic and achievable goals’

e.g. in CINAHL, emergency N2 nurs* will retrieve o

‘emergency nurses’

o

‘emergency department nurse’

o

‘nurses in emergencies’

La Trobe University

57

Developing your search strategy – keywords

La Trobe University

58

Designing your search – adding keywords to your logic grid CONCEPT 1 Depression

CONCEPT 2 Cognitive-behavioural therapy

CONCEPT 3 Qualitative studies

exp Depression/

exp Cognitive Therapy/

exp Qualitative Research/

exp Mood Disorders/

exp Behavioral Therapy/

exp Interview/

depress* adj5 (symptom* OR mood* OR patient*)

cogniti* adj3 (behavio* OR intervention* OR psychotherap* OR technique* OR therap* OR treat*)

interview*

dysthymi*

CBT

themes qualitative

focus group* thematic* adj3 analy* narrative adj2 analy* La Trobe University

59

Designing your search strategy – combining your search terms

Boolean operator

Effect

Total results

Function

Explore relationship between different concepts

AND

Narrows your search

Fewer

OR

Broadens your search

Greater

Group similar terms together

Warning: you can use NOT to exclude unwanted terms, however – this is an extremely potent tool to be used very carefully. I would not recommend using it in a systematic review search strategy, as it is often misused with the effect of accidentally excluding relevant records.

La Trobe University

60

Database limits 

Limits can help to narrow down your search ̶

Age ̶

Gender ̶

Publication date (e.g. last 10 years) ̶

Language

 Very effective for fast, clinical searches  For systematic reviews, be wary

̶

Be particularly cautious of limits that tend to “over-reach” such as those for population age and human studies

 Always apply at the very END of your search http://hlwiki.slais.ubc.ca/index.php/Human_studies_filter

La Trobe University

61

Search filters



Methodological search filters ̶

Randomised controlled trials (Cochrane Handbook filter) ̶

Qualitative studies filter ̶

Quantitative studies filter

 Subject-based search filters ̶

e.g. indigenous Australians, palliative care, etc

La Trobe University

62

Search filters  Cochrane Handbook RCT filter (MEDLINE Ovid) 1

randomized controlled trial.pt.

2

controlled clinical trial.pt.

3

randomized.ab.

4

placebo.ab.

5

drug therapy.fs.

6

randomly.ab.

7

trial.ab.

8

groups.ab.

9

1 or 2 or 3 or 4 or 5 or 6 or 7 or 8

10

exp animals/ not humans.sh.

11

9 not 10

La Trobe University

63

Search filters – learn more



HLWiki – search filters http://hlwiki.slais.ubc.ca/index.php/Search_filters_%26_hedges

 Flinders Filters http://www.flinders.edu.au/clinical-change/research/flinders-filters/

 Find updated search filters at ISSG https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home

La Trobe University

64

Line-by-line search logic: tying it all together

#

Searches

1

Concept 1 subject headings

2

Concept 1 keywords

3

#1 OR #2

4

Concept 2 subject headings

5

Concept 2 keywords

6

#4 OR #5

7

Concept 3 subject headings

8

Concept 3 keywords

9

#7 OR #8

10

#3 AND #6 AND #9

La Trobe University

65

Line-by-line search logic: tying it all together # Searches 1 exp Depression/ 2 exp Mood Disorders/ 3 depress* adj5 (symptom* OR mood* OR patient*) 4 1 OR 2 OR 3 5 exp Cognitive Therapy/ 6 Cogniti* adj3 (behavio* or intervention* or psychotherap* or technique* or therap* or treat*) 7 5 OR 6 8 exp Qualitative Research/ 9 exp Interview/

Results 97649

10 qualitative

186872

11 themes

48395

12 interview*

326457

13 8 OR 9 OR 10 OR 11 OR 12

470867

14 4 AND 7 AND 13

La Trobe University

109804 115886 240805 22774 57980 59182 34372 27920

1131

66

Line-by-line search logic: tying it all together

Ang, M., Evans, J. R., & Mehta, J. S. (2014). Manual small incision cataract surgery (MSICS) with posterior chamber intraocular lens versus extracapsular cataract extraction (ECCE) with posterior chamber intraocular lens for age‐related cataract. The Cochrane Library. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008811.pub3/full

La Trobe University

67

Beyond databases – finding grey literature How to find unpublished studies?

Clinical trial registries o

ANZCTR, ClinicalTrials.gov, WHO ICTRP

Conference and meeting proceedings/abstracts

o

Conference websites, professional association conference programs

Dissertations and theses o

ProQuest Dissertations & Theses Global, institutional repositories, library catalogues, Google Scholar

Government reports, research white papers o

Australian Policy Online, national libraries, institutional repositories, Google Scholar Stansfield, C., Dickson, K., & Bangpan, M. (2016). Exploring issues in the conduct of website searching and other online sources for systematic reviews: how can we be systematic?. Systematic Reviews, 5(1), 191.

La Trobe University

68

Beyond databases – other methods

Contact topic experts and colleagues 

Email co-authors from key papers related to your topic



Ask about unpublished reports or clinical trials

Stansfield, C., Dickson, K., & Bangpan, M. (2016). Exploring issues in the conduct of website searching and other online sources for systematic reviews: how can we be systematic?. Systematic Reviews, 5(1), 191.

La Trobe University

69

Beyond databases – other methods  Harvest references from key papers 

Use Scopus or Web of Science for exploring reference lists



Track backwards AND forwards in time  Don’t limit yourself to references the paper has cited, but also examine who has cited it since it was published

La Trobe University

70

HANDS ON: finding unpublished clinical trials

Search ClinicalTrials.gov for unpublished randomised controlled trials https://clinicaltrials.gov/

OR Search Australian Policy Online for grey literature (government reports, white papers, etc) http://apo.org.au/

La Trobe University

71

Testing your search strategy  Use a set of known relevant studies to test your search strategy – does it retrieve all of them? ̶

If it misses known relevant studies, examine these records

̶

What subject headings or keywords have you missed?

La Trobe University

72

Documenting your search strategy

 Use PRISMA flow diagram as a guide  Methods section (narrative description) ̶

Describe all sources/databases you’ve searched ̶

Describe other methods you’ve deployed (e.g. grey literature searches, searching clinical trial registries, etc)

 Results section ̶

Display your PRISMA flow diagram

 Document your full search strategy in detailed syntax

̶

Put this in the Appendix ̶

Cover at least one database search strategy in full

La Trobe University

73

Documenting your search strategy

Vaniyapong T, Chongruksut W, Rerkasem K. Local versus general anaesthesia for carotid endarterectomy. Cochrane Database of Systematic Reviews 2013, Issue 12. Art. No.: CD000126. DOI: 10.1002/14651858.CD000126.pub4. La Trobe University

74

Documenting your search strategy http://prisma-statement.org/PRISMAStatement/FlowDiagram.aspx

La Trobe University

75

Exporting and organising references 

Organise references into folders based on which databases retrieved from



Use "Find Duplicates" function in EndNote to eliminate duplicate records



Document the number of records at each stage for your PRISMA Flowchart

La Trobe University

76

Accessing full-text articles for screening

For records allocated to be included (or unsure), obtain full-text

1. EndNote can automate retrieval of many/most full-text PDFs ̶

See the library’s EndNote guide for details (or ask a librarian!) http://latrobe.libguides.com/c.php?g=443224&p=3020833

2. Use Library Search

La Trobe University

77

Software for systematic reviews & meta-analysis

La Trobe University

78

Common mistakes to avoid

 Don't underestimate time and effort required



̶

Potentially more than a year ̶

Plan systematically (that’s what your protocol is for!) ̶

Avoid “shortcuts”

Don't go it alone ̶

MINIMUM of one other individual to cross-check your search strategy and screen records

La Trobe University

79

Support for meta-analysis



La Trobe Statistics Consultancy Platform (general statistical/meta-analysis advice) •

5 hours of free consultations for research students

http://www.latrobe.edu.au/research-infrastructure/research-facilities/statisticsconsultancy-platform

• La Trobe Digital Research (advice on specific software e.g. R for power analysis) •

Monthly drop-in sessions



Advice from Ghulam Murtaza

http://www.latrobe.edu.au/research-infrastructure/digital-research

La Trobe University

81

Further resources

La Trobe University

82

Further resources

http://latrobe.libguides.com/systematicreviews La Trobe University

83

Further resources

Free online course: https://www.coursera.org/learn/systematic-review La Trobe University

84

Thank you latrobe.edu.au Steven Chang – [email protected] La Trobe University Library La Trobe University CRICOS Provider Code Number 00115M © Copyright La Trobe University 2017