latrobe.edu.au
Systematic Searching for Systematic Reviews: an introduction Steven Chang –
[email protected] La Trobe University Library
Wednesday 7th February 2018
La Trobe University CRICOS Provider Code Number 00115M
Overview Difference between a systematic review and a traditional literature review Transforming your research question into a rigorous search strategy Choosing the right databases / sources to search for your topic area Developing a robust and comprehensive search strategy Translating searches across multiple databases and platforms Becoming aware of key software to support your systematic review (and meta-analysis, if you’re doing one)
La Trobe University
2
In brief:
What’s your background?
Why are you interested in learning to do a systematic review?
La Trobe University
3
What is a systematic review?
“A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias…” - Cochrane Library
Cochrane Library (2017). About Cochrane Systematic Reviews and Protocols [online]. Available at: http://www.cochranelibrary.com/about/about-cochranesystematic-reviews.html [accessed 21/04/17].
La Trobe University
4
What is a systematic review?
https://www.youtube.com/watch?v=egJlW4vkb1Y La Trobe University
5
Study 1
Study 2
Study 3
Systematic review and meta-analysis La Trobe University
6
Case study – post-traumatic psychological interventions
For traumatic events, psychological debriefing was standard practice until 2002 Cochrane Review in 2002 identified nine studies to evaluate the effectiveness of early single-session psychological debriefing for preventing post-traumatic stress disorder (PTSD) Shows how systematic reviews can shine a light on the whole picture to change
standard practice
Rose, S. C., Bisson, J., Churchill, R., & Wessely, S. (2002). Psychological debriefing for preventing post traumatic stress disorder (PTSD). The Cochrane Library. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD000560/full
Rose, S., Bisson, J., & Wessely, S. (2003). A systematic review of single-session psychological
interventions (‘debriefing’) following trauma. Psychotherapy and psychosomatics, 72(4), 176184. http://www.karger.com/article/abstract/70781
La Trobe University
7
Why do we need systematic reviews?
Managing information overload
Building a comprehensive, trustworthy picture of a topic Answering what is known and what is unknown Enable informed decision-making by clinicians, policymakers, and researchers Minimise the risk of duplicating research efforts and wasting research resources
La Trobe University
8
The hierarchy of evidence Systematic reviews, meta-analyses
RCTs
Quality
Cohort studies
Case-control studies
Case reports La Trobe University
9
The key stages Formulate a focused, answerable research question
Define scope and inclusion/exclusion criteria to be used
Systematically search for all relevant research (both published AND unpublished) Assess the quality and risk of bias for each study included in your review Synthesise included studies via meta-analysis (if possible) La Trobe University
10
How is a systematic review different from a traditional narrative review? Traditional/narrative review
Systematic review
Search strategy not explicitly stated
Literature gathered using explicit search methodology (must be replicable)
Studies may be “cherry-picked” by author
Explicit inclusion & exclusion criteria
Prone to selection bias
Minimises selection bias
No standard consensus on minimum items
Standard protocols i.e. PRISMA guidelines
May describe a broad overview of a topic
Focuses on a specific, structured question
Based on author’s subjectivity and expertise
Based on structured, scientific methodology
Traditional narrative reviews may be at high risk of bias. Systematic reviews aim to minimise this bias. La Trobe University
11
Systematic review vs “systematic literature review”
A systematic review is a very specific type of study ̶
Uses particular methodology which must be explicitly reported/documented ̶
Follows standardised guidelines (PRISMA, PRISMA-P, etc) ̶
Included studies usually undergo assessment of quality and risk of bias ̶
It is NOT simply a literature search that is more systematic than usual
Are you doing an actual systematic review?
Or are you doing a "normal" literature review that you want to be systematic with?
These are two different things
La Trobe University
12
Consider other types of reviews
Scoping review
Rapid review
Umbrella review/overview of reviews (systematic reviews of systematic reviews!)
And many more…!
Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108.
La Trobe University
13
What you need to get started
Time! A team ̶
Subject experts for developing research question and screening records
̶
Librarian / information specialist for performing a systematic search ̶
Statistician / methodologist for undertaking meta-analysis
La Trobe University
15
What you need to get started
WARNING: Slow and deliberate science
Systematic reviews are a hefty investment of time, labour, and effort! ̶
Average of five authors ̶
Mean of 67.3 weeks from start to finish (to be published) ̶
Average yield of relevant items per search: 2.94% (e.g. only 1 relevant study per 30 search results!)
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ open, 7(2), e012545.
La Trobe University
16
Before you start – is a review actually needed? There is a huge amount of duplication and research waste being done Solution – check PROSPERO before you start your systematic review! Don’t forget to register YOUR protocol when it’s completed
Moher, D., Booth, A., & Stewart, L. (2014). How to reduce unnecessary duplication: use PROSPERO. BJOG: An International Journal of Obstetrics & Gynaecology, 121(7), 784-786. Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012). The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Systematic Reviews, 1(1), 2.
La Trobe University
17
HANDS ON: searching PROSPERO (3 minutes)
Can you find existing systematic reviews on your topic, or related to your topic?
Search PROSPERO Tip: Don’t overcomplicate it – just chuck in a few keywords https://www.crd.york.ac.uk/PROSPERO/
La Trobe University
18
Before you start – develop a plan (protocol): PRISMA-P! PRISMA-P checklist http://www.prisma-statement.org/documents/PRISMA-P-checklist.pdf
La Trobe University
19
Before you start – develop a plan (protocol): PRISMA-P! PRISMA-P elaboration and explanation (this is your step-by-step guide!) ̶
http://www.bmj.com/content/349/bmj.g7647
La Trobe University
20
Formulating your research question
La Trobe University
21
Formulating your research question
Use mnemonics to design a specific and answerable research question
Quantitative research question? Go with PICO(S)
̶
C (comparison intervention) and O (outcomes) often omitted from search strategy, but included for developing inclusion/exclusion criteria
Qualitative? Try SPICE, SPIDER, or others.
La Trobe University
22
Formulating your research question: PICO(S)
P
Patient / population
Patients with heart failure
I
Intervention
Telemedicine
Comparison intervention
Conventional clinical assessment and treatment
O
Outcome(s)
Functional outcome, as measured by modified Rankin Score (mRS)
S
Study type
Randomised controlled trials
C
How effective and safe is telemedicine for patients with heart failure, compared to conventional clinical assessment and treatment? La Trobe University
23
Formulating your research question: SPIDER (qualitative) S
Sample group
Pre-school, primary, and secondary students
P I
Phenomenon of Interest
Mindfulness programs
Design (specific methodology)
Quasi-experimental design (QED)
D
E
Evaluation
Socio-emotional outcomes Behavioural outcomes Academic outcomes
R
Research type (e.g. mixed methods, quant, or qual)
Mixed methods
Do mindfulness programs improve the academic, behavioural, and socioemotional functioning of primary and secondary students? La Trobe University
24
Formulating your research question: SPICE (qualitative) S
Setting (where?)
n/a (global – all countries)
Perspective (for who or what?)
Legumes
I
Intervention (phenomenon of interest)
Elevated CO2
C
Comparison
No elevation in CO2
E
Evaluation (outcome)
Seed mass Germination Seed vigour
P
What is the effect of climate change on the seed quality of legume crops? La Trobe University
25
Formulating your research question: PICo (qualitative)
La Trobe University
26
HANDS ON: Formulating your research question (3 minutes) 1. Describe your research question of interest in one or two sentences. 2. Break down your research question into PICO/SPIDER format
No hard and fast rules
It’s just a guide to help you conceptually
You can leave some parts blank! (usually the C and O of PICO…)
La Trobe University
27
Selecting sources to search
La Trobe University
28
Selecting sources to search – health sciences databases
MEDLINE
Cochrane Library
CINAHL
PsycINFO
EMBASE
La Trobe University
29
Selecting sources to search – databases outside health sciences Education: ERIC, AEI Informit (Australian)
Sociology: Sociological Abstracts, Anthropology Plus, Informit/FAMILY (Australian), ProQuest
Legal: AGIS Informit (Australian), CINCH, HeinOnline, WestLaw
Biosciences/life sciences: Biological Abstracts (BIOSIS), CAB Abstracts, ScienceDirect
Multidisciplinary: Scopus, Web of Science, ProQuest, ScienceDirect, Informit
Humanities: JSTOR, ProQuest Central, and many others! (ask our Humanities librarians)
La Trobe University
30
QUICK TEST: Selecting sources to search Which scholarly databases would you search for the topic below?
What do we know about the experiences of patients with depression undergoing cognitive-behavioural therapy?
La Trobe University
31
QUICK TEST: Selecting sources to search Which scholarly databases would you search for the topic below?
What do we know about the experiences of patients with depression undergoing cognitive-behavioural therapy?
MEDLINE
PsycINFO
EMBASE
CINAHL
Cochrane Library
Scopus / Web of Science
La Trobe University
32
How many databases do I need to search?
•
Broadly speaking, 3-4 minimum
•
Depends on your discipline area and topic – how long is a piece of string?
La Trobe University
33
Designing your search strategy
La Trobe University
34
Designing your search strategy – logic grid CONCEPT 1 Depression
CONCEPT 2 Cognitive-behavioural therapy
CONCEPT 3 Qualitative studies
What do we know about the experiences of patients with depression undergoing cognitive-behavioural therapy?
La Trobe University
35
How is searching different in systematic reviews?
Highly comprehensive, highly sensitive
Searches both published AND unpublished literature
Uses a combination of subject headings (e.g. MeSH) AND keywords
Considers all synonyms and alternative search terms
Careful translation of search strategy across databases
Accurately documented search strategy
Search must be replicable
La Trobe University
36
Sensitivity vs precision
Sensitivity: comprehensiveness of a search • Higher sensitivity often lowers precision
Precision: relevancy of search results (or, accuracy) • Higher precision often lowers sensitivity
How to build a comprehensive search strategy that strikes a balance between sensitivity and precision?
La Trobe University
37
Sensitivity vs precision An example…
High sensitivity, low precision
1 study included per 382 abstracts screened
Northey JM, Cherbuin N, Pumpa KL, et al Exercise interventions for cognitive function in adults older than 50: a systematic review with meta-analysis Br J Sports Med Published Online First: 24 April 2017. doi: 10.1136/bjsports-2016-096587
La Trobe University
38
Identifying your search terms
It's all about synonyms (and antonyms!)
Substance-related disorders Substance abuse
How might others describe/articulate your concepts?
Substance addiction
What terminology is used internationally? (e.g. Australia vs US)
Substance dependence
Different disciplines, different words
Substance-induced organic mental disorders
Has the terminology changed over time?
Drug habituation Drug use disorder Drug abuse
La Trobe University
39
Identifying your search terms
Consult your fellow subject experts and colleagues
Browse existing relevant articles – scan title, abstract, and subject headings to extract candidate search terms
Find Cochrane Reviews similar to your topic – examine full detailed search strategies (documented in Appendix, usually)
Text mining and word frequency analysis – use PubMed ReMiner (Google it!)
La Trobe University
40
Identifying your search terms Use “proxy terms” that implicitly speak to concepts you are interested in, despite not formally being synonyms.
One good tactic is to search for the *absence* of what you're looking for.
e.g. for patient advocacy… ̶
Unsupported ̶
Vulnerable ̶
Empowerment ̶
Protection of rights ̶
Autonomy ̶
Lack of support ̶
Decisional capacity ̶
Legal rights, human rights
La Trobe University
41
Identifying your search terms – subject headings
Golden rule: use both keywords AND subject headings What’s the difference?
Keywords: words and phrases found in titles and abstracts ̶
Scattergun approach. Highly sensitive. Low precision.
Subject headings: human-curated terms, organised into a hierarchical thesaurus of categories
̶
La Trobe University
More precision. BUT – not all records have subject headings, and human curation is imperfect.
42
La Trobe University
43
La Trobe University
44
Developing your search strategy – subject headings
La Trobe University
45
Developing your search strategy – subject headings
La Trobe University
46
Developing your search strategy – subject headings
La Trobe University
47
Developing your search strategy – subject headings
La Trobe University
48
Developing your search strategy – subject headings
La Trobe University
49
Developing your search strategy – subject headings
La Trobe University
50
Developing your search strategy – subject headings
La Trobe University
51
Designing your search – adding subject headings to your logic grid CONCEPT 1 Depression
CONCEPT 2 Cognitive-behavioural therapy
CONCEPT 3 Qualitative studies
exp Depression/
exp Cognitive Therapy/
exp Qualitative Research/
exp Mood Disorders/
exp Behavioral Therapy/
exp Interview/
La Trobe University
52
HANDS ON: using subject headings (4 minutes)
1. Go to www.latrobe.edu.au/library -> Databases tab -> M -> MEDLINE 2. Pick ONE concept from your topic. Find TWO subject heading terms to use.
You may not find any subject headings for your concept at all – that’s fine!
Use the scope note to find out more and explore other candidate search terms
Click on the subject heading itself to explore the thesaurus tree
Would you ‘explode’ your subject heading, or not? Why or why not?
La Trobe University
53
Developing your search strategy – keywords
Acronyms: search both the full term and acronyms ̶
e.g. for Intensive Care Unit: ICU, Intensive Care Unit, PICU, NICU
However! Consider whether your acronym may accidentally retrieve unrelated sets of records. ̶
e.g. ED as emergency department may retrieve records featuring ED as erectile dysfunction
La Trobe University
54
Developing your search strategy – keywords
Truncations: find variants and alternate spellings ̶
e.g. physiotherap* retrieves physiotherapy, physiotherapist, physiotherapists ̶
e.g. teen* retrieves teen, teens, teenage, teenager, teenagers
Watch out! If you truncate too early, it might open the floodgates… ̶
e.g. car* retrieves carrying, careful, cars, carotid arteries ̶
e.g. literat* (as in literate) will retrieve literature
La Trobe University
55
Developing your search strategy – keywords
Wildcards: replace a letter inside a word with a wildcard symbol to search for variants ̶
e.g. organi*ation retrieves organisation OR organization
La Trobe University
56
Developing your search strategy – keywords
Proximity: find a word within a specified range of other words
̶
̶
e.g. in MEDLINE, goal* adj5 set* will retrieve o
‘setting of goals’
o
‘goal setting’
o
‘setting realistic and achievable goals’
e.g. in CINAHL, emergency N2 nurs* will retrieve o
‘emergency nurses’
o
‘emergency department nurse’
o
‘nurses in emergencies’
La Trobe University
57
Developing your search strategy – keywords
La Trobe University
58
Designing your search – adding keywords to your logic grid CONCEPT 1 Depression
CONCEPT 2 Cognitive-behavioural therapy
CONCEPT 3 Qualitative studies
exp Depression/
exp Cognitive Therapy/
exp Qualitative Research/
exp Mood Disorders/
exp Behavioral Therapy/
exp Interview/
depress* adj5 (symptom* OR mood* OR patient*)
cogniti* adj3 (behavio* OR intervention* OR psychotherap* OR technique* OR therap* OR treat*)
interview*
dysthymi*
CBT
themes qualitative
focus group* thematic* adj3 analy* narrative adj2 analy* La Trobe University
59
Designing your search strategy – combining your search terms
Boolean operator
Effect
Total results
Function
Explore relationship between different concepts
AND
Narrows your search
Fewer
OR
Broadens your search
Greater
Group similar terms together
Warning: you can use NOT to exclude unwanted terms, however – this is an extremely potent tool to be used very carefully. I would not recommend using it in a systematic review search strategy, as it is often misused with the effect of accidentally excluding relevant records.
La Trobe University
60
Database limits
Limits can help to narrow down your search ̶
Age ̶
Gender ̶
Publication date (e.g. last 10 years) ̶
Language
Very effective for fast, clinical searches For systematic reviews, be wary
̶
Be particularly cautious of limits that tend to “over-reach” such as those for population age and human studies
Always apply at the very END of your search http://hlwiki.slais.ubc.ca/index.php/Human_studies_filter
La Trobe University
61
Search filters
Methodological search filters ̶
Randomised controlled trials (Cochrane Handbook filter) ̶
Qualitative studies filter ̶
Quantitative studies filter
Subject-based search filters ̶
e.g. indigenous Australians, palliative care, etc
La Trobe University
62
Search filters Cochrane Handbook RCT filter (MEDLINE Ovid) 1
randomized controlled trial.pt.
2
controlled clinical trial.pt.
3
randomized.ab.
4
placebo.ab.
5
drug therapy.fs.
6
randomly.ab.
7
trial.ab.
8
groups.ab.
9
1 or 2 or 3 or 4 or 5 or 6 or 7 or 8
10
exp animals/ not humans.sh.
11
9 not 10
La Trobe University
63
Search filters – learn more
HLWiki – search filters http://hlwiki.slais.ubc.ca/index.php/Search_filters_%26_hedges
Flinders Filters http://www.flinders.edu.au/clinical-change/research/flinders-filters/
Find updated search filters at ISSG https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home
La Trobe University
64
Line-by-line search logic: tying it all together
#
Searches
1
Concept 1 subject headings
2
Concept 1 keywords
3
#1 OR #2
4
Concept 2 subject headings
5
Concept 2 keywords
6
#4 OR #5
7
Concept 3 subject headings
8
Concept 3 keywords
9
#7 OR #8
10
#3 AND #6 AND #9
La Trobe University
65
Line-by-line search logic: tying it all together # Searches 1 exp Depression/ 2 exp Mood Disorders/ 3 depress* adj5 (symptom* OR mood* OR patient*) 4 1 OR 2 OR 3 5 exp Cognitive Therapy/ 6 Cogniti* adj3 (behavio* or intervention* or psychotherap* or technique* or therap* or treat*) 7 5 OR 6 8 exp Qualitative Research/ 9 exp Interview/
Results 97649
10 qualitative
186872
11 themes
48395
12 interview*
326457
13 8 OR 9 OR 10 OR 11 OR 12
470867
14 4 AND 7 AND 13
La Trobe University
109804 115886 240805 22774 57980 59182 34372 27920
1131
66
Line-by-line search logic: tying it all together
Ang, M., Evans, J. R., & Mehta, J. S. (2014). Manual small incision cataract surgery (MSICS) with posterior chamber intraocular lens versus extracapsular cataract extraction (ECCE) with posterior chamber intraocular lens for age‐related cataract. The Cochrane Library. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008811.pub3/full
La Trobe University
67
Beyond databases – finding grey literature How to find unpublished studies?
Clinical trial registries o
ANZCTR, ClinicalTrials.gov, WHO ICTRP
Conference and meeting proceedings/abstracts
o
Conference websites, professional association conference programs
Dissertations and theses o
ProQuest Dissertations & Theses Global, institutional repositories, library catalogues, Google Scholar
Government reports, research white papers o
Australian Policy Online, national libraries, institutional repositories, Google Scholar Stansfield, C., Dickson, K., & Bangpan, M. (2016). Exploring issues in the conduct of website searching and other online sources for systematic reviews: how can we be systematic?. Systematic Reviews, 5(1), 191.
La Trobe University
68
Beyond databases – other methods
Contact topic experts and colleagues
Email co-authors from key papers related to your topic
Ask about unpublished reports or clinical trials
Stansfield, C., Dickson, K., & Bangpan, M. (2016). Exploring issues in the conduct of website searching and other online sources for systematic reviews: how can we be systematic?. Systematic Reviews, 5(1), 191.
La Trobe University
69
Beyond databases – other methods Harvest references from key papers
Use Scopus or Web of Science for exploring reference lists
Track backwards AND forwards in time Don’t limit yourself to references the paper has cited, but also examine who has cited it since it was published
La Trobe University
70
HANDS ON: finding unpublished clinical trials
Search ClinicalTrials.gov for unpublished randomised controlled trials https://clinicaltrials.gov/
OR Search Australian Policy Online for grey literature (government reports, white papers, etc) http://apo.org.au/
La Trobe University
71
Testing your search strategy Use a set of known relevant studies to test your search strategy – does it retrieve all of them? ̶
If it misses known relevant studies, examine these records
̶
What subject headings or keywords have you missed?
La Trobe University
72
Documenting your search strategy
Use PRISMA flow diagram as a guide Methods section (narrative description) ̶
Describe all sources/databases you’ve searched ̶
Describe other methods you’ve deployed (e.g. grey literature searches, searching clinical trial registries, etc)
Results section ̶
Display your PRISMA flow diagram
Document your full search strategy in detailed syntax
̶
Put this in the Appendix ̶
Cover at least one database search strategy in full
La Trobe University
73
Documenting your search strategy
Vaniyapong T, Chongruksut W, Rerkasem K. Local versus general anaesthesia for carotid endarterectomy. Cochrane Database of Systematic Reviews 2013, Issue 12. Art. No.: CD000126. DOI: 10.1002/14651858.CD000126.pub4. La Trobe University
74
Documenting your search strategy http://prisma-statement.org/PRISMAStatement/FlowDiagram.aspx
La Trobe University
75
Exporting and organising references
Organise references into folders based on which databases retrieved from
Use "Find Duplicates" function in EndNote to eliminate duplicate records
Document the number of records at each stage for your PRISMA Flowchart
La Trobe University
76
Accessing full-text articles for screening
For records allocated to be included (or unsure), obtain full-text
1. EndNote can automate retrieval of many/most full-text PDFs ̶
See the library’s EndNote guide for details (or ask a librarian!) http://latrobe.libguides.com/c.php?g=443224&p=3020833
2. Use Library Search
La Trobe University
77
Software for systematic reviews & meta-analysis
La Trobe University
78
Common mistakes to avoid
Don't underestimate time and effort required
̶
Potentially more than a year ̶
Plan systematically (that’s what your protocol is for!) ̶
Avoid “shortcuts”
Don't go it alone ̶
MINIMUM of one other individual to cross-check your search strategy and screen records
La Trobe University
79
Support for meta-analysis
•
La Trobe Statistics Consultancy Platform (general statistical/meta-analysis advice) •
5 hours of free consultations for research students
http://www.latrobe.edu.au/research-infrastructure/research-facilities/statisticsconsultancy-platform
• La Trobe Digital Research (advice on specific software e.g. R for power analysis) •
Monthly drop-in sessions
•
Advice from Ghulam Murtaza
http://www.latrobe.edu.au/research-infrastructure/digital-research
La Trobe University
81
Further resources
La Trobe University
82
Further resources
http://latrobe.libguides.com/systematicreviews La Trobe University
83
Further resources
Free online course: https://www.coursera.org/learn/systematic-review La Trobe University
84
Thank you latrobe.edu.au Steven Chang –
[email protected] La Trobe University Library La Trobe University CRICOS Provider Code Number 00115M © Copyright La Trobe University 2017
