Sources of information on adverse effects: a ... - Wiley Online Library

DOI: 10.1111/j.1471-1842.2010.00901.x

Sources of information on adverse effects: a systematic review Su Golder* & Yoon K. Loke† *Centre for Reviews and Dissemination (CRD), University of York, York, UK and †School of Medicine, Health Policy and Practice, University of East Anglia, Norwich, UK

Abstract Background: Systematic reviews can provide accurate and timely information on adverse effects. An essential part of the systematic review process is a thorough search of the literature. This often requires searching many different sources. However, it is unclear which sources are most effective at providing information on adverse effects. Objective: To identify and summarise studies that have evaluated sources of information on adverse effects. Methods: Studies were located by searching in 10 databases as well as by reference checking, hand searching, citation searching and contacting experts. Results: A total of 6218 citations were retrieved yielding 19 studies which met the inclusion criteria. The included studies tended to focus on the adverse effects of drug interventions and compare the relative value of different sources using the number of relevant references retrieved from searches of each source. However, few studies were conducted recently with a large sample of references. Conclusions: This review suggests that EMBASE, Derwent Drug File, MEDLINE and industry submissions may potentially provide the greatest number of relevant references for information on adverse effects of drugs. However, a systematic evaluation of the current value of different sources of information for adverse effects is urgently required.

Key Messages Implications for Practice d

d

d

Because of methodological weaknesses in the existing evidence, it is difficult to produce definitive guidance on the most effective sources for retrieving information on adverse effects. Authors of systematic reviews may need to search a range of sources to retrieve information on adverse effects. EMBASE, Derwent Drug File, MEDLINE and industry submissions may be the best starting points for retrieving information on adverse drug reactions.

Implications for Policy d

Research is required into the relative value of different sources of information on adverse effects and any evaluation needs to take into consideration other factors in addition to the number of relevant records retrieved.

Introduction An essential part of the systematic review process is to identify as many relevant studies as possible that address the question posed. This process usuCorrespondence: Su Golder, Centre for Reviews and Dissemination (CRD), University of York, York, YO10 5DD, UK. E-mail: [email protected]

176

ally entails searching a range of different data sources. The selection of these data sources is an important consideration for authors of systematic reviews. Where information is sought is likely to influence the amount and type of data retrieved.1 There are many potentially useful sources for information on adverse effects. In addition to the sources that traditionally searched for effectiveness

ª 2010 The authors. Health Information and Libraries Journal ª 2010 Health Libraries Group Health Information and Libraries Journal, 27, pp.176–190

Sources of information on adverse effects, Su Golder & Yoon K. Loke

data, information on adverse effects can be identified from: specialist journals, bulletins and databases, drug monographs or summaries, trial registries, spontaneous reporting systems ⁄ post-marketing surveillance systems or prescription-event monitoring data. Despite the large number of potentially useful sources for information on adverse effects, a recent survey showed that most systematic reviews of adverse effects rely only on MEDLINE.2,3 This is probably due to familiarity, widespread availability and a lack of knowledge of other sources. However, MEDLINE is unlikely to be the most definitive source of information on adverse effects.4,5 Although MEDLINE may not be the most comprehensive source of information on adverse effects, it is not known which data sources are most fruitful to search for information on adverse effects. An evaluation of the comparative value of the different sources of information in identifying adverse effect data would, therefore, be useful for authors of systematic reviews. This systematic review aims to identify existing methodological research that has investigated the impact of different sources to identify information on adverse effects of healthcare interventions. Methods Search strategy Searches were undertaken in 10 electronic databases, including the Cochrane Methodology Register, MEDLINE and EMBASE to retrieve methodology papers related to all aspects of the incorporation of adverse effects into systematic reviews. Due to the limitations of searching for methodological papers, it was envisaged that relevant papers may be missed by searching databases alone. We therefore undertook citation searches of all included papers using ISI Web of Knowledge, handsearching of selected key journals, conference proceedings and web sources, contacted key experts in the field of adverse effects and checked the bibliographies of any eligible articles (Table 1). The search strategy used to identify relevant methodological studies in the Cochrane Methodology Register is described in Appendix 1 (available online). This strategy was translated as appropriate for the other databases. No date or language restrictions were applied to the searches.

Table 1 Sources searched for included studies Databases All databases were originally searched on 26 or 27 September 2007 with the most recent update searches carried out between 22 October and 6 November 2009 Cochrane Database of Systematic Reviews (CDSR): methodology reviews only: 2009 Issue 4 Cochrane Methodology Register (CMR): 2009 Issue 4 Database of Abstracts of Reviews of Effects (DARE): November 2009 EMBASE: 1980–2009 Week 42 Health Technology Assessment (HTA) Database: November 2009 Health Management Information Consortium (HMIC): September 2009 Index to Theses: November 2009 Library, Information Science & Technology Abstracts (LISTA): mid-1960s to October 2009 MEDLINE: 1950 to October Week 3 MEDLINE in process: 22 October 2009 Handsearching of Journals BMC Clinical Pharmacology – 2001;1(1) to 2009;9(17) BMC Medical Research Methodology – 2001;1 to 2009;9(69) Drug Safety – 1998;18(1) to 2009;32(11) Health Information and Libraries Journal (formerly Health Libraries Review) – 1994;11(1) to 2009;26(3) Journal of Clinical Epidemiology – 1998 to 2009;62(12) Journal of Information Science – 1979;1(1) to 2009;35(5) Journal of Librarianship and Information Science – 1969;1(1) to 2009;41(3) Journal of the Medical Library Association (formerly the Bulletin of the Medical Library Association) – from 2000;88(2) to 2009;97(4) Pharmacoepidemiology & Drug Safety – 1992;1(1) to 2009;18(11) Handsearching of Conference Proceedings Cochrane Colloquia 1994–2009 HTAi 2004–2009 Pharma-Bio-Med Conference and Exposition 2006–2008 Symposium on Systematic Reviews 1998–2002 Web Sources Agency for Healthcare Research and Quality (AHRQ) via http://www.ahrq.gov/ Searched: 28 ⁄ 10 ⁄ 2009 Health Technology Assessment Programme (HTA) via http://www.hta.ac.uk/index.shtml Searched: 28 ⁄ 10 ⁄ 2009

Inclusion criteria A research study was considered eligible for inclusion if it compared the effectiveness of two or more data sources in identifying information on


177

178


adverse effects of a healthcare intervention. Studies were excluded if they compared databases with only animal or laboratory studies or were not written in English and had no translation available at the British Library. Data extraction Two independent reviewers (SG and YKL) performed data extraction and validity assessment, and reached full consensus after discussion and re-checking of the source data. Information was collected on: the intervention and adverse effects searched for, the data sources evaluated, the type of outcome measures used (such as number of relevant references retrieved) and the results. Assessment of methodological quality The included studies were assessed using the following questions: 1 Generalisability: Did the search queries cover a range of interventions and adverse effects? Was a large enough sample of relevant references used to compare the sources of information to make inferences about the results? (evaluated using 95% confidence intervals). 2 Database overlap: Were the number of unique relevant references and the total number of relevant references recorded for each data source? 3 Limitations of the search strategies: Were any of the limitations of the search strategies used to identify the relevant references taken into consideration? For example, relevant references may have been available on a particular database but limitations of the search strategy meant these references were not retrieved. 4 Comparative outcomes: Were adequate comparisons made between the included sources of information? Data analysis It is envisaged that the nature of the outcomes used here will not enable any formal statistical analysis to be carried out. However, the rank order of the sources from the different studies will be compared wherever possible.

Results Included studies A total of 6218 citations were screened and 19 methodological evaluations were found to meet the inclusion criteria (Fig. 1).6–24 The majority of the methodological evaluations (11 ⁄ 19) included more than one search for adverse effect data (Table 2).6,8–10,13,16,19,21–24 Ninety-four separate searches were conducted for information on adverse effects, however, four methodological evaluations (representing 36 searches) did not present the results of each search separately.8–10,24 To be able to compare the results of all 19 methodological evaluations, the combined results of the searches within each methodological evaluation were used wherever possible. In two instances, combining the results within a methodological evaluation was deemed inappropriate. One methodological evaluation contained searches for two different types of interventions23 and another included searches of different data sources.21 Types of interventions. Most of the methodological evaluations (16 ⁄ 19, 84%) included searches for named drug interventions.6–14,16–18,20–23 Three methodological evaluations searched for natural products, such as herbal medicines15,24 or aromatherapy19 and one methodological evaluation included a search on tooth extraction (in addition to searches for a drug intervention).23 Types of adverse effects. Twelve methodological evaluations included searches for a wide range of adverse effects.6,7,9,12,14,16,18–22,24 Five methodological evaluations included searches for either named adverse effects,11,13,17,21,23 such as pancreatitis17 or hepatitis,21 or a group of adverse effects,11,23 such as gastrointestinal side-effects11 or effects on the heart.23 The other methodological evaluations did not provide any information on the adverse effects included.8,10,15 Excluded studies Thirty-seven studies were excluded from this review.25–61 Eighteen studies compared data



6218 unique records retrieved from electronic database searches.

291 background papers or related to other aspects of systematic reviews of adverse effects, such as search strategies, publication bias, study designs and role of funding source.

From checking title and abstract we removed 5871 articles that did not look at adverse effects of interventions. For example, self-harm, patient safety and health risks of smoking.

347 full papers retrieved.

56 papers potentially meeting the inclusion criteria.

37 excluded papers.

19 included publications

Figure 1 Flow chart for included studies

drug information,26,29– eight did not present the results for identifying information on adverse effects,25,27,28,40,41,47,51,54 eight were descriptive studies,34,35,39,44,46,49,50,56 two studies did not contain enough information43,48 and one study included consumer information sources only.36 sources

for

other

33,37,38,42,45,52,53,55,57–61

Summary of methodological quality Generalisability. The generalisability of many of the methodological evaluations was limited by the low number of relevant references studied and in some cases by the limits on the type of interventions and adverse effects included. Database overlap. Whilst most methodological evaluations compared data sources by the number of relevant references retrieved, only four fully described both the total number of relevant references

and the unique relevant references11–13,22 and five partially took account of total and unique relevant references.9,16,18,20,23 This makes assessment of the overlap between sources and the value of combinations of sources difficult. Limitations of the search strategies. The majority of the studies did not take into account any limitations of the search strategies used to identify the relevant references. Only three methodological evaluations11,12,23 recorded the number of relevant references available in each database but not retrieved by searching. Comparative outcomes. Twelve methodological evaluations compared data sources by the number of relevant references. The other evaluations compared the number of case reports of adverse events,17 the unique relevant publications18 or gave scores to the sources of information for ability to


179

180

Sources of information on adverse effects, Su Golder & Yoon K. Loke Table 2 Characteristics of Included Studies References

Study details

Al Hefzi et al. (1987)6

Sample size: 51–250 relevant references Adverse effect: major side effects Intervention: five antineoplastic drugs Sample size: 52 included papers64 Adverse effect: numerous Intervention: atypical antipsychotics

Bagnall et al. (2002)7

Sources searched

Results

IDIS De Haen’s Drugs in Use† IPA

Relevant references 159 89 7

EMBASE MEDLINE

Industry Submissions Reference Lists ⁄ Hand Searching PsycINFO ADIS LMS Drug Alerts Online* BIOSIS Previews* TOXLINE ADIS Inpharma* British Library Inside Conferences* CAB HEALTH* CPI* Derwent Drug File* ExtraMED* IDIS* IPA* JICST-EPlus* Mental Health Abstracts*† NTIS* PASCAL* Pharma marketing* PHARMLINE* SEDBASE† Belgado (1997)8

Sample size: NR Adverse effect: NR Intervention: five unnamed drug interventions

Drugdex Facts and Comparisons Lexi-Comp’s Clinical Reference Library Clinical Pharmacology Drug Information Fulltext Physician’s GenRx Physician’s Desk Reference

Clinical Pharmacology Drugdex Physician’s GenRx Lexi-Comp Clinical Reference Library Physician’s Desk Reference Facts and Comparisons Drug Information Fulltext

Included papers (after deduplication at retrieval stage) 16 13 11 7 2 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Median score for overall ability to answer five queries A A B C C C D Median score for user-friendliness (0 is most user friendly, 100 least) 14 14 21.5 28.5 35.5 65.5 70



Table 2 (Continued) References

Study details

Biarez et al. (1991)9

Sample size: 838–1826 relevant references Adverse effect: any Intervention: 10 named drugs

Sources searched

EMBASE

TOXLINE MEDLINE

BIOSIS Previews IDIS Core MEDLINE PHARMLINE PASCAL IPA

EMBASE

BIOSIS Previews TOXLINE IDIS PASCAL PHARMLINE IPA MEDLINE

Clauson et al. (2007)10

Sample size: NR Adverse effect: NR Intervention: 13 unspecified drugs

Clinical Pharmacology Micromedex Facts and Comparisons Lexi-Comp Online RxList.com Epocrates Online Premium Epocrates Online Free Fishman et al. (1996)11

Golder et al. (2006)12

Sample size: seven relevant references Adverse effect: gastrointestinal side effects Intervention: cyclosporine Sample size: 84 included references65 Adverse effect: any Intervention: seven named antiepileptic drugs

MEDLINE

IPA

EMBASE MEDLINE

Effectiveness search TOXLINE Industry Submission Personal Communication


Results Mean relevant references, precision (%), cost per relevant item 84, 52%, $1.10 31, 31%, $0.66 22, 28%, $0.64 19, 31%, $1.96 16, 43%, $1.30 9, 32% 6, 21%, $3.6 3, 41%, $3.40 3, 30%, $1.52 Unique references (for 1 of the 10 drugs only) 62 26 10 8 7 3 2 2 Mean scores for scope (correct answer for any of 13 queries), completeness (1 = cursory, 3 = complete answer), ease of use (number of steps or clicks to reach answer) 13, 2.69, 3.62 13, 3.00, 2.69 12, 2.92, 3.00 12, 2.92, 2.00 10, 2.40, 3.00 9, 2.89, 1.78 9, 2.89, 1.78 Relevant references 6 (5 unique) 3 (2 unique)

Included references 73 (11 unique) 67 (5 unique) 57 (1 unique from SCI) 5 (1 unique) 3 (3 unique) 1 (1 unique)

181

182



Study details

Haramburu et al. (1991)13

Sample size: 151 relevant references Adverse effect: 30 named adverse effects Intervention: 30 named drug interventions

Kahn and Joseph (2004)14

Sample size: NR Adverse effect: NR Intervention:. Carbamexepine

Sources searched

and PASCAL Reference books (Martindale The Extra Pharmacopoeia, Meyler’s Side Effects of Drugs, Side Effects of Drugs Annuals, Textbook of Adverse Reactions, Clin Alert, and Reactions) MEDLINE

Micromedex Lexi-Comp Lapidus and Bond (2008)15

Sample size: NR Adverse effect: NR Intervention: garlic

Madden et al. (1977)16

Sample size: 40–171 relevant references Adverse effect: any Intervention: three named drugs

Roush et al. (1991)17

Sample size: 50–56 case reports Adverse effect: pancreatitis Intervention: enalapril Sample size: 111 unique publications (to one database) Adverse effect: any Intervention: carbamazepine Sample size: 8–24 relevant references Adverse effect: any Intervention: two named natural products

Sodha et al. (1994)18

Stone et al. (1998)19

Micromedex The Review of Natural Products Natural Medicines Natural Standard Excerpta Medica Drugdoc (EMBASE) Medlars (MEDLINE) TOXLINE ADIS De Haen† IDIS Manufacturer MEDLINE

FDA

Ciba-Geigy internal database (CG-DOC) RingDoc (now Derwent Drug File) EMBASE MEDLINE

MEDLINE

CINAHL Alta Vista EMBASE

IPA Uncover CCIS Health Reference Center

Results Relevant references 100 (68 unique) 83 (51 unique)

Total points (graded for completeness and accuracy of 21), search time (min) 11, 36 11, 57 Score (0 = no information, 3 = extensive information) 3 3 3 3 Relevant references 57 38 28 26 12 10 Cases reported 50 3 (5 references) 3 Unique publications 82 19 7 3 Relevant references 10 4 4 4 1 1 0 0




Study details

Thompson Scientific (2004)20

Sample size: 32 relevant references Adverse effect: any Intervention: Velcade (bortezomib)

Sources searched

Derwent Drug File EMBASE MEDLINE

Tourville and McLeod (1975)21

Van Putte (1991)22

Search 1 Sample size: 11–32 relevant references Adverse effect: any Intervention: amoxcillin and ampicillin Searches 2–6 Sample size: 386 relevant references Adverse effect: five named adverse effects Intervention: five named drugs

Sample size: 97–283 relevant references Adverse effect: any Intervention: two named drug interventions

Derwent Drug File MEDLINE and EMBASE Search 1: Amoxcillin and ampicillin DeHaen Drugs in Research† DeHaen Drugs in Use: Card Search† IPA IDIS DeHaen Drugs in Use: Index Search† Searches 2–6: five drugs DeHaen Drugs in Use: Card Search† IPA IDIS DeHaen Drugs in Use: Index Search†

RingDoc (now Derwent Drug File) EMBASE MEDLINE

Verheijen-Voogd (1974)23

Search 1 Sample size: 18 relevant references Adverse effect: endocarditis lenta Intervention: tooth extraction Search 2 Sample size: 48–54 relevant references Adverse effect: effects on the heart Intervention: succinlycholine

BIOSIS Previews Search 1: Tooth extraction MEDLINE

EMBASE

Results Relevant references, references with abstract 16 (44% unique), 16 14, 5 6, 2 Number of adverse effects identified 39 10 (no unique) Relevant references, Search time (min) 11, 4 7, 8 6, 15 5, 20 3, 7

122, 122 93, 99 90, 106 85, 41 Relevant references, unique references, non-relevant references 153 (111 unique), 75 86 (41 unique), 103 85 (36 unique), 87 57 (32 unique), 59 Relevant references, precision 16 (15 unique), 89% (16 ⁄ 18) 3 (2 unique), 100% (3 ⁄ 3)

Search 2: Succinylcholine EMBASE

MEDLINE

Search 1: Tooth extraction

MEDLINE EMBASE


36 (33–36 unique), 67% (36 ⁄ 54) 18 (15–18 unique), 67% (18 ⁄ 27) Relevant references not retrieved (because of errors in indexing or search formulation) 2 8

183

184

Sources of information on adverse effects, Su Golder & Yoon K. Loke Table 2 (Continued) References

Study details

Walker (2002)24

Sample size: NR Adverse effect: any Intervention: eight unspecified herbal products

Sources searched

AltMedDex Natural Medicines Comprehensive Database The Natural Pharmacist The Lawrence Review of Natural Products The Complete German Commission E Monograph Physicians Desk Reference for Herbal Medicines Tyler’s Honest Herbal

Results Number of questions answered (of 8) 7 6 5 2 2 0 0

*Searched with effectiveness search strategy only; †database now closed. CCIS, Micromedex Computerised Clinical Information Service; CPI, Conference Papers Index; FDA, Food and Drug Administration; IPA, International Pharmaceutical Abstracts; IDIS, Iowa Drug Information Service (IDIS); JICST-EPlus, Japan Science and Technology Corporation, Information Center for Science and Technology; NTIS, National Technical Information Service; NR, not reported.

answer specific queries on adverse effects.8,10,14,15,24 The search functionality and cost of searching the databases was rarely considered. However, three methodological evaluations8,9,21 considered cost implications, three recorded a score for the ease of use8,10,14 and two recorded the time spent searching.14,21 The results of these cost and ease of use elements will now be outdated as the cost and search interfaces for data sources changes rapidly over time. Database comparisons MEDLINE. In 4 of the 7,9,11–13,16–20,22,23

12 methodological evaluations that included searches of MEDLINE, MEDLINE provided the highest number of relevant records. However, these evaluations either compared MEDLINE with the International Pharmaceutical Abstracts (IPA) database only,11 compared MEDLINE together with PASCAL against reference books13 or searched for non-pharmaceutical drugs.19,23 The sources which retrieved more relevant references than MEDLINE were EMBASE (eight of nine methodological evaluations),7,9,12,16,18–20,22,23 Derwent Drug File (three of four methodological evaluations),7,18,20,22 TOXLINE (one of four

methodological evaluations),7,9,12,16 an internal database (one of one methodological evaluations)18 or industry submissions (one of three methodological evaluations).7,12,17 EMBASE.

In five of the nine methodological evaluations that included EMBASE in their assessment,7,9,12,16,18–20,22,23 searching EMBASE retrieved the highest number of relevant references.7,9,12,16,23 The only databases which retrieved more relevant references than EMBASE in any search were Derwent Drug File (three of four methodological evaluations),7,18,20,22 MEDLINE (two of nine methodological evaluations both of non-pharmaceutical drugs),19,23 and an in-house company database (one of one methodological evaluation).18 Derwent Drug File. In two of the four methodological evaluations20,22 that included searches of Derwent Drug File, Derwent Drug File provided the highest number of relevant records. Three of the four methodological evaluations indicated Derwent Drug File’s potential value over both EMBASE and MEDLINE.18,20,22 The other methodological evaluation only carried out a search on Derwent Drug File for effectiveness studies and may therefore have missed relevant references on adverse effects.7 In one methodological evaluation



searching an in-house company database retrieved more references than Derwent Drug File.18 TOXLINE. Although TOXLINE did not retrieve the highest number of relevant references in any of the methodological evaluations, in all searches it did provide at least one relevant reference and in at least two searches provided more relevant references than Iowa Drug Information Service (IDIS), PHARMLINE, PASCAL and IPA.7,9,12,16 BIOSIS previews. In the three methodological evaluations that included BIOSIS, it was found that MEDLINE and EMBASE retrieved more relevant records than BIOSIS Previews.7,9,22 However, BIOSIS Previews retrieved more relevant records than IDIS, IPA, PASCAL and PHARMLINE in the two methodological evaluations that also searched these databases.7,9 International Pharmaceutical Abstracts. Six evaluations included IPA6,7,9,11,19,21 and in all but one21 IPA retrieved either the lowest or joint lowest number of relevant records. Iowa Drug Information Service. Five methodological evaluations included searches of IDIS6,7,9,16,21. In the three methodological evaluations that also included MEDLINE, EMBASE and TOXLINE – these databases retrieved more relevant references than IDIS. In three of the four methodological evaluations that compared IDIS with IPA, searching IDIS retrieved more relevant references than IPA.6,7,9,21 Other databases. Many of the databases were only searched in one or two methodological evaluations making any comparison between studies difficult. However, it is notable that these databases did not tend to retrieve a high number of relevant references, with the exception of an in-house industry database.18

value of different databases when unique references were considered. Comparisons of non-database sources The majority of the methodological evaluations focused on bibliographic databases as sources of information, however, other sources reviewed were: drug monographs, industry submissions, internet sites, bulletins, reference checking, hand searching, personal communication, the Food and Drug Administration (FDA) website and an in-house company database. All these sources retrieved relevant references, except for Health Reference Centre19, Physicians Desk Reference for Herbal Medicines and Tyler’s Honest Herbal.24 Few of the non-database resources were included in more than one methodological evaluation and in some circumstances, the number of relevant references or case reports retrieved was not recorded. This was particularly the case when drug monographs were evaluated, these sources tend to provide summary information and were therefore more likely to be given a ‘score’ (such as, on a scale of A–D) or assessed in terms of answering specific questions regarding adverse drug reactions (ADRs).8,10,14,15,24 Although the overlap between source studies was generally low, three methodological evaluations included both Micromedex and Lexi-comp and in each evaluation Micromedex scored higher than Lexi-comp.8,10,14 The four methodological evaluations which included industry submissions or a company database also demonstrated the value of doing so.7,12,17,18 In two methodological evaluations, the manufacturer provided the most information on adverse effects,17,18 in one methodological evaluation industry submissions only retrieved less relevant records than MEDLINE and EMBASE7 and in another provided unique references.12 Comparisons of the data retrieved

Unique references Seven methodological evaluations9,11–13,18,20,22,23 recorded the number of unique relevant references for each data source. However, all the databases within these evaluations produced unique records and there was little change in the comparative

As well as the number of relevant references retrieved other attributes of the references retrieved are important in terms of their impact on the identification and quantification of adverse effects. One methodological evaluation compared the number of different adverse effects identified from the


185

186


included databases and the number of relevant references retrieved with an abstract.20 However, this did not substantially alter the results. One methodological evaluation18 found that although a search of an in-house company database retrieved the most number of relevant records, the majority of these were conference abstracts and that unique records identified in EMBASE tended to be from non-English language sources. Conference abstracts often do not contain enough information to be included in a systematic review and non-English studies may be difficult or costly to translate. Discussion The lack of consistency of outcome measures and the use of different information sources in each methodological evaluation makes direct comparisons difficult. However, some patterns in retrieval from the different sources did emerge, such as the relative value of EMBASE for ADRs and the potential value of Derwent Drug File over either MEDLINE or EMBASE. It is not surprising that searches in EMBASE tended to retrieve more relevant references for ADRs than MEDLINE, as EMBASE is a large pharmacological and biomedical database renowned for its drug-related literature. However, it would be interesting to assess the value of this database and others in retrieving non-drug-related adverse effects, especially as the only methodological evaluations to include non-pharmaceutical interventions found that MEDLINE retrieved more relevant references than EMBASE.19,23 The relative value of Derwent Drug File over EMBASE and MEDLINE merits further analysis. Derwent Drug File covers aspects of drug development, synthesis, evaluation, manufacture and use including adverse effects. The only methodological evaluation that did not identify more records on Derwent Drug File than on MEDLINE and EMBASE only searched Derwent Drug File for effectiveness data despite searching for both effectiveness data and adverse effects data on MEDLINE and EMBASE.7 Of the three methodological evaluations which indicated that searching Derwent Drug File retrieves more relevant records than other sources, one was produced by the providers of Derwent

Drug File,20 one recorded only unique records,18 and two are over 15 years old.18,22 A major limitation of this review is the dearth of recent research identified in this area. This review did not incorporate a cut-off date, however, it is realised that any papers regarding the value of different sources of information (such as databases) will quickly become out-dated as electronic and paper sources are susceptible to content change, closure ⁄ out-of-print or the emergence of new information sources. For instance, the size of electronic databases has increased dramatically since many of these methodological evaluations were published. There are many other potentially useful sources of data not covered in these studies, particularly, database gateways (such as TOXICOLOGY which searches 40 different databases: http://library.dialog.com/bluesheets/html/bloT.html), sources of conference proceedings, sources of post-marketing surveillance data (such as Vigibase), industry clinical trial registries, specialist bulletins, textbooks and journals, discussion web sites49,62,63, contacting authors and citation searching. Full-text searching was also rarely explored but may be useful for searching for adverse effects which are often addressed as secondary outcomes and therefore not contained in the title, abstract or keywords of an article. Although one methodological evaluation indicated the value of using AltaVista,19 another more popular search engine which it would be more useful to evaluate is Google or another internet source worth exploring may be Google Scholar. However, the majority of web pages (68%, 354 ⁄ 519) have been found to inadequately cover safety warnings56 and any evaluation of internet search engines is problematic because of the inconsistency of repeating searches on the Internet and the fast pace of change in Internet search engines. The majority of the methodological evaluations in this review used the number of relevant references from each database for comparison. However, this is not a sufficient criterion to evaluate the quality of a database. Other aspects such as number of unique references, search interface, the size and accuracy of the database thesaurus, precision, overlap, updating time, coverage (breadth of type of publications, years, etc.), type of data



retrieved and the range and frequency of adverse effects information identified are also important. For instance, the completeness and currency of sources will affect their appropriateness for more recently developed healthcare interventions. Furthermore, when conducting literature searches, the resource implications may need to be balanced against the potential yield of relevant information. This is particularly true when searching for adverse effects where data may be sparse and of limited quality and firm conclusions are difficult to reach. Comparing the cost of searching different data sources is difficult. One study did compare the cost from different online databases of retrieving one relevant reference.9 However, this study is relatively old and in recent years, there has been a decline in use of online dial-up databases (such as those provided by DIALOG, Datastar or STN) and a sharp increase in use of databases through the Internet, either freely or via subscriptions. Other methodological evaluations simply listed subscription rates for the electronic information resources included.8,21 However, subscription rates vary depending on provider (e.g. OVID, EBSCO), licensing agreements (e.g. large site licence), type of access (e.g. concurrent users), type of institution (e.g. private sector, charity) and format of access (e.g. CD-ROM, network) and are difficult to compare. Conclusion This review suggests that EMBASE, Derwent Drug File, MEDLINE and industry submissions may potentially provide the greatest number of relevant references for information on the adverse effects of drugs and that a range of sources may be useful in conducting a thorough search for information on adverse effects. However, many of the methodological evaluations included in this review are over 10 years old and ⁄ or include a small set of relevant references for comparison with little consideration of the effectiveness of the search strategies used. A systematic evaluation of the value of an extensive range of different sources using a number of outcome measures with a large reference set of records is urgently required to provide guidance on the current sources available for information on adverse effects.

Acknowledgements We thank Jane Burch of CRD, York for her kind assistance in screening the retrieved titles and abstracts. We also would like to thank Lindsey Myers of CRD, York for checking the database search strategies. Conflict of interest No conflicts of interest have been declared. Source of funding This research was undertaken by Su Golder as part of an MRC fellowship. The views expressed in this presentation are those of the authors and not necessarily those of the MRC. Supporting Information Additional Supporting Information may be found in the online version of this article: Appendix S1. Example search strategy. Please note: Wiley-Blackwell are not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article. References 1 Ioannidis, J. P., Mulrow, C. D. & Goodman, S. N. Adverse events: the more you search, the more you find. Annals of Internal Medicine 2006, 144, 298–300. 2 Golder, S., Loke, Y. & McIntosh, H. M. Room for improvement? A survey of the methods used in systematic reviews of adverse effects. BMC Medical Research Methodology 2006, 6, 3. 3 Golder, S., Loke, Y. & McIntosh, H. M. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. Journal of Clinical Epidemiology 2008, 61, 440–448. 4 Hatton, R. C., Haramburu, F., Begaud, B. & Godin, M. H. Comment: manual versus computer-based literature searches for adverse drug reactions. Dicp: The Annals of Pharmacotherapy 1991, 25, 875–877. 5 Aronson, J. K., Derry, S. & Loke, Y. K. Adverse drug reactions: keeping up to date. Fundamental and Clinical Pharmacology 2002, 16, 49–56. 6 Al Hefzi, A., Catania, P. N., Mergener, M. A. & Lum, B. L. Evaluation of three manual drug information retrieval


187

188


7

8

9

10

11

12

13

14

15

16

17

18

19

20

systems for investigational antineoplastic drugs. Drug Intelligence & Clinical Pharmacy 1987, 21, 196–200. Bagnall, A. M., Jones, L., Glanville, J. & Kleijnen, J. Assessing adverse events in a systematic review of atypical antipsychotics for schizophrenia. 4th Symposium on Systematic Reviews: Pushing the Boundaries, July 2002, Oxford, UK. Belgado, B. S., Hatton, R. C. & Doering, P. L. Evaluation of electronic drug information resources for answering questions received by decentralized pharmacists. American Journal of Health System Pharmacy 1997, 54, 2592–2596. Biarez, O., Sarrut, B., Doreau, C. G. & Etienne, J. Comparison and evaluation of nine bibliographic databases concerning adverse drug reactions. Dicp: The Annals of Pharmacotherapy 1991, 25, 1062–1065. Clauson, K. A., Polen, H. H. & Marsh, W. A. Clinical decision support tools: performance of personal digital assistant versus online drug information databases. Pharmacotherapy 2007, 27, 1651–1658. Fishman, D. L., Stone, V. L. & DiPaula, B. A. Where should the pharmacy researcher look first? comparing International Pharmaceutical Abstracts and MEDLINE. Bulletin of the Medical Library Association 1996, 84, 402– 408. Golder, S., McIntosh, H. M., Duffy, S. & Glanville, J. Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE. Health Information & Libraries Journal 2006, 23, 3–12. Haramburu, F. Manual versus computer-based literature searches for adverse drug reactions. The Annals of Pharmacotherapy 1991, 25, 215–216. Kahn, A. & Joseph, A. A Comparison fo the Micromedex and Lexi-Comp Medicines Information Databases. Harold Wood MIC, BHR Hospitals NHS Trust, 2004. Available from: http://www.ukmi.nhs.uk/activities/clinicalGovernance/ default.asp?pageRef=4 (cited 15 May 2008). Lapidus, M. & Bond, I. Evaluation of the four most-used alternative medicine databases to answer common herbal information questions. Journal of Electronic Resources in Medical Libraries 2008, 5, 338–345. Madden, M. & MacDonald, A. An evaluation and comparison of nine drug information retrieval services. Drug Information Journal 1977, 11, 47–59. Roush, M. K., McNutt, R. A. & Flint Gray, T. The adverse effect dilemma: quest for accessible information. Annals of Internal Medicine 1991, 114, 298–299. Sodha, R. V. & Van Amelsvoort, T. Multi-database searches in biomedicine: citation duplication and novelty assessment using carbamazepine as an example. Journal of Information Science 1994, 20, 139–141. Stone, V. L., Fishman, D. L. & Frese, D. B. Searching online and Web-based resources for information on natural products used as drugs. Bulletin of the Medical Library Association 1998, 86, 523–527. Thomson Scientific. Derwent Drug File: Definitive Drug Journal and Conference Information 2004. Available from:

21

22

23

24

25

26

27

28

29

30

31

32

33

34

http://scientific.thomson.com/media/dw/productpdfs/ ddf-compare.pdf (cited 15 May 2008). Tourville, J. F. & McLeod, D. C. Comparison of the clinical utility of four drug information services. American Journal of Hospital Pharmacy 1975, 32, 1153–1158. Van Putte, N. A comparison of four biomedical databases for the retrieval of drug literature. Health Information & Libraries Journal 1991, 3, 119–127. Verheijen-Voogd, C. & Mathijsen, A. A contribution to the comparison of the usefulness of the data bases of Excerpta Medica and MEDLARS in biomedical literature retrieval. Aslib Proceedings 1974, 26, 136–151. Walker, J. B. Evaluation of the ability of seven herbal resources to answer questions about herbal products asked in drug information centers. Pharmacotherapy 2002, 22, 1611–1615. Abuelsoud, N. N. & Alnaim, L. S. Comparative evaluation of drug information sources. Saudi Pharmaceutical Journal 2005, 13, 120–125. Akaho, E. & Miyake, A. Outcomes of online searches (EM and CA) and manual searches (IDIS and CA) for specific literature retrievals, and the need to establish a nationwide online system. Journal of Social and Administrative Pharmacy 1983, 1, 204–208. Alnaim, L. S. & Abuelsoud, N. N. Evaluation of electronic information resources for questions received by a college of pharmacy drug information center. Drug Information Journal 2007, 41, 441–448. Alper, B. S., Stevermer, J. J., White, D. S. & Ewigman, B. G. Answering family physicians’ clinical questions using electronic medical databases. Journal of Family Practice 2001, 50, 960–965. Anderson, C. A., Copestakea, P. T. & Robinsonb, L. A specialist toxicity database (TRACE) is more effective than its larger, commercially available counterparts. Toxicology 2000, 151, 37–43. Barillot, M. J., Sarrut, B. & Doreau, C. G. Evaluation of drug interaction document citation in nine on-line bibliographic databases. Annals of Pharmacotherapy 1997, 31, 45–49. Bawden, D. & Brock, A. M. Chemical toxicology searching: a collaborative evaluation, comparing information resources and searching techniques. Journal of Information Science 1982, 5, 3–18. Bell, J. E. Comparative evaluation of drugs interaction publications-1976. American Journal of Hospital Pharmacy 1976, 33, 1299–1303. Bergk, V., Haefeli, W. E., Gasse, C., Brenner, H. & MartinFacklam, M. Information deficits in the summary of product characteristics preclude an optimal management of drug interactions: a comparison with evidence from the literature. European Journal of Clinical Pharmacology 2005, 61, 327– 335. Brown, C. M. The benefits of searching EMBASE versus MEDLINE for pharmaceutical information. ONLINE & CDROM REVIEW 1998, 22, 3–8.


Sources of information on adverse effects, Su Golder & Yoon K. Loke 35 Butros, A. & McGuinness, S. Drug Information Sources: adverse effects, toxicology, poisoning. In: Boorkman, J. A., Huber, J. T., Roper, F. W. (eds). Introduction to Reference Sources in the Health Sciences. New York; London: Neal-Schuman Publishers, Inc., 2004: 174–179. 36 Choi, J. A., Sullivan, J., Pankaskie, M. & Brufsky, J. Evaluation of consumer drug information databases. Journal of the American Pharmacists Association 1999, 39, 683– 687. 37 Cluxton, R. J. J., Hunter, R. H. & Reed, R. C. Comparison of five information services as sources for bioavailability data. American Journal of Hospital Pharmacy 1979, 36, 1215–1218. 38 Costigan, A. & Wood, F. E. Comparison of search techniques (printed and comptuerised) with specific reference to the RTECS databank (Registry of Toxic Effects of Chemical Substances). Journal of Information Science 1986, 10, 79–86. 39 Day, R. P. Desktop Drug Information. Annals of Internal Medicine 1993, 119, 936–938. 40 Duffull, S. B. & Begg, E. J. Comparative assessment of databases used in a drug information service. Australian Journal of Hospital Pharmacy 1992, 22, 364–368. 41 Galt, K. A., Rule, A. M., Houghton, B., Young, D. O. & Remington, G. Personal digital assistant-based drug information sources: potential to improve medication safety. Journal of the Medical Library Association 2005, 93, 229– 236. 42 Gehanno, J. F., Paris, C., Thirion, B. & Caillard, J. F. Assessment of bibliographic databases performance in information retrieval for occupational and environmental toxicology. Occupational & Environmental Medicine 1998, 55, 562–566. 43 Haramburu, F., Elmalem, J., Pere, J. C., Begaud, B., Castot, A., Carlier, P., Carson, J. & Royer, R. J. Electronic bibliographic data bases: helpful for reviewing adverse drug reactions? The aspirin example. European Journal of Clinical Pharmacology 1989, 36, A165. 44 Ioannidis, J. P., Chew, P. & Lau, J. Standardized retrieval of side effects data for meta-analysis of safety outcomes. A feasibility study in acute sinusitis. Journal of Clinical Epidemiology 2002, 55, 619–626. 45 John, K. Medical literature searches – how many bibliographic databases are needed for sufficient retrieval in medical topics. Methods of Information in Medicine 1985, 24, 163–165. 46 Joy, M., Arana, C. & Gallo, G. Use of information sources at a university hospital drug information service. American Journal of Hospital Pharmacy 1986, 43, 1226–1229. 47 Kupferberg, N. & Hartel, L. J. Evaluation of five full-text drug databases by pharmacy students, faculty, and librarians: do the groups agree? Journal of the Medical Library Association 2004, 92, 66–71. 48 Majekodunmi, K., Moreno, B., Schwarte, S., Niederstadt, C. & Dintsios, C. M. Evaluation of Databases Containing Safety Information: A Comparative Review [Abstract]. Third Annual

49

50

51

52

53

54

55 56

57

58

59

60

61

62

Meeting Health Technology Assessment International, Adelaide, Australia, 2–5 July 2006: 166. Medawar, C., Herxheimer, A., Bell, A. & Jofre, S. Paroxetine, Panorama and user reporting of ADRs: consumer intelligence matters in clinical practice and postmarketing drug surveillance. International Journal of Risk & Safety in Medicine 2002, 15, 161–169. Medawar, C. & Herxheimer, A. A comparison of adverse drug reaction reports from professionals and users, relating to risk of dependence and suicidal behaviour with paroxetine. International Journal of Risk & Safety in Medicine 2003, 16, 5–19. Milne, A. A pilot study to examine the comparative usefulness of the Iowa Drug Information Service (IDIS) and MEDLINE in meeting the demands of a pharmacy-based drug-information service. Journal of Clinical Pharmacology 1978, 2, 227–237. Robinson, L., McIlwaine, I., Copestake, P. & Anderson, C. Comparative evaluation of the performance of online databases in answering toxicology queries. International Journal of Information Management 2000, 20, 79–87. Rosenberg, J. M., Fuentes, R. J., Kirschenbaum, H. L. & Dolinskey, D. Comparative content evaluation of the De Haen and Drugdex information systems. American Journal of Hospital Pharmacy 1983, 40, 1664–1669. Rovers, J. P., Janosik, J. E. & Souney, P. F. Crossover comparison of drug information online database vendors: dialog and MEDLARS. The Annals of Pharmacotherapy 1993, 27, 634–639. Snow, B. Online retrieval of pharmaceutical information. Drexel Library Quarterly 1982, 18, 64–83. Tatsioni, A., Gerasi, E., Charitidou, E., Simou, N., Mavreas, V. & Ioannidis, J. P. A. Important drug safety information on the internet: assessing its accuracy and reliability. Drug Safety 2003, 26, 519–527. Vidal, L., Shavit, M., Fraser, A., Paul, M. & Leibovici, L. Systematic comparison of four sources of drug information regarding adjustment of dose for renal function. BMJ 2005, 331, 263. Wilkinson, D. & Hollander, S. A comparison of drug literature coverage by Index Medicus and Drug Literature Index. Bulletin of the Medical Library Association 1973, 61, 431–432. Wright, L. L. Searching fee and non-fee toxicology information resources: an overview of selected databases. Toxicology 2001, 157, 89–110. Wukovitz, L. D. Using internet search engines and library catalogs to locate toxicology information. Toxicology 2001, 157, 121–139. Yokel, R. A., Sigell, L. T. & Nelson, E. D. A comparison of four toxicology resources in respect to rates of retrieval and time required. Journal of Pediatrics 1978, 92, 145– 148. Cross, M. Drug companies monitor web chat for adverse reaction reports. BMJ 2005, 330, 1347.


189

190

Sources of information on adverse effects, Su Golder & Yoon K. Loke 63 Herxheimer, A. & Mintzes, B. Antidepressants and adverse effects in young patients: uncovering the evidence [comment]. CMAJ Canadian Medical Association Journal 2004, 170, 487–489. 64 Bagnall, A.-M., Jones, L., Ginnelly, L., Lewis, R., Glanville, J., Gilbody, S. et al. A systematic review of atypical antipsychotic drugs in schizophrenia. Health Technology Assessment 2003, 7(13), 1–193.

65 Wilby, J., Kainth, A., Hawkins, N., Epstein, D., McIntosh, H., McDaid, C. et al. Clinical effectiveness, tolerability and cost-effectiveness of newer drugs for epilepsy in adults: a systematic review and economic evaluation. Health Technology Assessment 2005, 9(15), 172. Received 3 March 2010; Accepted 11 May 2010
