DATA SOURCES Articles were identified by searching MEDLINE and EMBASE databases. STUDY SELECTION Review articles published between 1986 and ...
RESEARCH *...*-..eeee
Comprehensiveness and bias n reporting cl incal tria s Study of
ofpneumococcal vaccine effectiveness
BRIAN G. HUTCHISON, MD, MSC ANDREW D. OXMAN, MD, MSC STEPHEN LLOYD, MD, MSC PURPOSE To assess the extent to which relevant controlled clinical trials are cited and summarized in review articles, and to determine whether citation of relevant clinical trials is biased as to study results. DATA SOURCES Articles were identified by searching MEDLINE and EMBASE databases. STUDY SELECTION Review articles published between 1986 and 1988 on the clinical effectiveness of pneumococcal vaccine. DATA EMACTION Proportion ofrelevant clinical trials cited and whether citation is biased by study results. DATA SYNTHESIS The proportion of relevant primary studies cited per review article ranged from 0% to 36% (mean 9%). The number of trials cited per review ranged from zero to six (mean 1.2). In nine of 17 reviews, no clinical trials were cited. Study populations and outcome(s) were specified and results presented quantitatively for 0 to 27% of relevant trials per review (mean 6%). Unsupportive trials were almost twice as likely to be cited as supportive trials. CONCLUSIONS Reporting of the results of relevant clinical trials in reviews of pneumococcal vaccine effectiveness is incomplete. Our findings suggest a need for greater scientific rigour in preparing, reviewing, and editing review articles.
BUT tvaluer A quel point les essais cliniques controles pertinents sont mentionnes et resumes dans les articles de synthese et determiner si la mention des essais cliniques pertinents presente des biais comparativement aux resultats de l'etude elle-meme. SOURCE DES DONNEES Articles identifies par une recherche dans les banques de donnees MEDLINE et EMBASE. SELECTION DES ARTICLES-,Articles -de synthese publies entre 1986 et 1988 sur l'efficacit6 clinique du vaccin
Dr Hutchison is an Associate Professor in the Department of Clinical Epidemiology and Biostatistics and the Department ofFamily Medicine and is afaculy member at the Centrefor Health Economics and Policy Analysis at McMaster Universi_y in Hamilton, Ont. Dr Oxman is a Senior Researcher in the Department ofHealth Services Research at the National Institute ofPublic Health in Oslo, Nownay. Dr Lloyd is Associate Professor in the Department of Family Medicine at McMaster
Universit_.
antipneumococcique.
CUEILLETTE DES DONNEES Proportion des essais cliniques pertinents mentionnes et degre de biais entre la citation et les veritables resultats de l'etude. SYNTHESE DES DONNEES La pioportion des etudes primaires pertinentes mentionne'es dans chacun des articles de synthese varie de 0% a 36% (moyenne 9%). Le nombre des essais mentionnes dans chacun des articles de synthese varie de zero a six (moyenne 1,2). Dans neuf des 17 articles de synthese, aucun essai clinique n'est mentionne. On y specifie les populations etudiees, et les conclusions et les resultats sont presentes quantitativement pour 0% a 27% des essais pertinents dans chacun des articles de synthese (moyenne 6%). Les essais defavorables etaient deux fois plus susceptibles d'etre cites que les essais favorables. CONCLUSIONS Dans les articles de synthese portant sur l'efficacite du vaccin antipneumococcique, les rapports concernant les resultats des essais cliniques pertinents sont incomplets. Nos constatations indiquent la necessite d'une plus grande rigueur scientifique dans la preparation, l'analyse et la redaction des articles de synthese. Can Fam Physidan 1995;41:1356-1360. IVEN THE VAST AND GROWING identified serious deficiencies in scienpool of primary research, tific quality.`3 Each of these studies clinicians, health policy evaluating the quality of review articles makers, and public health has examined articles covering many practitioners are heavily dependent on topics: an approach that provides an summaries of existing research for overall assessment of the quality of information and guidance. However, review literature but does not address evaluations of review articles have the extent to which relevant primary
1356 Canadian Family Physician VOL 41: August 1995
RESEARCH
research is cited and summarized or whether bias exists in the citation of studies. Two studies have examined the issue of citation bias. Gotzsche4 has identified biased citation of published trials in reports of clinical trials of nonsteroidal anti-inflammatory drugs for rheumatoid arthritis. Ravnskovs found that trials of cholesterol-lowering strategies, considered by their authors to support intervention, were much more likely to be cited in subsequent publications than trials that did not support intervention. This article reports on citation of clinical trials in review articles dealing with the clinical effectiveness of pneumococcal vaccine. We examined this material in the course of assessing the need for, and subsequently conducting, a meta-analysis of controlled clinical trials of pneumococcal vaccine.
METHODS We searched for review articles in MEDLINE and EMBASE bibliographic databases with the following criteria: English language publication; published between 1986 and 1988; review article, editorial or commentary, economic analysis, or policy statement; and a conclusion was reached regarding the clinical effectiveness of pneumococcal vaccine. EMBASE was searched using the terms pneumococcal or pneumonia and vaccin: or immunization, and review. MEDLINE was searched two ways: using MeSH headings *bacterial vaccine and explode pneumococcal infections/* prevention and control, or streptococcus pneumoniae and review; and using MeSH headings *bacterial vaccines and review. Citations that seemed relevant were retrieved and evaluated in relation to our inclusion criteria by two assessors. We also included a policy statement on pneumococcal vaccine prepared by the United States Immunization Practices Advisory Committee (ACIP)6
because we thought it might influence clinical practice. Target populations discussed and clinical trials of pneumococcal vaccine cited in each review were identified and summarized. The pool of relevant clinical trials was identified through a search of EMBASE and MEDLINE, review of reference lists, and correspondence with organizations producing immunization guidelines and with first authors of primary studies. Studies were included if they assessed clinical outcome among humans; were randomized, or quasi-randomized, controlled trials; and were published in a journal, book, or government document. For each overview, we calculated the proportion of relevant trials of pneumococcal vaccine effectiveness cited by the author(s). The denominator (all relevant clinical trials) was determined separately for each overview. Only those trials published at least 1 year before the date of publication of the overview were included. However, if a more recent trial was cited, that study was added to the denominator. For overviews with a restricted target population (eg, children or adults), the denominator included only studies of that population. However, if the authors of such overviews cited trials in other populations, all studies of those additional population groups were included. Studies of nonelderly adult populations were considered relevant to overviews of studies of the elderly. The relationship between number of relevant clinical trials and proportion of relevant trials cited was examined using linear regression analysis. We assessed citation bias by comparing the proportion of relevant supportive and nonsupportive trials cited in each review and in all reviews combined. Trials were classified as supportive if significantly fewer vaccinated than control subjects experienced the main study outcome (P
