DERMATOLOGICA SINICA 31 (2013) 2e6
Contents lists available at SciVerse ScienceDirect
Dermatologica Sinica journal homepage: http://www.derm-sinica.com
REVIEW ARTICLE
Evidence-based dermatology Ching-Chi Chi 1, 2, 3, 4, * 1
Department of Dermatology, Chang Gung Memorial Hospital, Chiayi, Taiwan Centre for Evidence-Based Medicine, Chang Gung Memorial Hospital, Chiayi, Taiwan 3 Chang Gung University College of Medicine, Taoyuan, Taiwan 4 Cochrane Skin Group, Nottingham, UK 2
a r t i c l e i n f o
a b s t r a c t
Article history: Received: Apr 30, 2012 Revised: Jun 4, 2012 Accepted: Jun 5, 2012
Evidence-based medicine (EBM) has become a hot topic in medical practice, education, and research. However, a large number of senior doctors did not have an opportunity to learn EBM in medical schools. Firstly, this article addresses the history of EBM and the principle of practicing EBM, i.e., asking, acquiring, appraisal, application, and auditing. Secondly, this article also provides a brief introduction to evidencebased dermatology and compares the introduction of clinical practice guidelines between Europe, the UK, and the US. Finally, this article addresses the present condition and future perspective of evidencebased dermatology in Taiwan. Copyright Ó 2012, Taiwanese Dermatological Association. Published by Elsevier Taiwan LLC. All rights reserved.
Keywords: clinical practice guideline (CPG) Cochrane Skin Group (CSG) evidence-based dermatology evidence-based medicine (EBM) health technology assessment
Introduction Evidence-based medicine (EBM) has become a hot topic in medical practice, education, and research. However, a large number of senior doctors did not have an opportunity to learn EBM in medical schools. The idea of using the evidence for medical practice could be traced back to the time of ancient Greece.1 However, it was not until the postwar period that EBM began to appear in modern medicine. Archibald L. (Archie) Cochrane (1909e1988), a British clinical epidemiologist, was the first to advocate the principle of EBM. In his most celebrated book entitled “Effectiveness and Efficiency. Random Reflections on Health Services,”2 he proposed that the health-care resources are always limited and should be “efficiently” utilized based on evidence to provide “effective” health care to as many people as possible. The best evidence of the effectiveness of interventions is from randomized controlled trials (RCTs). The same principle applies not only to treatments, but also to diagnostic tests and screening procedures. Inspired by Archie Cochrane, the academic centers in the UK and Canada, in particular the Universities of Oxford and York in England and McMaster University in Ontario (Canada), established the explicit systematic methods to obtain the “best evidence.” One of
* Department of Dermatology, Chang Gung Memorial Hospital, Chiayi, 6, Sec West, Chia-Pu Road, Puzih, Chiayi 61363, Taiwan. Tel.: þ886 5 362 1000x2754. E-mail addresses:
[email protected],
[email protected].
the most important methods of acquiring the best evidence is meta-analysis, i.e., using a statistical method to obtain a pooled intervention effect estimate from independent RCTs of adequate quality to minimize biases and random error. In 1904, Karl Pearson, a British pioneer statistician, first proposed to combine the results of individual studies to minimize random errors.3 In 1920s, Ronald Fisher, another renowned British statistician, developed a statistical method of meta-analysis which is known as Fisher’s combined probability test.4 Gene Glass, an American statistician, coined the term “meta-analysis” and illustrated its use in 1976. Sir Iain Chamlers, a pioneer in EBM, was the first to employ meta-analysis in clinical medicine successfully. In 1990, his group based in Oxford published the most famous meta-analysis entitled “The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials.”5 By 1989, there had been seven RCTs examining the effects of prenatal administration of corticosteroid to women who were about to give birth prematurely. However, none of the studies had adequate power to conclude the effects of corticosteroid on infant mortality. Sir Iain Chamlers conducted a meta-analysis and found that a short, inexpensive course of corticosteroids given to these women could prevent infant respiratory distress syndrome and reduce the odds of the babies of these women dying from the complications of immaturity by 30e50%.5 Because no relevant meta-analysis had been published until 1989, most obstetricians did not realize that this simple treatment was so effective. Tens of thousands of premature babies might thus have died unnecessarily. This is just
1027-8117/$ e see front matter Copyright Ó 2012, Taiwanese Dermatological Association. Published by Elsevier Taiwan LLC. All rights reserved. http://dx.doi.org/10.1016/j.dsi.2012.06.002
C.-C. Chi / Dermatologica Sinica 31 (2013) 2e6
one of many examples of loss of lives resulting from failure to perform up-to-date systematic reviews (SRs) of RCTs of health care. In the meantime, the McMaster University group led by Professors Gordon Guyatt and David Sackett in Canada advocated the EBM paradigm for medical practice and education. They officially announced the term “EBM” in their article published in 1992, and defined EBM as “the conscientious, explicit, and judicious use of current best evidence in making clinical decisions about the care of individual patients.”6 Practice of EBM The whole idea of EBM is to use the best evidence to resolve clinical uncertainty and to provide effective and efficient health-care interventions to promote the well-being of people. The complete practice of EBM is composed of five basic steps (also known as “5As”): (1) asking a clinical question, (2) acquiring the evidence by a thorough search of the literature, (3) appraisal of the validity and relevance of the retrieved articles, (4) applying the evidence to clinical practice, and (5) auditing the performance of the practice. (1) Asking a clinical question In the first step, clinicians encounter a scenario where decisions have to be made to resolve a clinical problem, and become aware of the limitation in knowledge. EBM formulates a focused clinical question using the PICO structure of four elements: patient/population/problem (P), intervention (I), comparator (C), and outcome (O). For example, a dermatologist was frequently asked about the efficacy of probiotic intake in atopic eczema by parents of affected children. This dermatologist then organized this uncertainty into a structured clinical question: “Are probiotics useful in treating atopic eczema?” Thus, the four elements of the question are atopic eczema as P, probiotics as I, placebo or no interventions or another intervention as C, and efficacy as O. It is important to identify the four elements, because in this process one will have a clear understanding of the specific question she/he is asking and these elements will be used to design a search strategy for finding relevant evidence. (2) Acquiring the evidence In the second step, the evidence is acquired by a thorough search of the literature. This step is one of the major differences between EBM and traditional narrative review. Narrative review articles rely on the review authors to collect evidence and therefore may be biased due to authors’ preference, access to bibliographies, and conflicts of interest. EBM uses a transparent systematic way of collecting evidence by a rigorous search of bibliographic databases. The electronic databases frequently used include MEDLINE/ PubMed, EMBASE, the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, and the Latin American and Caribbean Health Science Information (LILACS) database. The Airiti Library is a Taiwanese bibliographic database which should be searched for obtaining relevant local evidence. As the number of medical publications from China has been increasing rapidly over the past few years, Chinese bibliographic databases may be searched as well. However, there is a concern for the quality of Chinese studies which the readers shall pay attention to.7 Trials with significant statistical results are often accepted early and easily for publication than those with nonsignificant results.8 This “publication bias” leads us to find positive studies more easily than negative studies. Therefore, the International Committee of Medical Journal Editors announced in 2004 that for a clinical trial to be considered for publication in one of its journals,
3
the trial must be registered by September 2005 in a public clinical trials register.9 The mandatory registration of trials before commencement keeps a record of all clinical trials irrespective of whether the results of a study are published or not. The Cochrane Collaboration encourages its authors to search trial registers and to correspond with trialists and/or pharmaceutical companies for unpublished data in order to minimize publication bias.10 A comprehensive search of the electronic databases needs a properly designed search strategy. This search strategy is usually a Boolean combination of (“P” AND “I” AND “O”) or just (“P” AND “I”), the aforementioned elements of the clinical question proposed. Medical subject headings (MeSH) and free text words are often used to increase the sensitivity of the search strategy. To design a proper search strategy, balancing sensitivity and specificity needs frequent practice (trial and error) and the learning curve is sometimes quite long. Inexperienced people may seek the assistance of experienced systematic reviewers or information specialists. (3) Appraisal of the evidence Mainly based on study design, the Oxford Centre for EBM proposed a classification of levels of evidence (Figure 1).11,12 We usually only adopt the evidence from SRs (Level 1) and RCTs (Level 2), and use the evidence from nonrandomized studies only when no evidence of higher level is available. After identifying relevant RCTs from the search results, the individual quality of the studies has to be appraised. This critical appraisal usually covers three aspects of the evidence: validity, clinical relevance, and applicability. There are a number of critical appraisal tools, with the tools developed by the UK Critical Appraisal Skills Programme and Oxford Centre for EBM being the most widely used ones in Taiwan.13,14 With regard to the risk of bias, the Cochrane Collaboration has its own tool.15 The Cochrane tool is the benchmark for assessing the risk of bias in RCTs, but it only examines the validity and is not an easy-to-use tool for inexperienced people. The dermatologist who proposes the aforementioned clinical question has searched PubMed and found a relevant German RCT out of 116 citations.16 Before adopting the study results for clinical practice, a critical appraisal of the RCT needs to be done. I have devised a critical appraisal sheet which can easily be used by beginners and show the results of critical appraisal for this RCT in Table 1. The evidence from this RCT was judged valid and revealed that probiotics were ineffective in treating atopic eczema.
Figure 1 Levels of evidence pyramid (for treatment benefit).
4
C.-C. Chi / Dermatologica Sinica 31 (2013) 2e6
Table 1 Critical appraisal tool. Appraisal
Comments
Validity (1) Was the assignment of patients to interventions randomized?
Low risk of bias
(2) Was the allocation concealed? (3) Were the groups similar at the start of the trial?
Unclear risk of bias Low risk of bias
(4) Was follow-up of patients sufficiently long and complete?
Low risk of bias
(5) Were all patients analyzed in the groups to which they were randomized? (Intention-to-treat analysis)
Unclear risk of bias
(6) Were measures objective or were the patients and clinicians kept “blind” to which intervention was being received? (7) Were groups treated equally, apart from the experimental therapy
Low risk of bias
The participants were randomized to either the probiotic or placebo group No relevant description No significant differences in baseline characteristics and severity between the probiotic and placebo groups The treatment duration of the study was 12 weeks, which was longer than those of previous studies 102 out of the 106 enrolled subjects completed the trials, with 4 being withdrawn from the trial due to protocol violation. The data on the 102 participants were used for analysis. Therefore, this study used available case analysis, instead of intention-to-treat analysis Double-blind study design
Clinical relevance (1) What is the magnitude of the intervention effect?
(2) How precise is the estimate of the intervention effect? Applicability (1) Is our patient so different from those in the trial that its results cannot apply? (2) Is the intervention feasible in our setting? (3) What are our patient’s potential benefits and harms from the therapy?
(4) What are our patient’s values and expectations for both the outcome we are trying to prevent and the intervention we are offering?
Low risk of bias
The participants in both groups were encouraged to apply ordinary skin care and avoid allergens or irritants
No therapeutic effects of probiotics
No significant effects of probiotics on symptom load, SCORAD index score, etc. The primary outcome parameter e the mean reduction of the symptom load score (SLS) at the end of week 12 e was 4.94 (SD 14.73; median 7.1, range 0.92e8.96) in the probiotics group and 8.53 (SD 10.90; median 9.5, range 5.37e11.70) in the control group (P ¼ 0.144) 54 vs. 48 participants
Low precision No Yes Benefits: No significant potential benefits Harms: No significant potential harms Patient’s values: his atopic eczema is annoying and needs to be treated Patient’s expectation: wish for an effective and safe treatment
(4) Applying the evidence to clinical practice After having the evidence at hand, the fourth step is to judge the applicability of the evidence to specific patients. The evidence is from a trial on a group of German children and thus may have limited external validity. However, there is no convincing reason indicating a different response of Taiwanese children to probiotics. The aforementioned dermatologist uses his expertise to judge that this evidence could be applied to his patients. He decides to inform parents of affected children that the current evidence found no efficacy of probiotics in treating established atopic eczema and there is no scientific ground for the children to take probiotics.
Act If pilot works, spread horizontally, otherwise replans
Plan Steps 1-3 of EBM, i.e. asking, acquiring, and appraisal
(5) Auditing the performance of the practice The fifth step of EBM is to audit and get feedback of the clinical practice to improve performance. This step is similar to the “check phase” in the planedoeactecheck cycle that is frequently used in quality improvement. After agreeing on an act to be implemented, efficacy and/or quality outcomes should be introduced and used for evaluating the performance of the evidence-based practice. If the practice works, standard operating protocols (SOPs) and/or guidelines shall be developed and horizontally spread to colleagues to improve the quality of the health-care system. If the practice does not work, a recheck of the barriers to implementation and an analysis of the reasons for failure should be done. Modification of the practice should be done and a new planedoeactecheck cycle follows (Figure 2).
Check
Do
Step 5 of EBM, auditing
Step 4 of EBM, application
Figure 2 The relationship between planedoeactecheck cycle and evidence-based medicine (EBM).
C.-C. Chi / Dermatologica Sinica 31 (2013) 2e6
As mentioned previously, EBM researchers utilize a systematic and explicit method, SR, to acquire, appraise, and synthesize the best evidence on a clinical question, with the aim of reducing biases and random errors. SRs are transparent and reproducible, which are in contrast to traditional narrative reviews. Narrative reviews are often written by prestigious experts in a specific field. Narrative reviews frequently cover a broad topic and serve as articles for continuing medical education. However, they are based on authors’ knowledge and preference of available data. The process of preparing narrative reviews is nontransparent and subjective. The results of narrative reviews are thus irreproducible and subjective to biases. Therefore, SRs are more methodologically rigorous than narrative reviews. In SRs, meta-analyses may be used to obtain a statistically pooled effect estimate. Cochrane reviews are conducted under the guidance and surveillance of Cochrane Review Groups, and thus have a higher quality than non-Cochrane SRs.17 Cochrane reviews are published in the Cochrane Database of Systematic Reviews. To overcome the language barrier in using the best evidence in health care in Taiwan, the National Health Research Institute has prepared Mandarin abstracts of over 3,500 Cochrane reviews and provides them at a Mandarin website: http://clc.nhri.org.tw/admin/ clcmain1.aspx.18 Having illustrated the principle, EBM, however, is not “cookbook” medicine. It should be kept in mind that evidence is the base, not the host, for making decisions. It requires the integration of the best research evidence with clinicians’ expertise, patients’ unique values, and circumstances for making clinical decisions (Figure 3). Evidence-based dermatology Evidence-based dermatology is the utilization of EBM in dermatology. The Cochrane Skin Group (CSG) is an international organization which was established in 1997 with the aim of preparing, maintaining, and promoting the accessibility of Cochrane reviews of the effects of health care interventions in dermatology.19 To date, the CSG has over 100 Cochrane reviews covering various topics, ranging from efficacy of interventions to the safety of drugs.20,21 Interested readers may find further details from the book authored by the CSG Editors.22 Evidence-based clinical practice guidelines (CPGs) can assist busy clinicians in making clinical decisions based on the best evidence and skipping the tedious process of acquiring and appraisal of evidence. CPGs can also promote efficient use of limited health-care resources. In countries where patients are mainly cared for by general practitioners and are transferred to specialists only when necessary, CPGs are very important as they provide general
Figure 3 Evidence-based medicine is not cookbook medicine but requires the integration of the best research evidence with clinicians’ expertise, patients’ unique values, and circumstances for making clinical decisions.
5
practitioners with a useful guide in effectively and efficiently delivering health care services. However, clinicians should be aware that the CPGs reflect the best evidence available at the time they are prepared. Caution should be exercised in interpreting the data; the results of future studies may require alteration of the conclusions or recommendations in the CPGs. Adherence to CPGs does not guarantee treatment success in every clinical situations (remembering that EBM is not cookbook medicine but instead needs to integrate evidence, expertise, and patient’s expectation and circumstances). It may be necessary or even desirable to depart from the CPGs in special circumstances. Therefore, CPGs shall not be interpreted as standards of care and thus deviating from them should not be necessarily deemed negligent in litigation. The European Dermatology Forum was founded in 1997 by a group of leading European Professors of Dermatology. The European Dermatology Forum has an explicit SOP for developing CPGs and has published over 30 CPGs. The British Association of Dermatologists (BAD) also has an SOP for developing CPGs which is accredited by the NHS Evidence.23 The BAD has a Therapy and Guidelines Subcommittee and has published 30 CPGs. Compared with the BAD, the American Academy of Dermatology (AAD) has a slower pace in developing CPGs. The AAD now has a Clinical Guidelines Committee, but only has nine published CPGs (with six on psoriatic diseases). Rise of evidence-based dermatology in Taiwan Researchers in Taiwan have joined the CSG since 2006 and have published two Cochrane reviews and are preparing two other reviews.20,21,24 Dr Ching-Chi Chi (Chang Gung Memorial Hospital, Chiayi) started to serve as a CSG Editor since 2011. Other dermatologists have published a few non-Cochrane SRs.25e27 Dr Yu-Hung Wu (Mackay Memorial Hospital, Taipei) has developed the Mandarin version of the Fresno test for evaluating EBM education. Drs Wu and Chi are currently Directors of the Centers for EBM in their hospitals, respectively. The teams organized by them have won a number of gold medals in the yearly EBM Application Contest of the Healthcare Quality Improvement Campaign held by the Taiwan Joint Commission on Hospital Accreditation. Dr Yao-Kun Chiu was the Director of the Centers for EBM in Mennonite Christian Hospital, Hualien. All the three dermatologists are devoted to EBM education. Future perspectives In Taiwan, EBM has been introduced in medical education for years, and is now an essential component of postgraduate year training. Compared with the older generation, the majority of young dermatologists have received EBM education in medical schools. This year, the written test for Taiwan’s Board of Dermatology Specialist for the first time contained a question on the levels of evidence. As more young dermatologists are aware of and involved in EBM, it is expected that the output of evidence will increase and dermatologists will have more evidence support in decision making. Compared with the BAD and AAD, the Taiwanese Dermatological Association is slow in developing CPGs and does not have a guideline committee yet. However, as the financial constraints of Taiwan’s National Health Insurance have become tighter over the past few years, the governing body has introduced evidencebased health technology assessment in policy making, such as the reimbursement approval and criteria of ustekinumab.28 There has been a call for the Taiwan Government to establish an organization like the UK National Institute for Health and Clinical Excellence to meet the increasing need for health technology assessment. Dermatologists in Taiwan have been practicing based
6
C.-C. Chi / Dermatologica Sinica 31 (2013) 2e6
on evidence to a certain degree. Undoubtedly, as other clinicians, dermatologists will be directed to a more evidence-based practice in the future. References 1. Woolf SH, George JN. Evidence-based medicine. Interpreting studies and setting policy. Hematol Oncol Clin North Am 2000;14:761e84. 2. Cochrane AL. Effectiveness and efficiency. Random reflections on health services. London: Nuffield Provincial Hospitals Trust; 1972. 3. Pearson K. Report on certain enteric fever inoculation statistics. Br Med J 1904;2:1243e6. 4. Fisher RA. Statistical methods for research workers. Edinburgh: Oliver and Boyd; 1925. 5. Crowley P, Chalmers I, Keirse MJ. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynaecol 1990;97:11e25. 6. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312:71e2. 7. Watts G. Data rising from the East. BMJ 2011;343:d7480. 8. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457e65. 9. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal. N Engl J Med 2004;351:1250e1. 10. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley-Blackwell; 2008. 11. OCEBM Levels of Evidence Working Group. The Oxford 2011 Levels of Evidence. Available at http://www.cebm.net/index.aspx?o¼5653. [accessed 5.12.11]. 12. Chi CC. Advances in EBM: Oxford CEBM Levels of Evidence 2011. Evidence Counts 2012;3:4e5. 13. Critical Appraisal Skills Programme. Available at http://www.phru.nhs.uk/ Pages/PHD/CASP.htm. [accessed 30.11.11]. 14. Oxford Centre for Evidence-Based Medicine. Critical Appraisal Sheets. Available at http://www.cebm.net/index.aspx?o¼1157. [accessed 30.11.11].
15. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. 16. Grüber C, Wendt M, Sulser C, et al. Randomized, placebo-controlled trial of Lactobacillus rhamnosus GG as treatment of atopic dermatitis in infancy. Allergy 2007;62:1270e6. 17. Collier A, Heilig L, Schilling L, Williams H, Dellavalle RP. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol 2006;155:1230e5. 18. Chiu YW, Weng YH, Lo HL, Shih YH, Hsu CC, Kuo KN. Impact of a nationwide outreach program on the diffusion of evidence-based practice in Taiwan. Int J Qual Health Care 2010;22:430e6. 19. Collier A, Johnson KR, Delamere F, Leonard T, Dellavalle RP, Williams H. The Cochrane Skin Group: promoting the best evidence. J Cutan Med Surg 2005;9:324e31. 20. Chi CC, Kirtschig G, Baldo M, Brackenbury F, Lewis F, Wojnarowska F. Topical interventions for genital lichen sclerosus. Cochrane Database Syst Rev 2011:CD008240. 21. Chi CC, Lee CW, Wojnarowska F, Kirtschig G. Safety of topical corticosteroids in pregnancy. Cochrane Database Syst Rev 2009:CD007346. 22. Williams H, Bigby M, Diepgen T, Herxheimer A, Naldi L, Rzany B. Evidencebased dermatology. 2nd ed. Oxford: Blackwell Publishing; 2008. 23. Bell HK, Ormerod AD, BAD Therapy and Guidelines Subcommittee, September 2008. Writing a British Association of Dermatologists clinical guideline: an update on the process and guidance for authors. Br J Dermatol 2009;160: 725e8. 24. Lee C, Chi C-C, Hsieh S-C, et al. Interventions for treatment of herpes simplex labialis (cold sores on the lips). Cochrane Database Syst Rev 2011: CD009375. 25. Yang YW, Tsai CL, Lu CY. Exclusive breastfeeding and incident atopic dermatitis in childhood: a systematic review and meta-analysis of prospective cohort studies. Br J Dermatol 2009;161:373e83. 26. Huang YC, Li YC, Chen TJ. The efficacy of intravenous immunoglobulin for the treatment of toxic epidermal necrolysis: a systematic review and meta-analysis. Br J Dermatol 2012;167:424e32. 27. Chiu HY, Tsai TF. Topical use of systemic drugs in dermatology: a comprehensive review. J Am Acad Dermatol 2011;65. 1048.e1ee22. 28. Center For Drug Evaluation. Health technology assessment report on ustekinumab, 2011.
