ARTICLES
nature publishing group
Quality of Methods for Assessing and Reporting Serious Adverse Events in Clinical Trials of Cancer Drugs SM Belknap1,2,3, CH Georgopoulos1, DP West2,3, PR Yarnold3,4 and WN Kelly The validity of information regarding drug toxicity in humans depends on the quality of the methods and instruments used to assess adverse drug events (ADEs). This study evaluates the quality of instruments used to assess and report ADEs to institutional review boards (IRBs) at US cancer centers. Forms from all 49 National Cancer Institute (NCI)-designated centers were assessed for utility in abstracting event type, severity, and causality; patient demographics; safety monitoring; and consequent changes in the conduct of the relevant study. Of the 55 items considered essential for ADE reporting, one item (event description) was present on all the forms. Seventy-eight percent of the instruments prompted for global introspection of the investigator, a method known to be unreliable. Of the 34 items that our panel of experts considered essential for event description, the median number of items present was four (domain = 1–11). The use of a validated tool to describe and assess event type, severity, and causality may lead to more timely, accurate identification of safety signals in cancer treatment. Timely identification of previously unknown toxicities of new drugs is an important goal that has yet to be attained. Of the 548 drugs introduced to the US market between 1975 and 1999, an estimated 20% were withdrawn or acquired a new “black box” warning in the 25-year period following initial approval by the US Food and Drug Administration (FDA).1 The 25 safety revisions to cancer drug package inserts between 2000 and 2002 occurred after a median interval of 5.4 years after FDA approval, and one of these safety revisions occurred 36 years after FDA approval.2 These delays have been attributed to multiple causes, including difficulties in distinguishing the effects of drug toxicity from those of disease progression or comorbidity,3,4 underreporting of postmarketing events in clinical settings,5,6 inadequate FDA funding,7,8 and conflicts of interest between sponsors and investigators.9 The nature of the methods and instruments used by clinical trial investigators to detect, assess, and report serious adverse drug events (ADEs) has not been studied as a factor in ADE signal detection. Here, we assess the extent to which institutional review board (IRB) ADE forms prompt for entry of the data necessary for description of event type and severity using the Common Terminology Criteria for Adverse Events
(CTCAE v4.0). Additionally, we assess the utility of these forms as a means of prompting for the data necessary for determining the Naranjo Adverse Drug Reaction Probability Scale score (a validated tool for assessing the causality of an ADE) and also for the Bradford Hill criteria (a well-known set of criteria used for assessing causality in epidemiology).10–13 We also assess the methodology used to abstract key regulatory information, such as patient demographic data and safety monitoring, and information about changes to the study protocol and related documents that were prompted by the event. RESULTS
Of the 55 items considered for ADE evaluation in this study, one item (event description) was present on the IRB ADE forms of all 49 of the National Cancer Institute (NCI) centers. Seven of the 49 centers (14%) had IRB ADE forms prompting for event descriptors of the NCI CTCAE v4.0 taxonomy for description of event type and severity. Only 20 (41%) of the forms prompted for an assessment of event severity, and only half of the forms prompted for patient outcome. None of the forms used a validated method for assessing causality, with 38 of the 49 (78%) implicitly prompting for
1Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA; 2Department of Dermatology, Northwestern University
Feinberg School of Medicine, Chicago, Illinois, USA; 3Robert H. Lurie Comprehensive Cancer Center, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA; 4Department of Emergency Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA. Correspondence: SM Belknap (
[email protected]) Received 7 January 2010; accepted 8 April 2010; advance online publication 23 June 2010. doi:10.1038/clpt.2010.79 CLINICAL PHARMACOLOGY & THERAPEUTICS | VOLUME 88 NUMBER 2 | AUGUST 2010
231
ARTICLES global introspection by the investigator—a method known to be unreliable.4,14,15 Only one-third of the forms provided the domain of terms recommended by the FDA for assessment of causality by global introspection: (“unrelated,” “unlikely,” “possibly,” “probably,” and “definitely”).16 More than one-quarter of the forms failed to prompt for even 1 of the 10 Naranjo causality items.
Of the 49 forms, one-quarter did not prompt for whether the adverse event resulted in consent-form revisions—an important ethical and regulatory matter. Two-thirds of the reporting forms did not prompt for any patient demographic and clinical descriptors, and one-quarter did not prompt for a subject identification code. These omissions would obscure (or sever) the link
Table 1 Adverse event description utility score for NCI Comprehensive Cancer Center ADE reporting to IRBs Number of forms (from 49 centers)
Item
Percentage of forms
Event description (type, expectedness, seriousness, and patient outcome) and event severity Asks for classification of event type (using CTCAE v3.0 criteria)
7
14
Asks for event “expectedness”
28
57
”Expectedness” defined on form
10
20
Asks for “serious event”
35
71
”Serious event” defined on form
10
20
Asks for “patient outcome”
25
51
”Patient outcome” defined on form Asks for event “severity” ”Severity” defined on form
0
0
20
41
1
2
Event causality Asks for PI’s assessment of the association’s causality
38
78
Asks for strength of association (BH1)—toxic effect improved with drug discontinuation or treatment (N3)
1
2
Asks for strength of association (BH1)—toxic effect reappeared when placebo administered (N6)
0
0
Asks for strength of association (BH1)—similar event occurred in previous exposure to same agent (N9)
0
0
Asks for strength of association (BH1)—alternative causes that could have caused the event (N5) Asks for consistency of association (BH2)—previous conclusive reports of this event (N1)
2
4
12
24
Asks for specificity of association (BH3)—suspected agent causes other events
0
0
Asks for temporality of association (BH4)—toxic effect appeared after agent was administered (N2)
7
14
Asks for biological gradient of association (BH5)—toxic effect reappeared with re-administration of agent (N4)
0
0
Asks for biological gradient of association (BH5)—severity change correlated with dosage change (N8)
1
2
Asks for plausibility of association (BH6)—mechanism of agent’s action known
0
0
Asks for plausibility of association (BH6)—agent’s mechanism of action supports occurrence of toxic effect
0
0
Asks for coherence of association (BH7)—body of scientific information supports occurrence of toxic effect
0
0
Asks for experimental evidence of association (BH8)—toxic concentration of agent found in body fluids (N7)
2
4
Asks for experimental evidence of association (BH8)—objective evidence of the toxic effect provided (N10)
26
53
Asks for analogous associations (BH9)—what events occur with agents from same drug class
0
0
Asks for analogous associations (BH9)—event resembles those of agents from same drug class
0
0
Asks for patient’s sex
15
31
Asks for patient’s age
13
27
Asks for patient’s weight
1
2
Asks for patient’s height
0
0
Asks for patient’s underlying disease/condition and other comorbidity
7
14
Asks for list of other medications taken concurrently
5
10
Asks for patient’s medical history
7
14
Asks for relevant lab results from patient’s medical records
6
12
Patient population descriptors
IRB adverse event report forms were reviewed for their collection of all variables listed. The “adverse event description utility” score is based on these variables. In the two columns on the right, the number and percentage of forms that collected a given variable are shown. See Supplementary Table S7 online for further clarification. ADE, adverse drug event; CTCAE, Common Terminology Criteria for Adverse Events; IRB, institutional review board; NCI, National Cancer Institute; PI, principal investigator. 232
VOLUME 88 NUMBER 2 | AUGUST 2010 | www.nature.com/cpt
ARTICLES Table 2 Oversight action utility score for NCI Comprehensive Cancer Center ADE reporting to IRBs
40
Item
30
Number Percentage of forms of forms
Data safety monitoring information Minimal data set
20
10
0
Better
Worse
Figure 1 Adverse event description utility score of institutional review board adverse drug event report forms at National Cancer Institute cancer centers. The score was a sum of the items within the categories of “event description (type, expectedness, seriousness, and patient outcome) and event severity,”“causality,” and “patient demographic descriptors,” which were present on each form (see Table 2). A perfect score required the presence of all 34 relevant items. As shown, the median number of items present was four (domain = 1–11).
Asks for subject identification
36
73
Asks for reporter identification
27
55
Asks for date on which reporter discovered the event
13
27
Asks for date on which report was filed
43
88
Asks whether report is initial or follow-up
24
49
Asks for presence of study title on report
40
82
Asks what documentation of event risk was provided to subjects before consent was given
7
14
Asks for number of subjects enrolled in study
9
18
Asks whether occurrence of event changed the relative risks and benefits of the study
10
20
Revision of study protocol required
33
67
Revision of study consent form required
37
76
Changes to protocol and associated documents
Justification to continue study required from PI
4
8
Asks whether the study will be continued or stopped due to occurrence of the event
18
37
15
Asks whether the affected research subject will continue the study
7
14
10
Asks what other regulatory/oversight bodies were notified
13
27
Asks whether other research subjects were notified of event occurrence
20
41
Required attachment of revised protocol
22
45
Required attachment of forms submitted to other regulatory/oversight bodies
8
16
Required attachment of re-consent/assent forms for current research subjects who were notified of event
6
12
Required attachment of current IRB consent form
2
4
28
57
20 Minimal data set
5
0
Better
Worse
Figure 2 Oversight description utility score of institutional review board adverse drug event report forms at National Cancer Institute cancer centers. The score was a sum of the items within the categories of “data safety monitoring information” and “changes to protocol and associated documents,” which were present on each form. A perfect score required the presence of all 21 relevant items. As shown, the median number of items present was nine (range 1–14).
Required attachment of revised consent form
between the ADE report and source documentation, hampering oversight and auditing efforts of the IRB. Of the 34 items labeled as key descriptors of the event (see Table 1), the median number of items present was 4 (domain = 1–11, Figure 1). Of the 21 items considered key descriptors required for IRB oversight (see Table 2), the median number of items present was 9 (domain = 1–14, Figure 2). DISCUSSION
The primary purpose of the IRB is to ensure that human subjects are protected according to the ethical principles of beneficence, respect for persons, and justice.17 Although IRBs also assume obligations to regulatory bodies such as the FDA and to sponsors such as pharmaceutical manufacturers, these obligations do not supplant the primary purpose. In this regard, it is essential that CLINICAL PHARMACOLOGY & THERAPEUTICS | VOLUME 88 NUMBER 2 | AUGUST 2010
IRB adverse event report forms were reviewed for their collection of all variables listed. The “oversight description utility” score is based on these variables. In the two columns on the right, the number and percentage of forms that collected a given variable are shown. See Supplementary Table S7 online for further clarification. ADE, adverse drug event; IRB, institutional review board; NCI, National Cancer Institute; PI, principal investigator.
adverse event reports convey enough information so that causality and severity can be ascertained. Here, we assess whether the forms used to report ADEs provide adequately for reporting of this essential information. IRBs require investigators to report ADEs that are serious or unexpected. Some ADEs can be anticipated on theoretical grounds, on the basis of drug class considerations, from in vitro studies, from animal studies, or from prior experience in human subjects. Other toxicities cannot be anticipated on the basis of knowledge available before a clinical trial. Therefore, to adequately protect subjects, the instruments and methods used 233
ARTICLES during clinical trials must accurately detect and identify both expected and unexpected adverse events and facilitate efficient abstraction of the data required for assessment of event type, severity, and causality. More than 75% of the IRB ADE forms used in cancer drug clinical trials at NCI-designated cancer centers prompt for event description and severity assessment via unstructured free text, with no specification for a taxonomy. Taxonomies facilitate a clear, consistent communication of the medical condition at a granularity that distinguishes the described medical condition from others and facilitates identification of medically related conditions that occur at a disproportionate rate.18 Taxonomies used in the United States and abroad include CTCAE, WHO-ART, COSTART, ADROIT, and MedDRA. The FDA and the NCI recommend that investigators conducting cancer drug clinical trials use the CTCAE v4.0 taxonomy for primary data abstraction because this taxonomy provides descriptors that are specifically relevant for cancer chemotherapy.19 The use of the CTCAE v4.0 taxonomy for primary data abstraction is consistent with use of MedDRA or other taxonomies for purposes of data aggregation and regulatory requirements.20 Four-fifths of the IRB ADE forms at NCI centers rely on a global introspection methodology to assess causality. This methodology has been shown to produce unreliable, ambiguous results.4,14,15 Regardless of diligence and expertise, an investigator or study coordinator using these forms would be likely to provide incomplete and ambiguous descriptions of the event, leading to inaccurate assessment of its severity and cause. In the absence of an operational procedure such as the one we describe here, between-reviewer agreement on event causality is low, even when the reviewers are experts.21 The use of free-form data fields and global introspection may contribute to the poor quality of ADE information derived from clinical trials and thereby contribute to delays in detection of ADE signals. One barrier to adoption of validated tools and methods for ADE assessment is the fact that the FDA continues to accept and implicitly promote the use of global introspection for ADE reporting, thereby impeding efforts to move to validated tools and methods. Validated and reliable methodologies for assessment of ADE causality include the Naranjo Adverse Drug Reaction Probability Scale and the Bayesian Adverse Reactions Diagnostic Instrument. These two methods produce similar assessments of ADE causality; however, Bayesian model development requires multidisciplinary input and is laborious, whereas the Naranjo scale can be implemented in a more rapid, straightforward fashion.22 Perhaps for these reasons, the Naranjo scale is more commonly used both by ADE researchers and in clinical settings. Assessing causality in oncology is a challenge, as there is often difficulty in distinguishing drug toxicity of the evaluated drug from toxicities of other drugs, from progression of the cancer, and from effects of comorbid conditions. Given that the NCI CTCAE v4.0 taxonomy is more familiar to oncologists and is simpler than MeDRA, with its more complex hierarchy, the former is probably a better choice for reporting ADEs in cancer drug clinical trials, despite its shortcomings. We suggest that consideration be given to expanding the use of the NCI CTCAE 234
v4.0 taxonomy to include ADE reporting for clinical trials of noncancer drugs. Prompts for free-text responses have value as a means of capturing narrative and contextual information, but such prompts should supplement validated methods rather than replace them. There is considerable heterogeneity across NCI-designated cancer centers in the forms and methods used to abstract and report ADE information. A model ADE form that presents the elements of a minimal data set, including taxonomy, definitions, and term sets, is provided in Supplementary Table S1 online. Such a data set could be validated for reliability and could also be evaluated for reliability and feasibility by oncology clinical investigators and data managers in a realistic clinical trial setting. If deemed valid, the data set could replace the existing means for describing adverse events and for assessing severity and causality. The use of electronic data systems for cancer clinical trials may provide an opportunity for improving the utility of IRB ADE reports. Such systems would make validated instruments for ADE abstraction generally available, eliminate considerable paperwork in clinical trials, facilitate data collection and subsequent audit trails, and standardize ADE reporting. Our findings are relevant to the recently created FDA Clinical Trials Transformation Initiative, which is currently assessing how reporting and interpretation of serious adverse events can be improved. The importance of our findings is also highlighted by a recent report of the Institute of Medicine (IOM) on the National Cancer Clinical Trials System. This report notes that “the lack of a standard required data set leads to inconsistency in the data collected for cancer trials that can affect the quality of the study and limit cross-study comparisons.”23 METHODS
Form retrieval. As of 2007, 49 centers had conducted clinical trials and
therefore had ADE forms for IRB reporting. We obtained copies of the IRB ADE forms for each of the NCI-designated cancer centers24 within a 1-month period and requested confirmation from each center’s IRB administrator(s) that the forms we had obtained were the actual forms routinely used for reporting adverse events at that center. Specifically, IRB representatives for each institution were contacted and asked a series of questions to confirm that the correct and most updated versions of the ADE form were available. These IRB representatives either sent printable “paper” forms via fax or email or gave further directions about the location and use of the form posted on the cancer center’s website. Electronic forms with data fields were used by only three of the institutions at the time; in these cases, the contents and prompts in the forms were obtained directly from their respective cancer centers. Minimal data set. We constructed a minimal data set (see Supplementary
Table S2 online) to define the essential elements required to adequately describe the event and assess severity and causality, relying on the following sources: the NCI CTCAE v3.0 taxonomy (re-reference CTCAE criteria), the Naranjo Adverse Drug Reaction Probability Scale (see Supplementary Table S3 online),25 Bradford Hill’s nine epidemiologic criteria (see Supplementary Table S4 online),26 the Medwatch reporting form,27 the Yellow Card reporting form,28 the findings of a review of ADE reporting in medical literature,29 a published guide to ADE reporting,30 the findings of the RADAR postmarketing surveillance program,31 and the guidelines for ADE reporting endorsed by the International Society for Pharmacoepidemiology and the International Society of Pharmacovigilance.32 From these sources, an expert panel consisting of a clinical pharmacologist, two oncologists, a statistician, pharmacists, VOLUME 88 NUMBER 2 | AUGUST 2010 | www.nature.com/cpt
ARTICLES and IRB administrators identified the variables comprising the minimal data set considered essential for ADE assessment. The members of the panel had extensive expertise and experience as clinical trial investigators and administrators. The oncologists on the panel had extensive expertise and experience as cancer clinical trial investigators. The consensus item set established by the expert panel is similar to that recently established under ICH Topic E2A definitions and standards for expedited reporting.16 Some of the oncology clinical trial consortiums in the United States have recently adopted standardized reporting. However, because most reporting is defaulted to CTCAE v4.0 toxicity grading, subitem details are often still omitted. This minimal data set was used to develop the model adverse event form provided in Supplementary Table S1 online. Event description (type, expectedness, seriousness, and patient outcome) and event severity. We evaluated how effectively each IRB’s ADE
form prompts a description of the event’s type, expectedness, seriousness, and severity as well as patient outcome. We recorded whether the forms provided a descriptive taxonomy and definitions for these terms (see Supplementary Table S5 online). For expectedness, we considered the term set “expected, unexpected” to be satisfactory. For event seriousness, we considered the term set “death, life-threatening, in-patient hospitalization, prolonged hospitalization, significant disability, congenital abnormality, and nonserious event” acceptable, as recommended by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH-e2a). For patient outcome, we considered the term set “resolved, resolved with sequelae, ongoing, and death” to be satisfactory. For severity, we considered the term set “grade 1, grade 2, grade 3, grade 4, grade 5” to be adequate, in accordance with the CTCAE v3.0 criteria (CTCAE criteria) and the NIH-sponsored National Center for Research (adverse event grading). Causality. Except for Naranjo item N1 (see Supplementary Table S3 online), which asks for previous conclusive reports on the event, the scope of the Naranjo Score is limited to a single subject. In contrast, the scope of the Bradford Hill criteria is more broadly concerned with observations within a population. In order to discover whether each form appropriately prompts for the data needed to assess causality, we evaluated the forms for the presence of elements congruent with Naranjo score items and Bradford Hill items. Liberal criteria were applied to determine whether the prompts required for a particular Naranjo score item or Bradford Hill item were present, allowing alternative phrasing that did not substantially change the meaning of the item. An item was also considered present if a form provided groups of subqueries that, when taken together, would elicit answers to a particular item, even when the item itself was not explicitly prompted as a distinct query. We assessed each form for the presence of these subqueries (Supplementary Table S6 online). We also determined whether forms prompted for the principal investigator’s assessment of causality as determined via global introspection. For the assessment via global introspection, the FDA recommends the term set “unrelated, unlikely related, possibly related, probably related, and definitely related”.
initial report or a follow-up to a previous report. Items in this category are important for the organization and management of adverse event information. Changes to protocol and associated documents. Changes to protocol and
associated documents included those made to the application of the original risk/benefit ratio to the event, termination of the protocol, a justification to continue the protocol after the event, revision(s) of the protocol and consent forms because of the event, whether the subject continued to participate in the study, the notification of other subjects regarding the event, the notification of other regulatory or oversight parties about the event, and prompts to attach a revised protocol, other submitted forms, the original IRB consent form, or the revised IRB consent form to the ADE report form. These items provide information about the principal investigator’s response to the event. Statistical analysis. Frequency distributions were computed for the
number of forms that prompted for answers to each item within the five aforementioned categories. Additionally, each form was awarded scores in two categories: “event description utility” and “oversight description utility.” The “event description utility” score was a sum of the items within the categories of “the event description” (type, expectedness, seriousness, and patient outcome), “event severity,” “causality,” and “patient demographic descriptors,” which were present on each form. A perfect score required the presence of all 34 relevant items. The “oversight description utility” score was a sum of the items within the categories of “data safety monitoring information” and “changes to protocol and associated documents,” which were present on each form. A perfect score required the presence of all 21 relevant items. In both these scores, points were awarded for each item listed in Tables 1 and 2. Reference in Supplementary Table S5 online.33 SUPPLEMENTARY MATERIAL is linked to the online version of the paper at http://www.nature.com/cpt ACKNOWLEDGMENTS S.M.B. was supported by American Cancer Society grant RS-GHP-05-21501-CPPB. We appreciate the technical assistance of the IRB personnel at Northwestern University Feinberg School of Medicine and at other NCI Cancer Centers. This work was conducted as part of the Research on Adverse Drug Events and Reports (RADAR) Project. CONFLICT OF INTEREST The authors declared no conflict of interest. © 2010 American Society for Clinical Pharmacology and Therapeutics
1. 2.
Patient demographic descriptors. Patient demographic descriptors
included sex, age, weight, height, diagnosis, medications, medical history, and available laboratory results. These variables may confound statistical identification of an event and may also be useful in identifying patients at greater or lesser risk for a given event.
Data safety monitoring information. Data safety monitoring informa-
tion included the study title, the known risks of study participation as delineated in the informed consent document(s), the number of subjects enrolled in the study, the Health Insurance Portability and Accountability Act–compliant identifier(s) of the subject, the identifier of the person recording the event, the date the reporter became aware of the event, the date the event was officially reported, and whether the report was an CLINICAL PHARMACOLOGY & THERAPEUTICS | VOLUME 88 NUMBER 2 | AUGUST 2010
3. 4. 5. 6. 7.
Lasser, K.E., Allen, P.D., Woolhandler, S.J., Himmelstein, D.U., Wolfe, S.M. & Bor, D.H. Timing of new black box warnings and withdrawals for prescription medications. JAMA 287, 2215–2220 (2002). Ladewski, L.A. et al. Dissemination of information on potentially fatal adverse drug reactions for cancer drugs from 2000 to 2002: first results from the research on adverse drug events and reports project. J. Clin. Oncol. 21, 3859–3866 (2003). Jones, J.K. Determining causation from case reports. In: Pharmacoepidemiology (ed. Strom, B.L.) 557–570 (John Wiley & Sons, Philadelphia, PA, 2005). Karch, F.E., Smith, C.L., Kerzner, B., Mazzulo, J.M., Weintraub, M. & Lasagna, L. Adverse drug reactions – a matter of opinion. Clin. Pharmacol. Ther. 19, 489–492 (1976). Hazell, L. & Shakir, S.A. Under-reporting of adverse drug reactions: a systematic review. Drug Saf. 29, 385–396 (2006). Furberg, C.D., Levin, A.A., Gross, P.A., Shapiro, R.S. & Strom, B.L. The FDA and drug safety: a proposal for sweeping changes. Arch. Intern. Med. 166, 1938–1942 (2006). Finkelstein, J.B. Institute of Medicine, Congress seek solutions to FDA’s safety woes through reform, funding. J. Natl. Cancer Inst. 99, 104–107 (2007). 235
ARTICLES 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
236
Carpenter, D., A proposal for financing postmarketing drug safety studies by augmenting FDA user fees. Health Aff. 10, 1377 (2005). Friedberg, M., Saffran, B., Stinson, T.J., Nelson, W. & Bennett, C.L. Evaluation of conflict of interest in economic analyses of new drugs used in oncology. JAMA 282, 1453–1457 (1999). Shakir, S.A. & Layton, D. Causal association in pharmacovigilance and pharmacoepidemiology: thoughts on the application of the Austin BradfordHill criteria. Drug Saf. 25, 467–471 (2002). Perrio, M., Voss, S. & Shakir, S.A. Application of the bradford hill criteria to assess the causality of cisapride-induced arrhythmia: a model for assessing causal association in pharmacovigilance. Drug Saf. 30, 333–346 (2007). Theradoc (2009). Medmarx Adverse Drug Event Reporting: Comparative Performance Reporting Helps to Reduce Adverse Drug Events. (2010). Koch-Weser, J., Sellers, E.M. & Zacest, R. The ambiguity of adverse drug reactions. Eur. J. Clin. Pharmacol. 11, 75–78 (1977). Kramer M. Assessing causality of adverse drug reactions: global introspection and its limitations. Drug Inf J 20, 433–437 (1986). European Medicines Agency. ICH Topic E2A Clinical Safety Data Management: Definitions and Standards for Expedited Reporting (2010). National Institutes of Health. Office of Human Subjects Research. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research (2009). Brown, E.G. Effects of coding dictionary on signal generation: a consideration of use of MedDRA compared with WHO-ART. Drug Saf. 25, 445–452 (2002). US Food and Drug Administration. Oncology Tools: Common Toxicity Criteria (2009). Radiation Therapy Oncology Group. MedDRA v9.0 codes. (2009). Arimone, Y. et al. Inter-expert agreement of seven criteria in causality assessment of adverse drug reactions. Br. J. Clin. Pharmacol. 64, 482–488 (2007).
22. Lanctôt, K.L. & Naranjo, C.A. Comparison of the Bayesian approach and a simple algorithm for assessment of adverse drug events. Clin. Pharmacol. Ther. 58, 692–698 (1995). 23. A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. (2010). 24. National Cancer Institute. Cancer Centers Program (2007). 25. Naranjo, C.A. et al. A method for estimating the probability of adverse drug reactions. Clin. Pharmacol. Ther. 30, 239–245 (1981). 26. Bradford-Hill, A. The environment and disease: association or causation? Proc. R. Soc. Med. 58, 295–300 (1965). 27. US Food and Drug Administration. Medwatch: The FDA Safety Information and Adverse Event Reporting Program (2009). 28. Medicines and Healthcare Products Regulatory Agency. Report of an Independent Review of Access to the Yellow Card Scheme (2009). 29. Kelly, W.N. The quality of published adverse drug event reports. Ann. Pharmacother. 37, 1774–1778 (2003). 30. Nebeker, J.R., Barach, P. & Samore, M.H. Clarifying adverse drug events: a clinician’s guide to terminology, documentation, and reporting. Ann. Intern. Med. 140, 795–801 (2004). 31. Bennett, C.L. et al. The Research on Adverse Drug Events and Reports (RADAR) project. JAMA 293, 2131–2140 (2005). 32. Kelly, W.N. et al.; International Society for Pharmacoepidemiology; International Society of Pharmacovigilance. Guidelines for submitting adverse event reports for publication. Drug Saf. 30, 367–373 (2007). 33. US Food and Drug Administration. FDA Regulatory Information. E2B(R) Clinical Safety Data Management: Data Elements for Transmission of Individual Case Safety Reports (2010).
VOLUME 88 NUMBER 2 | AUGUST 2010 | www.nature.com/cpt
