0145-6008/03/2701-0067$03.00/0 ALCOHOLISM: CLINICAL AND EXPERIMENTAL RESEARCH
Vol. 27, No. 1 January 2003
Validity of Brief Alcohol Screening Tests Among Adolescents: A Comparison of the AUDIT, POSIT, CAGE, and CRAFFT John R. Knight, Lon Sherritt, Sion Kim Harris, Elizabeth C. Gates, and Grace Chang
Background: Adolescents should be screened for alcohol misuse as part of routine care. The objective of this study was to compare the criterion validity of the Alcohol Use Disorders Identification Test (AUDIT), the Problem Oriented Screening Instrument for Teenagers substance use/abuse scale (POSIT), and the CAGE and CRAFFT questions among adolescents. Methods: Fourteen- to 18-year-old patients arriving for routine healthcare at a large, hospital-based adolescent clinic completed the four screens and the criterion standard Adolescent Diagnostic Interview, which yields DSM-IV diagnoses of alcohol abuse and dependence. Receiver operating characteristic (ROC) curves were plotted to determine optimal cut-points. Areas under the ROC curves of the four screens were compared, and sensitivities and specificities were calculated. Results: Participants’ past 12-month alcohol diagnostic classifications were as follows: no use (58.6%), nonproblem use (13.0%), problem use (20.8%), abuse (5.4%), and dependence (2.2%). Optimal cut-points associated with problem use or higher were 2 for AUDIT, 1 for POSIT, 1 for CAGE, and 1 for CRAFFT. ROC curve area of the CAGE was significantly lower compared with areas of all other screens. Sensitivities (95% confidence intervals) were AUDIT 0.88 (0.83– 0.93), POSIT 0.84 (0.79 – 0.90), CAGE 0.37 (0.29 – 0.44), and CRAFFT 0.92 (0.88-0.96); specificities were AUDIT 0.81 (0.77– 0.85), POSIT 0.89 (0.86 – 0.92), CAGE 0.96 (0.94 – 0.98), and CRAFFT 0.64 (0.59 – 0.69). Conclusions: The AUDIT, POSIT, and CRAFFT have acceptable sensitivity for identifying alcohol problems or disorders in this age group. The CAGE is not recommended for use among adolescents. Key Words: Substance-Related Disorders, Alcoholism, Substance Abuse Detection, Sensitivity and Specificity, Adolescence.
M
ISUSE OF ALCOHOL is one of our nation’s greatest health problems with an estimated annual cost of more than $185 billion (Wagenaar et al., 2000) and more than 100,000 alcohol-related deaths each year (National Institute on Alcohol Abuse and Alcoholism, 2001). Heavy drinking adversely affects all ages, including adolescents,
From the Department of Pediatrics (JRK, SKH), Division on Addictions (JRK, LS, SKH), and Department of Psychiatry (GC), Harvard Medical School, Boston; Center for Adolescent Substance Abuse Research (JRK, LS, SKH, ECG, GC), Division of General Pediatrics (JRK, LS, ECG), and Division of Adolescent/Young Adult Medicine (JRK, SKH), Children’s Hospital, Boston; and Department of Psychiatry (GC), Brigham and Women’s Hospital, Boston. Received for publication April 15, 2002; accepted October 24, 2002. Supported by Grant R01 AA12165 from NIAAA and The Substance Abuse and Mental Health Services Administration, and Grant 036126 from The Robert Wood Johnson Foundation. Other support was provided by Grant 5T20MC000-11-06 (JK) and Grant 5T71MC 00009-10 (LS, SH) from the Maternal and Child Health Bureau; and by grant K24 AA00289 (GC) from NIAAA. Reprint requests: John R. Knight, MD, Center for Adolescent Substance Abuse Research, Children’s Hospital Boston, 300 Longwood Avenue, Boston, MA 02115; Fax: 617-267-9397; Email:
[email protected]. Copyright © 2003 by the Research Society on Alcoholism. DOI: 10.1097/01.ALC.0000046598.59317.3A Alcohol Clin Exp Res, Vol 27, No 1, 2003: pp 67–73
and is associated with both serious short- and long-term consequences. Greater than 35% of fatal motor vehicle crashes involving drivers ages 15 to 20 years old are related to alcohol use, and almost 25% involve a legally intoxicated driver (Centers for Disease Control, 1999). Adolescent drinking is linked to other serious health risk behaviors such as early sexual activity, truancy, violence, and weapon carrying (Kokotailo, 1995). Heavy drinking also interferes with normal adolescent cognitive, emotional, and social development and is linked to psychiatric disorders and delinquency (Hicks et al., 1993). Those who begin drinking before age 15 are four times as likely to develop alcohol dependence and more than twice as likely to develop alcohol abuse compared with peers who delay drinking until age 21 (Grant, 1997; Robis and Przybeck, 1985). Primary care physicians and other healthcare providers can play an important role in preventing the harm associated with alcohol misuse by recognizing problems early and providing appropriate intervention. A number of published guidelines emphasize the importance of routine alcohol screening of adolescents, including the American Medical Association’s Guidelines for Adolescent Preventive Services (Elster and Kuznets, 1994), the Bright Futures Guidelines (Green, 1994), and the American Academy of Pedi67
68
atrics’ Guidelines for Health Supervision III (American Academy of Pediatrics, 1997). Although all guidelines recommend an annual screening for alcohol and drugs, they do not provide details on how it should be performed. Screening is a procedure generally applied to populations and is intended to identify individuals at risk for a disease. Assessment is a procedure generally applied to individuals who have screened positive and is intended to confirm the presence of the disease and measure its severity. The ideal tool for screening adolescents should be developmentally appropriate, valid, reliable, and practical for use in busy medical offices (Knight, 2001). Some alcohol screening tests can be administered by the clinician as part of the general health interview or while performing the physical examination. To be practical, these must be brief and easy to score and remember. Simple yes/no questions that lend themselves to oral administration and mnemonic acronyms are ideal. Questionnaires, by way of comparison, are usually administered to patients in the waiting room. To be practical, these must be capable of self-completion within the usual wait time, and scoring procedures must be sufficiently streamlined so that minimal staff training and time are required and results can be given to the clinician before the start of the medical visit. A number of brief screening instruments meet these criteria, including the four selected for this study. The Alcohol Use Disorders Identification Test (AUDIT) (Babor et al., 1992) was developed by the World Health Organization as a screening tool for hazardous alcohol consumption (Saunders et al., 1993). The AUDIT’s 10 multiple-choice response items require approximately 2 min for administration and another 3 min for scoring. Standard instructions state that a total score of eight or more indicates a strong likelihood of harmful alcohol consumption (Allen and Columbus, 1995). The AUDIT has been shown to be a valid means of identifying current hazardous use of alcohol among adults and college students (Allen et al., 1997; Fleming et al., 1991; Miles et al., 2001), and one study found the AUDIT to have good sensitivity and specificity for identifying alcohol disorders among 13to 19-year-old emergency ward patients (Chung et al., 2000). However, another study found the AUDIT to be less practical for use among adolescents seen in a primary care setting (Foster et al., 1997). The Problem Oriented Screening Instrument for Teenagers (POSIT) is a multiple-problem screening tool developed by the National Institute on Drug Abuse specifically for adolescents aged 12 through 19 (Rahdert, 1991). The POSIT includes a total of 139 yes/no questions divided into 10 scales, one of which is the 17-item substance use/abuse scale (Allen and Columbus, 1995). A score of one or higher on this scale is considered a “red flag” for substance-related problems (Rahdert, 1991). The POSIT has been shown to have good reliability and validity among adolescents, including those seen in primary care medical settings (Dembo et al., 1996; Knight et al., 2001; McLaney et al., 1994).
KNIGHT ET AL.
Although these two questionnaires have advantages, administration in the waiting room may pose a risk to adolescents’ confidentiality (Foster et al., 1997; McLaney et al., 1994). An alternative is an orally administered screen that physicians can administer during the course of the medical visit, after parents have left the room. The CAGE questions are a good example of this type of brief screen (Ewing, 1984). Originally developed for screening adults, the CAGE has become popular among medical clinicians (Mayfield et al., 1974). Its four yes/no questions are weighted equally (each yes answer ⫽ 1), and a total score of two or greater is considered clinically significant (Allen and Columbus, 1995). The CAGE has been well validated among adults and may be particularly suited to identifying patients with alcohol dependence (Bush et al., 1987; Magruder-Habib et al., 1993; Soderstrom et al., 1997). However, the CAGE has been found to perform less well among females, college students, and adolescent medical patients (Aertgeerts et al., 2000; Chung et al., 2000; Knight et al., 2000; O’Hare and Tran, 1997). The CRAFFT shares many of the CAGE’s advantages but was designed specifically to be developmentally appropriate for adolescents (Knight et al., 1999). CRAFFT is an acronym of the first letters of key words in the test’s six questions: “Have you ever ridden in a CAR driven by someone (including yourself) who was “high” or had been using alcohol or drugs? Do you ever use alcohol or drugs to RELAX, feel better about yourself, or fit in? Do you ever use alcohol or drugs while you are by yourself, ALONE? Do you ever FORGET things you did while using alcohol or drugs? Do your family or FRIENDS ever tell you that you should cut down on your drinking or drug use? Have you ever gotten into TROUBLE while you were using alcohol or drugs?” Like the POSIT, CRAFFT screens for use of both alcohol and drugs. A previous study concluded that the CRAFFT has adequate sensitivity and specificity for identifying adolescents with substance-related problems (Knight et al., 2002). The purpose of the current study was to compare the criterion validity of the AUDIT, POSIT, CAGE, and CRAFFT in identifying alcohol-related pathology in an adolescent medical clinic population. METHODS This criterion standard study was conducted between March 15, 1999, and September 14, 2000, at the Adolescent/Young Adult Medical Practice of Children’s Hospital Boston. This clinic serves both inner city and suburban youth from a wide range of social strata, racial groups, and ethnic backgrounds. Primary care providers, including staff physicians and fellows specializing in adolescent medicine, pediatric residents, and nurse practitioners, served almost 5000 patients through both routine well care and urgent care visits during the study recruitment period. The study sample was drawn from all 14- to 18-year-old patients arriving for routine care. Before clinic sessions began, the birth dates of all scheduled patients were reviewed and a study recruitment form was placed on the chart cover of each age-eligible patient. The form included items that recorded demographic information, the provider’s impression of the
69
COMPARISON OF THE AUDIT, POSIT, CAGE, AND CRAFFT
patient’s level of alcohol use, and the patient’s response to the invitation to participate. Patients who were unable to read and understand English were excluded, as were those with medical or psychiatric problems that precluded participation in research on the day of the clinic visit. Eligible patients met with a research assistant, who explained the study procedures and obtained signed assent. The Children’s Hospital Boston Committee on Clinical Investigation (institutional review board) waived the requirement for parental consent in accordance with current guidelines for adolescent health research (Department of Health and Human Services, 1991; Society for Adolescent Medicine, 1995). We identified 711 eligible patients but excluded a total of 41 because of cognitive impairment (n ⫽ 27), insufficient fluency in English (n ⫽ 9), severe hearing impairment (n ⫽ 2), anorexia nervosa (n ⫽ 2), and psychosis (n ⫽ 1). Of the remaining 670 who were invited, 538 (80%) agreed to participate. The reasons most commonly cited by the refusers (n ⫽ 132) included “not enough time” (56%), “not interested” (33%), or “came with a parent” (6%). Participants were told that the purpose of the study was to assess the value of screening questions on alcohol use and that their answers would be kept confidential. They were informed that the research team would notify their provider if a serious problem were identified so that appropriate care could be arranged. After completing the assessment battery, each participant received a $25 merchandise certificate as compensation for his or her time. The assessment battery included the 10-item AUDIT, the 17-item POSIT substance use/abuse scale (hereinafter referred to as the POSIT), the four CAGE questions, the six CRAFFT questions, and the Adolescent Diagnostic Interview (ADI) (Winters and Henly, 1993). The ADI is a 30to 90-min structured psychiatric diagnostic interview that yields alcoholand drug-related diagnoses (i.e., abuse and dependence) according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), the criterion standard for this study (American Psychiatric Association, 1994). The ADI has been shown to have good reliability and validity among adolescents (Winters et al., 1993, 1999). A research assistant first orally administered the CRAFFT and CAGE questions, then monitored participants’ completion of the paper/pencil version of the POSIT and AUDIT, and conducted the ADI interview last. All research assistants were college graduates who underwent a structured initial training and periodic reassessments for competence in ADI administration. Initial training included observation of a model interview, practice sessions until a prescribed level of performance was reached, and a videotaped final assessment that was reviewed by study investigators and the author of the ADI (KW) to ensure competence. All data were independently entered twice into a specially designed data management program based on Access 97® (Microsoft, Redmond, WA), which included automatic range and logic checks and an entry tracking log. The dual entry files were compared and discrepancies reconciled by checking the original data source. The cleaned dataset was then imported into SPSS v10.0® (SPSS Inc., Chicago, IL) for analysis. Frequencies of sex, age, race/ethnicity, and provider impression of alcohol involvement in the study sample were compared with those of the refusers group by using 2 tests. To preserve adequate cell sizes for analyses, we transformed participant age into a dichotomous variable (i.e., younger youth and older youth) based on the sample median. The provider impression variable was similarly transformed into a trichotomous variable (i.e., no use/nonproblem use, problem use/abuse/dependence, and no impression), because provider impressions of abuse and dependence were uncommon. The refusers group did not differ significantly from the study sample in distribution of age, sex, race/ethnicity, or provider impressions of alcohol involvement. The study sample was also similar to the entire 14- to 18-year-old clinic population in age and race/ethnicity but included a greater proportion of females (68% vs. 59%, p ⬍ 0.001). Participants were classified into five mutually exclusive diagnostic groups based on their pattern of alcohol use during the previous 12 months as recorded in the ADI interview. “No use” was defined as no reported drinking during the past year. “Nonproblem use” was defined as
any reported drinking during the past year but no reported alcohol-related problems. “Problem use” was defined as one or more reported alcoholrelated problems that did not reach the diagnostic threshold for an alcohol-related disorder. “Abuse” and “dependence” were defined by the corresponding DSM-IV criteria. Each ADI was scored twice, first by study personnel using the standard written instructions and then by computer using an SPSS® syntax algorithm developed by the instrument’s authors (Winters and Henly, 1993). There were no instances of disagreement in alcohol diagnoses. Frequencies of demographic variables and participants’ diagnostic classifications were computed. Differences in proportions of diagnostic classifications for males versus females and younger versus older adolescents were examined by using Fisher’s exact test. Diagnostic classifications were used to define three alcohol-related screening categories: any problem (i.e., problem use, abuse, or dependence), any diagnosis (i.e., abuse or dependence), and dependence. Sensitivity and specificity were calculated for all three screening categories, and the bootstrap technique was used to estimate 95% confidence intervals (Carpenter and Bithell, 2000; Platt et al., 2000; Rutter, 2000). Confidence intervals could not be computed for the degenerate case, that is, where sensitivity or specificity equaled one. Receiver operating characteristic (ROC) curves of the AUDIT, POSIT, CAGE, and CRAFFT were plotted. The score associated with the point closest to the upper left corner of the ROC plot, which also indicates the maximum product of sensitivity and specificity, was chosen to indicate the optimal cut-point of each screen. However, this method does not account for the relative cost/benefit ratio of false positives and false negatives. ROC curve areas with asymptotic 95% confidence intervals were calculated as an overall indicator of test accuracy. We are not reporting results of analyses of criterion validity for age, sex, and race/ ethnicity subgroups because of inadequate power in this sample.
RESULTS
Participants were 68% female, 51% black non-Hispanic, 24% white non-Hispanic, 19% Hispanic, and 7% Asian/ other and were distributed almost evenly across years of age (Table 1). Greater than 40% of participants reported drinking during the 12 months preceding the study, 28% had experienced alcohol-associated problems, and almost 8% met criteria for a diagnosis of either alcohol abuse or dependence. Compared with males, more females drank (45% vs. 34%, p ⫽ 0.01). Compared with younger adolescents, more of the older adolescents drank (52% vs. 31%, p ⬍ 0.001) and were classified with “problem use” (38% vs. 19%, p ⬍ 0.0001). Alcohol abuse and dependence diagnoses were not significantly associated with participants’ sex or age. ROC curve areas, sensitivity, and specificity values for relevant cut-points for all three screening categories (i.e., any problem, any disorder, dependence) are presented in Table 2. The area under the ROC curve reflects the overall accuracy of a screening test. For the “any problem” screening category, the ROC areas of the AUDIT, POSIT, and CRAFFT were similarly high. The ROC area of the CAGE was significantly lower. The screens (cut-points) with the highest sensitivities were CRAFFT (1), AUDIT (2), and POSIT (1). The CRAFFT “optimal” cut-point was 2 (i.e., highest product of sensitivity ⫻ specificity), even though this value yielded significantly lower sensitivity than a score of 1. The sensitivity of the CAGE, even at its lowest possible cut-point (1), was
70
KNIGHT ET AL. Table 1. Demographic Characteristics and Alcohol Diagnostic Classifications of 538 Participants Nonproblem use
No Use All (n ⫽ 538) Sex Male (n ⫽ 170) Female (n ⫽ 368) Age 14 Years (n ⫽ 103) 15 Years (n ⫽ 85) 16 Years (n ⫽ 121) 17 Years (n ⫽ 127) 18 Years (n ⫽ 102) Race/ethnicity Black non-Hispanic (n ⫽ 272) White non-Hispanic (n ⫽ 130) Hispanic (n ⫽ 101) Asian/other (n ⫽ 35)
Problem use
Abuse
Dependence
315 (58.6)
70 (13.0)
112 (20.8)
29 (5.4)
12 (2.2)
113 (66.5) 202 (54.9)
17 (10.0) 53 (14.4)
30 (17.6) 82 (22.3)
7 (4.1) 22 (6.0)
3 (1.8) 9 (2.4)
85 (82.5) 56 (65.9) 68 (56.2) 61 (48.0) 45 (44.1)
9 (8.7) 14 (16.5) 15 (12.4) 20 (15.7) 12 (11.8)
4 (3.9) 10 (11.8) 28 (23.1) 35 (27.6) 35 (34.3)
4 (3.9) 4 (4.7) 8 (6.6) 7 (5.5) 6 (5.9)
1 (1.0) 1 (1.2) 2 (1.7) 4 (3.1) 4 (3.9)
180 (66.2) 56 (43.1) 60 (59.4) 19 (54.3)
29 (10.7) 21 (16.2) 11 (10.9) 9 (25.7)
51 (18.8) 37 (28.5) 19 (18.8) 5 (14.3)
7 (2.6) 12 (9.2) 8 (7.9) 2 (5.7)
5 (1.8) 4 (3.1) 3 (3.0) 0 (0.0)
Data presented as n (%); all diagnostic classifications based on Adolescent Diagnostic Interview. Table 2. ROC Curve Area, Sensitivity, and Specificity With 95% Confidence Intervals (95% CI) of Alcohol Screening Test Cut-Points for Three Screening Categories: Any Problem, Any Disorder, and Dependence
Total n (%) Positives (%) Test, ROC curve area (95% CI) AUDIT POSIT CAGE CRAFFT Test (cut-point), sensitivity (95% CI) AUDIT (2) AUDIT (3) AUDIT (5) AUDIT (8) POSIT (1) POSIT (2) POSIT (3) CAGE (1) CAGE (2) CRAFFT (1) CRAFFT (2) Test (cut-point), specificity (95% CI) AUDIT (2) AUDIT (3) AUDIT (5) AUDIT (8) POSIT (1) POSIT (2) POSIT (3) CAGE (1) CAGE (2) CRAFFT (1) CRAFFT (2)
Any problem*
Any disorder*
Dependence
538 (100) 153 (28)
538 (100) 41 (8)
538 (100) 12 (2.2)
.92 .88 .67 .88
(.89, (.85, (.61, (.85,
.94) .92) .72) .92)
.91 .93 .77 .88
(.87, (.89, (.67, (.83,
.95) .96) .86) .93)
.95 .95 .87 .89
(.91, (.91, (.74, (.83,
.88 .72 .50 .24 .84 .65 .48 .37 .18 .92 .70
(.83, (.65, (.43, (.18, (.79, (.58, (.40, (.29, (.12, (.88, (.63,
.93) .79) .58) .31) .90) .72) .56) .44) .24) .96) .77)
.93 .88 .73 .54 .98 .88 .80 .61 .37 .98 .83
(.84, (.76, (.58, (.38, (.92, (.76, (.67, (.45, (.22, (.92, (.71,
1.0) .97) .87) .69) 1.0) .97) .91) .76) .50) 1.0) .94)
1.0 (**) 1.0 (**) .83 (.57, 1.0) .75 (.46, 1.0) 1.0 (**) 1.0 (**) .92 (.71, 1.0) .83 (.57, 1.0) .42 (.14, .70) 1.0 (**) .92 (.71, 1.0)
.81 .89 .97 1.0 .89 .94 .97 .96 .99 .64 .94
(.77, (.86, (.95, (.99, (.86, (.91, (.95, (.94, (.98, (.59, (.91,
.85) .92) .99) 1.0) .92) .96) .99) .98) 1.0) .69) .96)
.66 .77 .88 .97 .73 .82 .90 .91 .97 .52 .81
(.62, (.73, (.85, (.95, (.69, (.79, (.87, (.88, (.95, (.47, (.77,
.70) .80) .91) .98) .77) .86) .92) .93) .98) .56) .84)
.63 .73 .85 .94 .69 .79 .86 .88 .95 .49 .77
(.59, (.70, (.82, (.92, (.65, (.75, (.83, (.86, (.93, (.45, (.74,
.99) .98) .99) .95)
.67) .77) .88) .96) .74) .82) .89) .91) .97) .53) .81)
* Any problem ⫽ alcohol problem use, abuse or dependence; any disorder ⫽ alcohol abuse or dependence. ** 95% CI cannot be computed.
significantly lower than that of all other screens. However, the specificity of the CAGE was higher. For the “any disorder” screening category, the ROC curve areas of the AUDIT, POSIT, and CRAFFT again did not differ significantly from one another. The ROC area of the CAGE was significantly lower than that of the AUDIT and POSIT. The screens (cut-point) with the highest sensitivities were CRAFFT (1), AUDIT (2), and POSIT (1). The sensitivity of the CAGE (1) was lower, but specificity was higher, compared with all other screens. The specificity
of the CRAFFT (1) was lower than all other screens, whereas specificity of the POSIT (1) was significantly lower than the CAGE (1) only. Optimal cut-points for “any disorder” were 3 for AUDIT, 2 for POSIT, 1 for CAGE, and 2 for CRAFFT. The sensitivity of the AUDIT was 0.54 and the CAGE 0.37 at their commonly recommended cutpoints (8 and 2, respectively). For the “dependence” category, all screens had equally high sensitivities except for CAGE (2), which was significantly lower than all others. The ROC curve areas of the
COMPARISON OF THE AUDIT, POSIT, CAGE, AND CRAFFT
four screens did not differ significantly when screening for dependence. DISCUSSION
This study provides strong supportive evidence for the criterion validity of the AUDIT, POSIT, and CRAFFT questions in an adolescent general outpatient clinic sample. Providers should carefully consider test characteristics, the level of alcohol-related pathology they wish to identify, and practical limitations of test administration and scoring before deciding which screen best suits the needs of their particular clinical practice. Sensitivity may be the single most important characteristic to consider when screening for alcohol disorders, because the risks of a false negative (i.e., missed diagnosis) far outweigh those of a false positive (i.e., additional assessment), although some adolescents may find additional assessment distressing. The CAGE does not have sufficient sensitivity and its use among adolescents is not recommended. Our findings also indicate that commonly used cut-points for adults must be lowered when alcohol screens are administered to adolescents. The AUDIT recommended cutpoint of 8 was found to be too high. Studies in college populations have recommended cut points of 6 and 11 (Aertgeerts et al., 2000; Fleming et al., 1991), and Chung et al. (2000) reported 4 as optimal in a study of 13- to 19year-old emergency ward patients. We found that a cutpoint of 2 was optimal for identifying alcohol problem use and 3 for identifying alcohol abuse or dependence. Compared with Chung’s study, our sample had a narrower age range and a lower prevalence of alcohol disorders (8% vs. 18%), which may account for variations in sensitivity. Similarly, the standard recommendation for the CAGE is a cut-point of 2. We found the sensitivity of CAGE too low even when its cut-point is lowered to 1. Test characteristics, including sensitivity, vary across screening categories, so providers should carefully consider the purpose of their screening before selecting a particular test. Do they wish to identify adolescents with alcohol problems, alcohol disorders, or alcohol dependence only? In most circumstances, providers will likely choose to screen for all alcohol problems and disorders (i.e., the “any problem” screening category in this study). Practical aspects of test administration and scoring are also important considerations. Each of the three recommended screening tests presents its own unique advantages and disadvantages. The AUDIT, for example, is the only recommended tool that screens for alcohol only. This may be desirable in certain situations, for example, when clinicians wish to identify adolescents who are possible candidates for a brief alcohol intervention. In most circumstances, however, the limitation of screening for alcohol only presents a disadvantage. Alcohol use is highly associated with illicit drug use, and therefore screening for all substances simultaneously seems most practical. The AUDIT also requires
71
staff time for scoring, although a computerized version could easily overcome this limitation. The AUDIT may offer an advantage to efficiency in clinic settings where patients can self-administer the test while waiting to see the provider. However, clinics must designate a private place for this purpose to ensure complete confidentiality. The POSIT is another written or computer-based questionnaire that possesses advantages and limitations similar to those of the AUDIT. Unlike the AUDIT, however, the POSIT screens for alcohol and drugs simultaneously. The full 139-item version of the POSIT can provide a wealth of additional, clinically relevant information from its nine other scales, but it requires 20 to 30 min for completion. The written version of the POSIT requires a substantial commitment of staff resources for scoring, whereas the computerized version does not. In fact, the computerized POSIT (PowerTrain, Inc., Landover, MD) automatically prepares a convenient, summary bar graph that compares individual scale scores with normative cut-points. This summary may be printed out and used to initiate a discussion between the adolescent and provider. The POSIT substance use/abuse scale’s recommended cut-point of 1 (Rahdert, 1991) yielded high sensitivity and specificity in our study, although a score of 2 yielded the highest product of these two characteristics. This is identical to the optimal cut-point (2) reported by Latimer et al. (1996) in a sample of adolescents from schools, clinics, and correctional settings. Latimer’s report also found similar sensitivity (0.95) and specificity (0.79) for POSIT in identifying substance use disorders to those found in this study. The POSIT is a valid screening tool for adolescents at either cut-point. The CRAFFT screen also presents unique advantages. Like the CAGE, it has a convenient mnemonic and requires only 1 or 2 min for administration and scoring. Like the POSIT, it screens for alcohol and drugs simultaneously. The CRAFFT is brief, and many providers will find it possible to ask the six questions during the course of the medical visit, sparing other staff’s time. A CRAFFT score of 1 demonstrates high sensitivity for identifying all three screening categories. The CRAFFT yielded lower specificity compared with the other screens. Providers should therefore follow positive responses to individual CRAFFT items with additional questions and decide what level of intervention, if any, their adolescent patient requires based on all the information. Raising the CRAFFT score cutpoint to 2 provides a better balance between sensitivity and specificity, especially for identifying alcohol disorders. This study presents certain strengths. It was conducted in a primary care setting, where routine screening is required. The study sample included patients arriving for both routine care and urgent care visits. The criterion standard was a well-validated diagnostic interview for adolescents, so that all screens could be compared directly with DSM-IV diagnostic criteria. All screens were administered before the diagnostic interview, and there were no preliminary questions on alcohol quantity and frequency, which might
72
lower subsequent screening test scores (Steinweg, 1993). In addition to abuse and dependence disorders, our analysis included a “problem use” diagnostic classification, which has been previously suggested by the American Academy of Pediatrics (Wolraich et al., 1996). Several limitations should also be noted. The study was conducted in one clinic only, and prevalence rates of alcohol problems and disorders in other clinic settings may be different. Our sample included a large proportion of racial and ethnic minorities, and females coming for medical care may be at higher risk than those in the general population. However, we would expect to find similar sensitivity and specificity for all tests in similar urban medical outpatient clinic populations. Our study results cannot be generalized to early adolescents (10 –13 years of age) because they were not included in the sample and screening test characteristics may differ significantly in a younger population. Future studies should address these limitations by including a variety of clinical settings and adolescents less than 14 years of age. We instructed providers to invite all their patients to participate. However, we cannot assess the degree to which they followed this instruction, and selection bias remains a possibility. Our study may also be limited in that it relied on adolescents’ self-report. The extent to which some participants may have overstated and others understated their use of alcohol is unknown. However, previous studies have shown that self-report compares favorably with other methods of substance use detection (Babor et al., 1989; Winters et al., 1990). Obtaining laboratory testing for alcohol was impractical in this study, and obtaining collateral reports of alcohol use from family members poses too great a risk to adolescents’ confidentiality. In addition, self-report may be influenced by order of administration. Since we administered the screens to all participants in an identical sequence, we cannot assess the nature or degree of possible bias from ordering. In certain aspects, this study is further limited by the size of the study population. Our original sample size calculation was based on achieving adequate power to determine with 0.95 confidence that screening test sensitivities were at least 0.80 across the entire study sample. This model is relatively underpowered for examining differences among demographic subgroups. Future studies should also address this limitation and place special importance on examining possible gender differences in screening test characteristics. Females account for approximately two thirds of patients and three fourths of visits at our clinic, and other practices are similar (Cherry et al., 2001). Brief alcohol screening tests must therefore demonstrate high sensitivities among females, because they use health services more frequently than males. We conclude that the AUDIT, POSIT, and CRAFFT have good criterion validity for screening adolescent medical patients for alcohol problems and disorders. We also conclude that the CAGE questions are not appropriate for
KNIGHT ET AL.
screening adolescents. The CRAFFT questions present a practical and valid alternative to CAGE. ACKNOWLEDGMENTS We thank Erin Gates, BA, Sarah Rosenberg, BA, Katherine O’Connor, BA, and Allison Arneill, MA, for their assistance in study implementation; the clinicians and staff of the Adolescent/ Young Adult Medical Practice at Children’s Hospital Boston for assistance in recruitment; Ken C. Winters, PhD, for consultation on the study measurement battery, and S. Jean Emans, MD, for review of the manuscript. REFERENCES Aertgeerts B, Buntinx F, Bande-Knops J, Vandermeulen C, Roelants M, Ansoms S, Fevery J (2000) The value of CAGE, CUGE, and AUDIT in screening for alcohol abuse and dependence among college freshmen. Alcohol Clin Exp Res 24:53–57. Allen J, Columbus M (eds) (1995) Assessing Alcohol Problems: A Guide for Clinicians and Researchers. National Institute on Alcohol Abuse and Alcoholism, Bethesda, MD. Allen J, Litten R, Fertig J, Babor T (1997) A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcohol Clin Exp Res 21:613– 619. American Academy of Pediatrics (1997). Guidelines for Health Supervision III. Committee on Psychosocial Aspects of Child and Family Health, Elk Grove Village, IL. American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders. 4th ed. American Psychiatric Association, Washington DC. Babor T, de la Fuente J, Saunders J, Grant M (1992) AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care. World Health Organization, Geneva, Switzerland. Babor TF, Kranzler HR, Lauerman RJ (1989) Early detection of harmful alcohol consumption: comparison of clinical, laboratory, and self-report screening procedures. Addict Behav 14:139 –157. Bush B, Shaw S, Cleary P, DelBlanco T, Aronson MD (1987) Screening for alcohol abuse using the CAGE questionnaire. Am J Med 82:231– 235. Carpenter J, Bithell J (2000) Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat Med 19:1141– 1164. Centers for Disease Control (1999) Alcohol involvement in fatal motorvehicle crashes—United States, 1997–1998. MMWR Morb Mortal Wkly Rep 48:1086 –1087. Cherry DK, Burt CW, Woodwell DA (2001) National Ambulatory Medical Care Survey: 1999 Summary. Advance Data Vital Health Stat 322, National Center for Health Statistics, Hyattsville, MD, pp 1–36. Chung T, Colby SM, Barnett NP, Rohsenow DJ, Spirito A, Monti PM (2000) Screening adolescents for problem drinking: performance of brief screens against DSM-IV alcohol diagnoses. J Stud Alcohol 61: 579 –587. Dembo R, Schmeidler J, Brodern P, Chin Sue C, Manning D (1996) Examination of the reliability of the Problem Oriented Screening Instrument for Teenagers (POSIT) among arrested youth entering a juvenile assessment center. Subst Use Misuse 31:785– 824. Department of Health and Human Services (1991) Protection of Human Subjects (Title 45: Code of Federal Regulations, Part 46), U.S. Department of Health and Human Services, National Institutes of Health, Office for Protection From Research Risks. Available at: http:// grants.nih.gov/grants/oprr/humansubjects/45cfr46.htm. Accessed January 17, 2000. Elster A, Kuznets N (eds) (1994) AMA Guidelines for Adolescent Preventive Services (GAPS). Williams & Wilkins, Baltimore.
COMPARISON OF THE AUDIT, POSIT, CAGE, AND CRAFFT
Ewing J (1984) Detecting alcoholism: the CAGE questionnaire. JAMA 252:1906 –1907. Fleming MF, Barry KL, MacDonald R (1991) The alcohol use disorders identification test (AUDIT) in a college sample. Int J Addict 26:1173– 1185. Foster AI, Blondell RD, Looney SW (1997) The practicality of using the SMAST and AUDIT to screen for alcoholism among adolescents in an urban private family practice. J Ky Med Assoc 95:105–107. Grant BF (1997) Prevalence and correlates of alcohol use and DSM-IV alcohol dependence in the United States: results of the National Longitudinal Alcohol Epidemiologic Survey. J Stud Alcohol 58:464 – 473. Green M (ed) (1994) Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents. National Center for Education in Maternal and Child Health, Arlington, VA. Hicks RD, Batzer WB, Batzer WB, Imai WK (1993) Psychiatric, developmental, and adolescent medicine issues in adolescent substance use and abuse. Adolesc Med State Art Rev 4:453– 468. Knight J (2001) The role of the primary care provider in preventing and treating alcohol problems in adolescents. Ambul Pediatr 1:150 –161. Knight J, Goodman E, Pulerwitz T, DuRant R (2000) Reliabilities of short substance abuse screening tests among adolescent medical patients. Pediatrics 105:948 –953. Knight J, Goodman E, Pulerwitz T, DuRant R (2001) Reliability of the Problem Oriented Screening Instrument for Teenagers (POSIT) in an adolescent medical clinic population. J Adolesc Health 29:125–130. Knight J, Shrier L, Bravender T, Farrell M, VanderBilt J, Shaffer H (1999) A new brief screen for adolescent substance abuse. Arch Pediatr Adolesc Med 153:591–596. Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G (2002) Validity of the CRAFFT substance abuse screening test among general adolescent clinic patients. Arch Pediatr Adolesc Med 156:607– 614. Kokotailo P (1995) Physical health problems associated with adolescent substance abuse. National Institute of Drug Abuse Research Monograph 156:112–129. Latimer WW, Winters DC, Stinchfield RD (1997) Screening for drug abuse among adolescents in clinical and correctional settings using the Problem Oriented Screening Instrument for Teenagers. Am J Drug Alcohol Abuse 23:79 –98. Magruder-Habib K, Stevens HA, Alling WC (1993) Relative performance of the MAST, VAST, and CAGE versus DSM-III-R criteria for alcohol dependence. J Clin Epidemiol 46:435– 441. Mayfield D, McLeod G, Hall P (1974) The CAGE questionnaire: validation of a new alcoholism screening instrument. Am J Psychiatry 131: 1121–1123. McLaney MA, Del Boca F, Babor T (1994) A validation study of the Problem Oriented Screening Instrument for Teenagers (POSIT). J Ment Health 3:363–376. Miles H, Winstock A, Strang J (2001) Identifying young people who drink too much: the clinical utility of the five-item Alcohol Use Disorders Identification Test (AUDIT). Drug Alcohol Rev 20:9 –18.
73
National Institute on Alcohol Abuse and Alcoholism (2001). Number of deaths and age-adjusted death rates per 100,000 population for categories of alcohol-related mortality, United States and States, 1979 –96, National Institute on Alcohol Abuse and Alcoholism, Bethesda, MD. Available at: http://www.niaaa.nih.gov/databases/armort01.txt. Accessed February 1, 2002. O’Hare T, Tran TV (1997) Predicting problem drinking in college students: gender differences and the CAGE questionnaire. Addict Behav 22:13–21. Platt RW, Hanley JA, Yang H (2000) Bootstrap confidence intervals for the sensitivity of a quantitative diagnostic test. Stat Med 19:313–322. Rahdert ER (ed) (1991) The Adolescent Assessment/Referral System Manual. U.S. Department of Health and Human Services (PHS) Alcohol, Drug Abuse, and Mental Health Administration, Washington, DC. Robis LN, Przybeck TR (1985) Age of onset of drug use as a factor in drug and other disorders. National Institute of Drug Abuse Res Monogr 56:178 –192. Rutter CM (2000) Bootstrap estimation of diagnostic accuracy with patient-clustered data. Acad Radiol 7:413– 419. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M (1993) Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption–II. Addiction 88:791– 804. Society for Adolescent Medicine (1995) Guidelines for adolescent health research. J Adolesc Health 17:264 –269. Soderstrom CA, Smith GS, Kufera JA, Dischinger PC, Hebel JR, McDuff DR, Gorelick DA, Ho SM, Kerns TJ, Read KM (1997) The accuracy of the CAGE, the Brief Michigan Alcoholism Screening Test, and the Alcohol Use Disorders Identification Test in screening trauma center patients for alcoholism. J Trauma 43:962–969. Steinweg DL (1993) Alcoholism: the keys to the CAGE. Am J Med 94:520 –523. Wagenaar AC, Harwood EM, Toomey TL, Denk CE, Zander KM (2000) Public opinion on alcohol policies in the United States: results from a national survey. J Public Health Policy 21:303–327. Winters K, Henly G (1993) Adolescent Diagnostic Interview (ADI) Manual. Western Psychological Services, Los Angeles. Winters KC, Latimer W, Stinchfield RD (1999) The DSM-IV criteria for adolescent alcohol and cannabis use disorders. J Stud Alcohol 60:337– 344. Winters K, Stinchfield R, Fulkerson J, Henly GA (1993) Measuring alcohol and cannabis use disorders in an adolescent clinical sample. Psychol Addict Behav 7:185–196. Winters KC, Stinchfield RD, Henly GA, Schwartz RH (1990) Validity of adolescent self-report of alcohol and other drug involvement. Int J Addict 25:1379 –1395. Wolraich ML, Felice ME, Drotar D (eds) (1996) The Classification of Child and Adolescent Mental Diagnoses in Primary Care (DSM-PC). American Academy of Pediatrics, Elk Grove Village, IL.
