Substance Use & Misuse, 49:1303–1310, 2014 C 2014 Informa Healthcare USA, Inc. Copyright ISSN: 1082-6084 print / 1532-2491 online DOI: 10.3109/10826084.2014.897730
ORIGINAL ARTICLE
Estimating the Prevalence of Illicit Drug Use Among Students Using the Crosswise Model Mansour Shamsipour1 , Masoud Yunesian2 , Akbar Fotouhi1 , Ben Jann3 , Afarin Rahimi-Movaghar4 , Fariba Asghari5 and Ali Asghar Akhlaghi1 1
Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran; 2 Department of Environmental Health Engineering, School of Public Health and Institute of Public Health Research, Tehran University of Medical Sciences, Tehran, Iran; 3 Sociology, University of Bern, Bern, Switzerland; 4 Imam Khomeini, Iranian National Center for Addiction Studies (INCAS), Iranian Institute for Reduction of High-Risk Behaviors, Tehran University of Medical Sciences, Tehran, Iran; 5 Medical Ethics and History of Medicine Research Center, Tehran University of Medical Sciences, Tehran, Iran (Ahmadi et al., 2001, 2004, 2006). It can be said that concern is often declared about the validity of survey data on illicit drug use. Data on illicit drug use are often collected using anonymous self-administered questionnaires. This approach is frequently criticized because relying on such self-reports gives rise to a number of methodological challenges. Respondents might refuse to answer the sensitive questions, leading to item nonresponse, or they might decide to provide a wrong answer, causing the resultant prevalence estimates to be biased. Indeed, reviews of bias in self-reported substance abuse indicate significant underreporting (;Griesler et al., 2008; Van Griensven et al., 2006; Weissman et al., 1986; Whitehead & Smart, 1972).
Objective: The aim of our study is to compare the prevalence of illicit drug use estimated through a technique referred to as the “crosswise model” (CM) with the results from conventional direct questioning (DQ). Method: About 1,500 students from Tehran University of Medical Sciences 2009–2010 were first interviewed by DQ and, then three months later, by the CM. Result: The CM yielded significantly higher estimates than DQ for lifetime prevalence of use of any illicit drug (CM = 20.2%,DQ = 3.0%, p < .001) and for lifetime prevalence of use of opium or its residue (CM = 13.6%, DQ = 1.0%, p < .001). Also, for use of any illicit drug in the last month and use of opium or its residue in the last month, the CM yielded higher point estimates than DQ, although these differences were not significant (any drug: CM = 1.5%, DQ = 0.2%, p = .66; opium: CM = 3.8%, DQ = 0.0%, p = .21). Conclusion: Our findings suggest that the CM is a fruitful data collection method for sensitive topics such as substance abuse.
Drug Testing
Although the available chemical drug test(s) were used to judge the validity of self-reported data on illicit drug use (Harrison, 1997; Kim & Hill, 2003), but they have their own limitations, urine tests have a narrow window of detectability, assays cannot determine the age of initiation of illicit drug use and individuals’ attitudes toward the risk of harm, which greatly reduce their usefulness (Harrison, 1995), also there may be many research contexts where testing is problematic, unfeasible, or impractical. Validation studies comparing self-reported drug use to laboratory analysis of hair and urine find substantial underreporting between 30 and 80% of respondents who tested positive, yet not admitting drug use (Fendrich et al., 1999; Feucht et al., 1994; Tourangeau & Yan, 2007). Although most of the surveys are confidential, as researchers most commonly do have, at least at some point, the ability to link completed interviews with potential identifiers,
Keywords sensitive questions, illicit drug use, substance abuse, crosswise model, randomized response technique, survey methodology, Iran
INTRODUCTION Prevalence Estimation
Precise prevalence estimation of illicit drug use is vital to the planning and evaluation of intervention programs. The results of existing studies about the prevalence of illicit drug use vary, and it is uncertain whether accurate prevalence rates are actually higher than those reported or not
Address correspondence to Masoud Yunesian, Department of Environmental Health Engineering, School of Public Health and Institute of Public Health Research, Tehran University of Medical Sciences, Tehran 1417653761, Iran; Email:
[email protected]
1303
1304
M. SHAMSIPOUR ET AL.
respondents answering sensitive questions might fear a breach of confidentiality. Such doubts might lead respondents to refuse to answer sensitive questions or to answer untruthfully. Alternative Survey Strategies
To overcome these problems, several methods such as the Randomized Response Technique (RRT) have been developed (see, e.g., Fox & Tracy, 1986; Lensvelt-Mulders et al., 2005). In RRT, respondents use a randomization device with a known probability distribution (e.g., coins, dice, or cards) to generate a probabilistic relationship between respondents’ answers and the true values of the sensitive characteristic. The RRT can therefore be used to obtain an estimate of the prevalence of a sensitive behavior in a population while at the same time guaranteeing complete anonymity to respondents. Methodological research on the RRT, however, suggests that the technique can be difficult to implement, as respondents do not easily understand it. Moreover, there is evidence that the RRT can yield invalid estimates due to respondents’ deliberate noncompliance with the RRT instructions (Coutts & Jann, 2011; Holbrook & Krosnick, 2010). Crosswise Model
In order to overcome some of the drawbacks of the RRT, Yu et al. (2008) proposed a new approach called the Crosswise Model (CM). The CM is based on a simple idea: Respondents are presented a pair of yes/no questions, a sensitive one and a nonsensitive one, and are then asked to provide a joint answer to both questions, that is, whether (A) the answers to both questions are the same (both “yes” or both “no”) or (B) whether the answers are different (one “yes” and one “no”). Since both A and B are valid answers for respondents to which the sensitive question applies, there is no obvious self-protective answer strategy for respondents who do not want to be associated with the sensitive behavior (Jann et al., 2012). In contrast, in direct questioning as well as in most implementation cases of the RRT, the best strategy to avoid association with the sensitive behavior is just to answer “no,” irrespective of the instructions. The CM is easy to understand and administer and has several other advantages over other RRT schemes. In particular, unlike most implementation cases of the RRT, respondents are not instructed to give a direct answer to the sensitive question, nor are they forced to provide a false “yes” answer. Assuming that respondents trust and understand the CM, the CM may therefore be expected to yield more valid estimates than direct questioning (DQ) or the RRT. This study, to our knowledge, is the first test of the CM for measuring illicit drug use, and will examine the case of use of opium and other illegal drugs among students in Iran. We evaluate the performance of the Crosswise Model (CM) by comparing it to direct questioning (DQ). In other areas also, merely one study has been conducted to test the model, an experimental survey implemented crosswise model and compared it to direct ques-
tioning on plagiarism in Swiss and German students in university classrooms. The results of this study suggest that the CM is a promising data collection method eliciting more socially undesirable answers than direct questioning (Jann et al., 2012). The assumption is that if the CM yields higher prevalence estimates than DQ, then the answers CM gives will presumably be more valid (moreis-better assumption, see Tourangeau & Yan, 2007 and Yu et al., 2008); if the respondents feel more protected by the CM, they will be more willing to admit illicit drug use than in DQ, thus yielding a higher overall prevalence estimate. METHOD Data Collection
For our study, two surveys on illicit drug use among students in Tehran were conducted; one using DQ and one using the CM. The first survey was conducted in January and February 2009. In total, 1,650 students of Tehran University of Medical Sciences attended classroom sessions during this time period. All of them were asked to participate and fill out a self-administered paper-and-pencil questionnaire. A total of 1,568 students were interviewed using direct questioning. To prevent students from taking part in the survey repeatedly, we only selected classes for which there was no overlap in participating students. Questionnaires were distribted at the beginning of each class and students were given time to fill out the questionnaires. The questionnaires contained various questions addressing illicit drug use as well as questions on demographic characteristics. To assess the reliability of the answers provided by the students, the following question was also included: “Think of your closest friend who is sitting in this room and is going to fill out the questionnaire. To what extent do you believe that he/she will provide honest answers to the questions addressing substance use?” (Translated from Persian). Three months later, in April and May 2009, the same body of students was interviewed a second time, now using the CM. Again students who attended classroom sessions during this time period were handed out paper-andpencil questionnaires (N = 1,490). Note that there is a large overlap between the samples of the two surveys, that is, for most part, the same students were interviewed. However, the individual students’ answers to the two surveys cannot be linked because the questionnaires were not personalized. The CM questionnaire contained information on the research objective and instructions for filling out the questionnaire on the first page. In addition, one of the researchers presented an example to the class to illustrate the instructions. The questionnaire continued with some demographic questions, the CM questions, and questions about trust and understanding. Demographic questions included gender, marital status, and residential status. Demographic information was kept at minimum to maintain a high level of privacy protection. The questionnaire contained four sets of CM questions, each pairing a sensitive question about illicit drug use with a nonsensitive question
USING THE CROSSWISE MODEL IN ESTIMATING THE PREVALENCE OF ILLICIT DRUG USE
(see Table 1). The sensitive questions were selected from the ones used in the direct questioning survey. The nonsensitive paired questions asked for information related to birth dates and the last digits of ATM passwords, cell phone numbers, or street addresses, all of which were assumed to follow a uniform distribution and are likely to be uncorrelated with illicit drug use. To assess if respondents provide consistent answers using the CM, we included a fifth set of questions in which two nonsensitive items were paired (see Table 3). Finally, the questionnaire contained two direct questions asking the respondents whether they think they understood the instructions for completing the questionnaire, and how confident they are that the method protects their anonymity.1 Before receiving the questionnaires, respondents were reassured that the research group would not have access to their personal information. Students then filled out the questionnaires in class and returned them to the researchers.
1305
lated, the probability of Y = 1 can be written as Pr (Y = 1) = Pr (S = 1) Pr (C = 1) + [1 − Pr (S = 1)] [1 − Pr (C = 1)] . Solving for Pr (S = 1) ,we can derive an estimator for the prevalence of the sensitive item as: CM (S = 1) = Pr (Y = 1) + Pr (C = 1) − 1 Pr 2 Pr (C = 1) − 1 with N 1 Pr (Y = 1) = Yi N i=1
and Pr (C = 1) = 0.5.
Data Analysis
Standard methods for proportions can be used for the statistical analysis of the data from the DQ survey. In particular, let S be the sensitive item, with S = 1 if the sensitive item applies and S = 0 otherwise. The prevalence of S can then be estimated as N DQ (S = 1) = 1 Si . Pr N i=1
Furthermore, an estimate for the sampling variance of CM (S = 1) an be derived as: Pr CM (S = 1)] = Vˆ [Pr
= 1)] Vˆ [Pr(Y [2 Pr(C = 1) − 1]2
where = 1)] = Pr(Y = 1)[1 − Pr(Y = 1)] . Vˆ [Pr(Y N −1
With sampling variance DQ (S = 1)] = PrDQ (S = 1)[1 − PrDQ (S = 1)] . Vˆ [Pr N −1 For the data from the CM survey, prevalence estimates for the sensitive behaviors can be obtained as follows (Jann et al., 2012; Yu et al., 2008)2 . Again, let S be the sensitive item, which is now unobserved. C is an indicator for the nonsensitive item. The probability that the nonsensitive item applies, Pr (C = 1) is known. In our study, Pr (C = 1) lies between 0.25 and 0.3285, depending on the item. Furthermore, let Y be the observed answer, where Y = 1 if S and C are the same (both “yes” and both “no”), and Y = 0 if S and C are different (one “yes” and one “no”). Assuming that S and C are uncorre1
The wording of the questions was (translated from Persian): “Did you understand the instructions to fill out the questionnaire?” with answer categories “Completely”, “To some extent”, and “Did not understand” and “To what extent do you believe that this questioning technique protects your anonymity?” with answer categories “Highly”, “Moderately”, “Little” and “Not at all”. 2 Note that the Crosswise Model is formally identical to Warner’s original RRT scheme Warner, S.L., 1965. Randomized response: A survey technique for eliminating evasive answer bias, Journal of the American Statistical Association 60: 63–69.
To compare the CM results with the results from DQ, we use two-tailed Z-tests for independent samples, where Z=
CM (S = 1) − Pr DQ (S = 1) Pr DQ (S = 1)] CM (S = 1)] + Vˆ [Pr Vˆ [Pr
is assumed to follow a standard normal distribution. Ethical Considerations
The students were given an explanation of the study objective in the classroom sessions. Furthermore, they were told that participation in the study was completely voluntary, and that they could leave the classroom if they did not want to take part, which only an ignorable number of students did (maximally 10 students overall). The study protocol was approved by the Ethics Committee of the Tehran University of Medical Sciences. RESULTS
Among the 1,568 students in the direct questioning survey, 33.2% were male and 66.8% were female. Likewise, in the CM survey (N = 1,490), there were 33.6% men and 66.4% women. Because illicit drug use behavior is likely to be
1306
M. SHAMSIPOUR ET AL.
TABLE 1. Sensitive items in the CM questionnaire (translated from Persian) Item Any illicit drug: Lifetime Any illicit drug: Last month Opium: Lifetime
Opium: Last month
Sensitive question Have you ever used any illicit drug (cannabis, opium, opium residue, crack, heroin, ice, or ecstasy) in your lifetime? Have you used any illicit drug (cannabis, opium, opium residue, crack, heroin, ice, or ecstasy) during the last month? Have you ever used either opium or its residue in your lifetime?
Take one of your friends or acquaintances whose birthday you remember. Is their birthday between the 1st and10th of the month? (p = .3285) Take the pin code of one of your ATM cards that you use frequently. Is the last digit 5, 6, or 7? (p = .3) Take one of your friends or acquaintances whose cell phone number you know by heart. Is the last digit 1, 2, or 3? (p = .3) Take the street number of your parents or one of your acquaintances, and do not change it. Is the last digit of this number 2, 4, or 6? (p = .3)
Have you used either opium or its residue during the last month?
different among men and women, we will report overall results as well as results by gender. Table 2 displays the prevalence estimates for the different items based on DQ and the CM. The last column shows the difference tests between the questioning modes. The CM yielded consistently higher estimates of illicit drug use than DQ for all items and for both genders. For lifetime use of any illicit drug and for lifetime use of opium, the differences are substantial and highly significant (except lifetime use of opium for men). For example, the overall prevalence of lifetime use of any illicit drug is 20.2% according to the CM, compared to 3.0% based on DQ3 . Likewise, the prevalence of lifetime use of opium is 13.6% according to the CM, whereas the estimate based on DQ is only 1.0%. For drug usage during the last month, the differences between the questioning modes are in the same direction, but less pronounced in absolute terms and insignificant. Note, however, that the true prevalence for these items is generally lower than for lifetime drug usage, which makes it more difficult to find mode effects (due to the relative inefficiency of the CM for probabilities close to 0 or 1). An interesting pattern visible in Table 2 is that the difference between men and women is more pronounced in the DQ survey than in the CM survey. From the DQ results, we would conclude that men have a higher prevalence of lifetime use of any illicit drug and lifetime use of opium than women (p < .001 and p < .01, respectively). According to the CM, however, the difference between men and women is not significant. This suggests that men and women may have similar drug behaviors, but that women are more reluctant to admit substance abuse in a DQ format. If we are willing to accept the more-is-better assumption, that is, that higher prevalence estimates are more valid, then these results indicate that the CM considerably outperformed DQ. The more-is-better assumption, however, can be violated. For example, if a substantial proportion of the respondents are confused by the technique and, essentially, give random answers, the 3
Nonsensitive question
Note that in the DQ study, the estimates for lifetime and last month use of any illicit drug are based on an index from separate questions for different types of drugs (the same types as listed in the question used in the CM study).
CM estimates will be inflated (random answers in the CM push the estimate toward 0.5, leading to a positive bias if the true prevalence is below 0.5). To evaluate the viability of the technique, we therefore included a test item for the CM in which two nonsensitive questions with known distributions were combined. If the technique works properly, it should be possible to accurately recover these distributions from the data. Table 3 shows the results of this test. Applying the CM formulas with the prevalence of one of the two questions set to its theoretical value, the estimate for the other question almost perfectly reflects the expected prevalence. The overall estimate for being born in summer, which we expected to be 25%, is 24.6%. The estimate for the prevalence of the last digits 2, 4, or 6 in the respondents’ birth certificate numbers4 is 29.7%, compared to an expected prevalence of 30%. Similar results would be found if we divided the sample by sex. As such, we conclude that the technique was effective. We also asked the CM respondents whether they think they understood the instructions to fill out the questionnaire, and whether they believed that the questioning technique protected their anonymity. Seventy-six percent of the respondents affirmed that they completely understood the instructions and another 17% indicated that they understood the instructions to some extent, leaving only 7% who thought they did not understand. Furthermore, 89% were highly or moderately confident that their anonymity was protected by the technique and only 11% had little or no confidence. There is a strong positive relationship between understanding the technique and the confidence in the level of anonymity using the technique (χ 2 = 109.6, df = 6, p < .001; Goodman and Kruskal’s γ = 0.36). Although these self-reports indicate that the CM was well understood and trusted, they may not be fully reliable. For example, in the DQ survey, the students were asked whether they believed that their peers would provide honest answers to the drug usage questions or not. A majority of 94% of the respondents expected honest answers and only 6% reported to believe that no or very few honest answers were given. Despite respondents’ confidence in their peers’ honesty, the comparison of the 4
This is a number known to students in Iran because the number is used in many contexts, for example as the password for enrolling in classes.
1307
USING THE CROSSWISE MODEL IN ESTIMATING THE PREVALENCE OF ILLICIT DRUG USE
TABLE 2. Prevalence estimates in percent (standard errors and number of observations in parentheses)
Any illicit drug: Lifetime Total Men Women Any illicit drug: Last month Total Men Women Opium: Lifetime Total Men Women Opium: last month Total Men Women +
Direct Questioning (DQ)
Crosswise Model (CM)
Difference (CM-DQ)
3.0 (0.4, 1520) 6.2 (1.1, 497) 1.4 (0.4, 1023)
20.2 (3.7, 1481) 23.9 (6.4, 497) 18.3 (4.5, 984)
17.2 (3.7)∗∗∗ 17.7 (6.3)∗∗ 16.9 (4.5)∗∗∗
0.2 (0.1, 1520) 0.4 (0.3, 497) 0.1 (0.1, 1023)
1.5 (3.0, 1480) 3.0 (5.2, 500) 0.8 (3.7, 980)
1.3 (3.0) 2.6 (5.2) 0.7 (3.7)
1.0 (0.3, 1533) 2.4 (0.7, 504) 0.3 (0.2, 1029)
13.6 (3.1, 1481) 11.3 (5.3, 498) 14.8 (3.8, 983)
12.6 (3.1)∗∗∗ 9.0 (5.3)+ 14.5 (3.8)∗∗∗
0.0 (0.0, 1533) 0.0 (0.0, 504) 0.0 (0.0, 1029)
3.8 (3.0, 1475) 3.1 (5.2, 496) 4.2 (3.7, 979)
3.8 (3.0) 3.1 (5.2) 4.2 (3.7)
p < .1, ∗ p < .05, ∗∗ p < .01, ∗∗∗ p
