Does Size Matter?

Does Size Matter? Salience of Follow-Up Payments in Drug Abuse Research BY JASON R. CROFT, DAVID S. FESTINGER, AND BETH J. ROSENWASSER linical outcomes studies frequently provide payment incentives to research participants for completing follow-up assessments. The payments are intended to compensate participants for their time and effort and to encourage those who might otherwise be difficult to relocate to stay in contact with the research team. Maintaining adequate follow-up rates is of particular concern in drug abuse research studies, which are often hindered by substantial attrition of research subjects prior to follow-up assessments. Dozens of drug abuse treatment outcome studies report average attrition rates ranging from 20% at six months postadmission to 35% at 12 months postadmission to 65% at 36 months postadmission.1 A meta-analysis of 85 drug abuse prevention studies similarly reported average attrition rates ranging from 27% at 12 months postadmission to 33% at 36 months postadmission.2 Importantly, these reviews and meta-analyses only included published studies, and it is reasonable to assume that many studies with lower follow-up rates were never published for the very reason that their follow-up rates were deemed to be unacceptable. Indeed, follow-up rates in large-scale

C

Jason R. Croft, David S. Festinger, Karen L. Dugosh, Douglas B. Marlowe, and Beth J. Rosenwasser, “Does Size Matter? Salience of Follow-Up Payments in Drug Abuse Research,” IRB: Ethics & Human Research 29, no. 4 (2007): 15-19.

IRB: E T H I C S & H U M A N R E S E A RC H

KAREN L. DUGOSH, DOUGLAS B. MARLOWE,

community evaluations of drug abuse treatment programs often fall well below an acceptable threshold of 70% and have been as low as 47%.3 Attrition from research cannot be assumed to be random or ignorable.4 The likelihood that a research subject will complete a follow-up assessment is influenced by such confounding factors as the participant’s demographic and drug-use characteristics, as well as the nature of the research interventions.5 This can contribute to systematic differences in attrition rates between research conditions, which would constitute a serious threat to the internal validity of a study. For example, a larger proportion of participants in an experimental treatment arm of a study might be lost to follow-up as compared to control participants attending treatment as usual. This could have the effect of undermining random assignment by making the experimental and control conditions systematically different from each other on outcome-biasing factors. Because such confounding effects may go unmeasured or undetected, they can never be confidently ruled out, thus leaving the findings in irresolvable doubt. Moreover, it cannot be confidently determined whether the follow-up sample sufficiently represented the original baseline cohort, which presents a serious threat to the external validity or generalizability of the study.6 Differential attrition also raises ethical concerns about the potential

for systematic exclusion of certain subgroups of clients from treatmentoutcome studies, such as clients who are geographically transient, emotionally withdrawn, or socially disenfranchised. If such individuals selfselect out of research studies, then new treatment-related findings and therapeutic advances might be less applicable to those very clients who may need the services the most. This process could potentially violate the ethical principle of justice, which necessitates publicly funded research programs to benefit all members of society in a fairly equivalent manner.7 One strategy that has proven successful in improving follow-up rates is the use of payment incentives. The literature on “contingency management” or “operant conditioning” is replete with evidence that payments can improve a wide range of behaviors among substance abusers, including increasing treatment attendance and drug abstinence,8 as well as enhancing recontact rates in research.9 In a prior study, consenting research subjects recruited from drug-free, outpatient substance abuse treatment programs were randomly assigned to receive various magnitudes of payment incentives ($10, $40, or $70) in either cash or gift certificates for attending a six-month follow-up research assessment. Results revealed that subjects who received higher-magnitude cash incentives had significantly better follow-up rates and reported greater J U LY-A U G U S T 2007

15

satisfaction with the research study.10 Importantly, neither the magnitude nor mode of the payment incentives influenced participants’ perceptions of coercion to participate in the research or increased new instances of drug use as determined by quantitative urine drug-screen laboratory analyses. Finally, participants receiving higher-magnitude cash incentives were significantly easier for the research staff to locate again, as evidenced by the fewer tracking efforts and phone calls required to reach them. The current analyses were performed on additional data derived from that study in an effort to better understand the mechanism of action of these effects. Laboratory studies have long demonstrated that performance on memory tests can be significantly improved if research participants received incentives of greater value for completing the recall tasks.11 In contrast, low-magnitude incentives had little impact on memory of study-related information. It is possible that low-magnitude incentives might exert weak effects on follow-up rates, at least in part, because participants view them as too inconsequential to remember. Lacking appreciable salience, the incentives may soon be forgotten. This could contribute to researchers’ difficulties recontacting participants, who may quickly forget about the study and thus fail to return phone calls or respond to mailings. The current study examined whether research participants were more likely to recall higher-magnitude cash incentives after six months, and whether this would elicit more productive or efficient follow-up efforts. The Institutional Review Board (IRB) at the Treatment Research Institute and the State of Delaware approved the study.

Methods „ Participants. Research participants (N = 220) were recruited from

16

J U LY-A U G U S T 2007

three outpatient substance abuse treatment programs located in Philadelphia, Pennsylvania. The consent rate was over 95% in each program, and there were no differences on demographic or drug-use variables between programs. The mean age of study participants was 38.80 (SD = ± 9.90) years. The majority of the sample was male (65%), African-American (75%) or Caucasian (15%), high school educated (11.55 ± 1.78 years of education), never married (64%), and unemployed (57%). The mean reported annual income was $12,048 (± $19,096). The most prevalent substances of abuse in the sample were cocaine (60%), alcohol (48%), heroin (18%), and cannabis (18%). A check on randomization confirmed that there were no differences on any of these demographic or drug-use variables across the six research conditions described below. „ Procedures. Using block-randomization, participants were assigned into six cells of approximately equal size. The six cells crossed three levels of payment magnitude ($10, $40, or $70) with two levels of payment mode (cash or gift certificates). As part of the consent process, all participants were informed of their rights as research participants and were given a detailed description of study procedures, including the randomization process and type of assessments to be administered. At intake, participants completed a brief research battery that was administered by trained research technicians. This battery included a detailed locator form to assist with follow-up efforts. The locator form inquired about addresses and phone numbers where the participants might be reached, as well as names and contact information for other people who might know of the participants’ whereabouts, including family members, friends, therapists, physicians, employers, welfare caseworkers, and probation or parole officers. All par-

ticipants were then scheduled to attend a follow-up appointment six months postadmission and were given an appointment card listing the date and location of the appointment, the amount they would be remunerated ($10, $40, or $70), and the type of remuneration (cash or gift certificate). The gift certificates were redeemable at a popular retail superstore with several locations in the surrounding area. We implemented a highly standardized platform of follow-up efforts to ensure that participants were treated equivalently across the research conditions. The follow-up procedures were intended to represent typical practice in many research studies. Participants received a postcard in the mail two weeks prior to their scheduled follow-up appointment to remind them about the nature of the study and where and when to report for the assessment. Additional efforts to reach participants by telephone were begun two weeks prior to the scheduled appointment and ended four weeks following the originally scheduled appointment if the participant had not yet been contacted. Once a participant was personally contacted by telephone and reminded about the follow-up appointment, no further follow-up efforts were made. Participants who attended the sixmonth follow-up were asked to provide a urine specimen that was tested for amphetamines, cannabis, cocaine, opiates, and phencyclidine. In addition, participants completed an Addiction Severity Index (ASI) and a modified version of the Perceived Coercion Scale of the MacArthur Admission Experience Survey, which assessed their perceptions of having been coerced or negatively influenced to participate in the study. Following completion of the assessment, participants received their previously assigned incentive payment and were asked to return again for a post-follow-up appointIRB: E T H I C S & H U M A N R E S E A RC H

Results

O

ut of the total of 220 participants, 134 (61%) were successfully recontacted by telephone. A chi-square analysis confirmed that there was no difference in the proportions of individuals who were contacted by telephone across the six study conditions, χ 2(5) = 5.43, p = .37. Although greater tracking efforts were required to reach participants in the lower-magnitude and gift-certificate conditions, equivalent percentages of participants were ultimately contacted in each of the conditions. In addition, there were no differences on demographic or druguse characteristics between partici-

IRB: E T H I C S & H U M A N R E S E A RC H

Figure 1. Percent of Participants’ Correct Recall by Type and Magnitude Cash Gift Certificate % of Participants

ment three days later. At this postfollow-up appointment, participants were asked to provide a second urine specimen, complete a research satisfaction survey, and answer questions about how they used or planned to use their incentive payments. All participants received a $40 gift certificate for attending the post-follow-up appointment. „ Dependent Variables. When participants were first contacted by telephone, they were asked if they could recall the amount and type of incentive they were assigned to receive at the follow-up appointment. Their responses were recorded dichotomously (correct or incorrect) for recall of the magnitude ($10, $40, or $70) of the incentive, as well as for recall of the mode (cash or gift certificate) of the incentive. Regardless of whether participants correctly recalled the details of their incentives, they were given the correct information and reminded of the time, date, and location of their follow-up appointment. The research technicians carefully logged whether they were able to reach each participant personally by telephone, as well as the number of phone calls that were required to contact each participant during the six-week follow-up window.

$10 Magnitude p