eg, parking problems, waiting in line the punishment ... - Europe PMC

JOURNAL OF APPLIED BEHAVIOR ANALYSIS

1985, 189 315-321

NUMBEFL

4 (wiNTER 1985)

IMPROVING PEDIATRIC APPOINTMENT KEEPING WITH REMINDERS AND REDUCED RESPONSE REQUIREMENT PATRICK C. FRIMAN, JACK W. FINNEY, MicHAEL A. RAPOFF, EDWARD R. CHRISTOPHERSEN

AND

UNIVERSITY OF KANSAS MEDICAL CENTER AND JOHNS HOPKINS SCHOOL OF MEDICINE

We evaluated the effectiveness of appointment reminders and a reduced response requirement for improving appointment keeping in a hospital ambulatory pediatric clinic. Participants received mailed and telephoned reminders along with a parking pass that reduced the time and effort required to attend the clinic. A multiple baseline analysis of 5,261 appointments over one fiscal year showed that the intervention increased the percentage of appointments kept and decreased the percentage of appointments broken in the continuity clinics of five pediatric health care providers. Social validation, consumer satisfaction, and cost-effectiveness measures, as well as an interrupted time-series analysis, all support the effectiveness of the intervention. DESCRIPTORS: appointment keeping, response requirement, reminders, compliance, behavioral pediatrics

Hospital ambulatory pediatric clinics are notorious for their high percentage of missed appointments (Pearce, O'Shea, & Wessen, 1979). Keeping a pedatric appointment at a hospital involves a complex response chain that begins with "remembering" the appointment and follows with many other responses, some of which are inherently punishing (e.g., parking problems, waiting in line with a sick child). One way to increase the probability that an appointment will be kept is to change the punishment/reinforcement ratio for appointment keeping either by decreasing the "response requirement" (Luce, Christian, Lipsker, & Hall, 1981; Solomon, 1948) or increasing the reinforcement for keeping appointments (Reiss, Piotrowski, & Bailey, 1976). Given the number of components of the appointment keeping response chain, an effective intervention may require combining several procedures. Intervention research on appointment keeping

has primarily focused on the mailed or telephoned appointment reminder (Frankel & Hovell, 1978). Much of the research suggests that reminders can improve appointment keeping (e.g., Nazarian, Mechaber, Charney, & Coulter, 1974). But, some studies suggest that reminders are either not effective (e.g., Barkin & Duncan, 1975) or that their effects are transitory (Morse, Coulter, Nazarian, & Napodano, 1981). The mixed results in the research on reminders may be due to the many methodological problems found in that literature (Frankel & Hovell, 1978). Given those problems, the utility of reminders remains in question. There are other unanswered questions in the research on appointment reminders. The social significance of broken appointments and the acceptability of the interventions to increase kept appointments are seldom assessed by questioning health providers and their patients (Wolf, 1978). Assessments of the reliability of dependent variables and integrity of independent variables are Preparation of this manuscript was partially supported by often not reported (e.g., Barkin & Duncan, 1975; grants from NICHD (HD 03144 and HD 02528) to the Bureau of Child Research, University of Kansas. This re- Morse et al., 1981; Nazarian et al., 1974). Finally, search was submitted by Patrick Friman to the Department although cost-effectiveness data from studies of reof Human Development in partial fulfillment of the requireminders in an adult mental health clinic (Turner ments for the PhD degree. We gratefully acknowledge the assistance of Barbara & Vernon, 1976) and in a dental clinic (Reiss et Cochrane, Ruth Cargo, Holly Richmond, Rosa Meagher, al., 1976) indicate that reminders may be costRuss Terry, Jim Rengel, and Janet Levy. Requests for reprints should be sent to Patrick Friman, effective, the generality of these findings for the Department of Psychology, The Kennedy Institute, 707 use of reminders in pediatric clinics has yet to be North Broadway, Baltimore, Maryland 21205. established. 315

316

PATRICK C. FRIMAN et al.

We evaluated a reminder package in a hospital pediatric clinic. The package included mailed and telephoned reminders, as well as a parking pass that reduced the response requirement necessary to keep an appointment. We also assessed the social significance of broken appointments, the acceptability of the intervention to patients and providers, the cost-effectiveness of the intervention, the reliability of the dependent measures, and the integrity of the independent variable.

METHOD Participants and Setting The participants in this study were the parents of patients in a hospital pediatric clinic seen by five pediatric providers (two pediatricians and three nurse practitioners). The general pediatrics clinic has a broad referral base in the greater Kansas City, Kansas, area but primarily serves the lower to middle income population of urban Kansas City. Appointments in the dinic are scheduled in person at the dinic desk or by telephone. Child health supervision, evaluation of acute illness, and acute and chronic illness follow-up comprise the majority of the health services delivered by the pediatric providers who participated in the study. Separate analyses by appointment type were not available. Measures Data were collected from each provider's daily dinic sheet where appointments were marked "kept" (KA), "cancelled" (CA), or "broken" (BA) by clinic clerks. Only appointments scheduled 5 or more weekdays in advance were induded in the analysis. Appointments could be cancelled by notifying the clinic at least 1 hour before the appointment. The percentage of kept appointments was calculated with the formula KA/(KA + CA + BA). The percentage of broken appointments was calculated with the formula BA/(BA + CA + KA).

providers' weekly clinics and a withdrawal in one of the baselines (Baer, Wolf, & Risley, 1968). A withdrawal was used with one provider's data because of an extended pregnancy leave during the intervention phase.

Experimental Procedures Before baseline, all five pediatric providers were informed of all details of the study except for when the intervention would begin for their patients. Baseline. Baseline measures involved only the calculation and recording of weekly percentages of appointments kept and broken. Reminders and reduced response requirement were not used during baseline. Mailed reminders. At least 5 weekdays before each patient's appointment, a reminder with the patient's name and the day, time, and date of the appointment were sent to the patient's parents. Telephoned reminders. At least 24 hours prior to each patient's appointment, a research assistant telephoned the parent and delivered a spoken reminder that specified the name of the patient, the time, the day, and the date of the appointment, as well as the name of the provider. Reduced response requirement. Included with the reminder was a parking pass that allowed the parents to park in a lot adjacent to the clinic. The passes were routinely given to parents at the clinic, but had to be picked up at the clinic desk and then placed on their car. Thus, mailing the pass saved the parent a trip to and from their car.

Reliability Reliability of the data recorded on the clinic sheets was assessed by comparing the clinic sheet information on 612 randomly chosen appointments with the information recorded by providers in the patient charts. Providers documented kept

appointments with a chart note; broken and cancelled appointments were documented by clerical staff. Only one disagreement was noted, yielding an agreement measure above 99%. Additional reDesign liability data were obtained from an observer coThe effects of the experimental procedures were vertly posted in the clinic for four 2-hour sessions. evaluated with a multiple baseline across the five The observer recorded patient arrivals, nonarrivals,

IMPROVING PEDIATRIC APPOINTMENT KEEPING

317

and cancellations for 78 appointments and checked the total number of appointments during the his record against the clinic sheet at the end of the phase = amount billed per appointment scheduled during that phase. The difference was then mulday. No disagreements were noted. tiplied by the clinic's average percentage of bills Independent Variable Integrity collected. Two methods were used to assess the integrity of the independent variable (Peterson, Homer, & RESULTS Wonderlich, 1982). First, 14% of the mailed reminders and passes were returned to the clinic sugAppointment keeping. Data were analyzed for gesting that as many as 86% of the mailed re- all kept and broken appointments across experiminders reached their subjects. Second, a maximum mental conditions. Appointment keeping rates were of three attempts were made for each telephoned increased and broken appointments decreased as a reminder, and 64% of calls were completed. Al- result of the intervention. Decreases in the perthough there were a number of participants who centage of broken appointments were not attribdid not receive a mailed or a telephoned reminder, utable to cancellations. A visual analysis of Figure their data are included so as to not compare dif- 1 suggests that an increase in the percentage of ferent groups in baseline and intervention. Unfor- appointments kept and a decrease in the percenttunately, those data cannot be retrieved for a sep- age of appointments broken occurred across all arate analysis. It seems probable, however, that the providers during the intervention phase. An interindusion of those data resulted in a conservative rupted time-series analysis quantified these differestimate of the intervention's effect. ences for four of the providers and showed that they were statistically significant. The mean differSocial Validation ence for kept appointments for Pediatric Nurse We assessed social validation to establish the Practioner (PNP) 1 was 60.9% baseline compared significance of broken clinic appointments and the with 75.1% intervention (p < .05); for Medical effects of the intervention on broken appointments Doctor (MD) 1 was 5 7.8% baseline compared with with a questionnaire completed by providers. We 77% intervention (p < .0 1); for PNP 2 was also assessed consumer (patient) satisfaction with 48.65% baseline compared with 75.1% interventhe procedures through separate calls to a sample tion (P < .01); and for PNP 3 was 59.1% baseof parents. The sample induded all parents with line compared with 77.8% intervention (p < .0 1). appointments in the first week of the study when The mean difference for broken appointments for all five providers were in a treatment phase (i.e., PNP 1 was 25.6% baseline compared with 10.1% Week 41). During the call, the parents were asked intervention (p < .01); for MD 1 was 31.37% if they wanted the procedures to continue. baseline compared with 11% intervention (p < The cost of the intervention was determined by .01); for PNP 2 was 38.48% baseline compared establishing the salary necessary to compensate a with 13.8% intervention (p < .01); and for PNP person to issue the reminders, as well as the print3 was 27.02% baseline compared with 10% ining, the postage, and the phoning costs for the tervention (p < .0 1). The data for MD 2 were reminders. The revenue from the intervention was not subjected to the interrupted time-series analysis determined by comparing the average amount because there were not enough data points in the billed per appointment during the intervention intervention phase (McCleary & Hay, 1980). The phase with the average amount billed per appoint- overall mean difference for all five providers for ment during baseline. The average amounts billed kept appointments was 56% baseline compared per appointment in baseline and in intervention with 75.3% intervention and for broken appointphases were calculated with the following formula: ments was 30.5% baseline compared with 9.8% sum of the total amount billed during the phase/ intervention.

318

PATRICK C. FRIMAN et al.

(,) cn

LLU I-

0. 0L.

Lo cc L&J

w z LUI 0.

s

to

Is

26

2S

n

A3

66

41

a6

Ss

CONSECUTIVE WEEKS Figure 1. Percentage of appointments kept (cirdes) and broken (triangles) during each week of the study. PNP denotes pediatric nurse practitioner and MD denotes medical doctor.

Social validation. The social validation data showed that clinic providers considered broken appointments to be a significant problem (see Table 1). The data also showed that the providers (a) noticed a positive result following the intervention, (b) identified acceptable kept and broken rates that were almost identical to the rates that occurred following the intervention, and (c) were satisfied with the results. Finally, in the assessment of consumer satisfaction 97 parents were asked about the intervention. Of those parents, 88 had received the full intervention, and all 88 said they wanted it to continue.

Cost-effectiveness. The amount of money billed per appointment scheduled increased across all providers yielding a cumulative increase of 18%. The average amount billed per appointment scheduled was $17.98 during baseline and $22.08 during the intervention yielding an average increase of $4.10. The clinic's collection rate was 78%, thus the estimated increase in revenue per appointment scheduled was $3.20 ($4.10 x 78%). There were 1,783 appointments scheduled during the intervention yielding an estimated increased revenue of $5,706 (1,783 x $3.20). The total costs of the described intervention for 1,783 appoint-

IMPROVING PEDIATRIC APPOINTMENT KEEPING Table 1 Social Validation Items

Responses

1. Increasing the percentage of appointments kept by my patients would result in improved medical care. Strongly Strongly agree Agree Neutral Disagree disagree 4 1 2 5 3 2. Broken appointments (DNKAs) are a problem in my dinic. Strongly Strongly Agree agree Neutral Disagree disagree 4 2 5 1 3 3. Were you induded in an appointment keeping study in fiscal year 1983? Yes No 4. If the answer to (3) was yes did you notice a change in appointments kept and broken? Yes No 5. What kind of change? Appointments kept: Substantial Moderate No Moderate Substarntial increase increase decrease change decreaase 5 4 2 3 1 Appointments broken: Substantial Moderate No Moderate Substantial increase increase change decrease decrease 4 5 2 1 3 6. The lowest acceptable kept rate for me is: 60% 50% 70% 80% 90% 7. The highest acceptable broken rate for me is: 10% 20% 30% 40% 50% 8. The results of this study for me were: Very Very satisfying Neutral Satisfying Dissatisfying disatisfying 4 5 2 3 1

ments, including personnel salary and fringe, and postage, phoning, and printing was estimated at $2,62 1. Therefore, the estimated additional net

the

revenue

accruing from this intervention was $3,085

($5,706

-

$2,621).

DISCUSSION Mailed and telephoned reminders combined with a parking pass that reduced response requirement significantly improved appointment keeping in a

M = 4.6

M = 4.2 100% yes

100% yes

M = 4.4

M = 1.4

M = 76% M = 14%

M = 4.4

hospital pediatric clinic. Social validation indicated that parents of patients were pleased with the reminder package and that pediatric providers rated broken appointments as a significant problem that was improved by the package. The intervention package was cost-effective: The increased billing for the increased number of kept appointments covered the cost of the package and resulted in increased revenue for the hospital. Reminders to improve appointment keeping are not new but combining reminders with reduced

320

PATRICK C. FRIMAN et al.

response requirement is a novel approach to improving appointment keeping. Furthermore, experimental evaluation of a reminder intervention with social validation, a cost-effectiveness analysis, reliability, and data on the integrity of the independent variable extend the research on appointment keeping. Maintenance of treatment effects across time is an important consideration for appointment keeping interventions. Morse et al. (1981) conducted a follow-up to the Nazarian et al. (1974) study and concluded that the effects of appointment reminders are transient. But the kept rates in the initial study were identical with the kept rates in the follow-up study (i.e., 64%). Thus, the data from Morse et al. (1981) could be used to support another condusion: that the effects of mailed reminders are stable over time. That Morse et al. conducted a 7-year follow-up is commendable. The single outcome measure allowed by their group study, however, may not be the most appropriate method of showing maintenance (or lack thereof) over time. We used repeated measures, and the effects of the intervention were stable across many weeks. There is no indication that the effects of the package waned, even in the longest treatment phase (i.e., 21 weeks, PNP 1, Figure 1). Nevertheless, an optimal analysis for future study would use repeated measures until stability is achieved in the data and then collect data at extended followup points (cf. Baer et al., 1968). Generality across settings is also an important concern. Other clinics may not have the parking arrangements found at the clinic in this study, rendering replication of the response requirement component impossible. But all clinics will have different types of response requirements associated with appointment keeping, the reduction of which could improve appointment keeping when used with reminders. Examples include reducing the waiting time to see the health care provider, providing babysitting services for siblings of patients, or providing transportation to the clinic. Component analyses are needed for package interventions used to improve medical compliance. Our assessment of the integrity of the independent

variable showed that over one-third of parents received only part of the intervention package (e.g., 36% of parents could not be contacted by telephone). Thus, on one hand, the improvements in appointment keeping could be even greater in settings where a higher percentage of patients received the entire package. On the other hand, there may be components of the package that do not contribute to improved appointment keeping. A component analysis of this study's intervention package would address that issue. A significant topic for further health care research is the determination of the health benefits that result from medical compliance (Finney, Friman, Rapoff, & Christophersen, 1985). This study suggests that hospital pediatric clinics could improve compliance with medical appointments, a possible prerequiste to improving the health of children, by reminding parents when their children are scheduled for medical services and by making it easier for parents to obtain those services. Whether such compliance is, in fact, a prerequisite to improving health has yet to be established. REFERENCES Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91-97. Barkin, R. M., & Duncan, B. (1975). Broken appointments: Questions, not answers. Pediatrics, 55, 747748. Finney, J. W., Friman, P. C., Rapoff, M. A., & Christophersen, E. R. (1985). Improving compliance with antibiotic regimens for otitis media: Randomized clinical trial in a pediatric clinic. American Journal of Diseases of Children, 139, 89-95. Frankel, B. S., & Hovell, M. F. (1978). Health service appointment keeping: A behavioral and critical review. Behavior Modification, 2, 435-464. Luce, S. C., Christian, W. P., Lipsker, L., & Hall, R. V. (1981). Response cost: A case for specificity. Behavior Analyst, 4, 75-81. McCleary, R., & Hay, R. A. (1980). Applied time series analysis for the social sciences. Beverly Hills: Sage. Morse, D. L., Coulter, M. D., Nazarian, L. F., & Napodano, R. J. (1981). Waning effectiveness of mailed reminders on reducing broken appointments. Pediatrics,

68, 846-849. Nazarian, L. F., Mechaber, J., Charney, E., & Coulter, M. D. (1974). Effect of a mailed appointment reminder on appointment keeping. Pediatrics, 53, 349-352.

IMPROVING PEDIATRIC APPOINTMENT KEEPING Pearce, T., O'Shea, J. S., & Wessen, A. F. (1979). Correlations between appointment keeping and reorganization of hospital ambulatory pediatric services. Pediatrics, 64, 81-87. Peterson, L., Horner, A. L., & Wonderlich, S. A. (1982). The integrity of the independent variable in behavior analysis. Journal of Applied Behavior Analysis, 15, 477-493. Reiss, M. L., Piotrowski, W. D., & Bailey, J. S. (1976). Behavioral community psychology: Encouraging low-income parents to seek dental care for their children. Journal of Applied Behavior Analysis, 9, 387-397.

321

Solomon, R. L. (1948). Effort and extinction rate: A confirmation. Journal of Comparative and Physiological Psychology, 41, 93-101. Turner, A.J., & Vernon,J. C. (1976). Prompts to increase attendance in a community mental health center. Journal of Applied Behavior Analysis, 9, 141-145. Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-215. Received February 4, 1985 Final acceptance September 18, 1985