Cluster Randomized Controlled Trial of Intervention to

Download PDF
0 downloads 0 Views 329KB Size Report
Comment
trolled trial of an intervention designed to improve participant retention in ..... guidelines, which included scripts, strategies, and case exam- ples. Nurses from each ... tus, household income, and Medicaid status were assessed at enrollment as ...
Prev Sci DOI 10.1007/s11121-015-0563-x

Cluster Randomized Controlled Trial of Intervention to Increase Participant Retention and Completed Home Visits in the Nurse-Family Partnership David L. Olds 1 & Pilar Baca 1 & Maureen McClatchey 3 & Erin M. Ingoldsby 2 & Dennis W. Luckey 3 & Michael D. Knudtson 1 & Joan M. Loch 4 & Mildred Ramsey 5

# Society for Prevention Research 2015

Abstract We conducted a cluster-based randomized controlled trial of an intervention designed to improve participant retention in community replication sites of the Nurse-Family Partnership (NFP). We registered 26 sites and randomized them into three groups: retention intervention (RI, N=9), delayed RI (DRI, N=6), or control (C, NFP as usual, N=11). The RI consisted of training nurses to give more explicit control over the frequency of visits and content of the program to the parent participants. Two of the sites assigned to the RI, two assigned to the DRI, and two out of four nurses in one other site assigned to the DRI chose not to participate in the intervention. Primary analyses (intention to treat) contrasted changes in participant retention and completed visits (the primary outcomes) in the two intervention groups (RI and DRI) compared to control sites, focusing on differences in performance among baseline cohorts compared to cohorts enrolled during the first year during which the retention intervention was implemented. Compared to baseline, retention declined in the control sites over time but stayed the same in the RI and DRI sites (p value for interaction = 0.099). Compared to

* David L. Olds [email protected] 1

Department of Pediatrics, School of Medicine, University of Colorado Denver, 13121 E. 17th Ave., Mail Stop #8410, Aurora, CO 80045, USA

2

Omni Institute, Denver, Colorado, USA

3

Department of Biostatistics and Informatics, Colorado Public School of Health, University of Colorado Denver, 13121 E. 17th Ave., Mail Stop #8410, Aurora, CO 80045, USA

4

Nurse Family Partnership National Service Office, Denver, CO 80203, USA

5

Oklahoma State Department of Health, Oklahoma City, OK, USA

baseline, the number of completed home visits declined over time in the control sites but did not in the RI and DRI sites, producing a significant treatment difference in change in mean completed home visits over time (2.71 visits, SE=1.164, p= 0.020). The intervention offset a decline over time in retention and completed home visits found in the control group during the time covered by this trial. Quantitative and qualitative evaluation of the intervention indicated that improvements are needed to promote its uptake. Keywords Home . Visits . Retention . Pregnancy . Infancy

Abbreviations NFP Nurse-Family Partnership RI Retention intervention DRI Delayed retention intervention MI Motivational interviewing Despite advances in the identification of evidence-based interventions that prevent crime, substance abuse, and adverse mental health outcomes, little research has been conducted to guide the effective implementation of those interventions in community practice (Institute of Medicine 2009). One of the most significant challenges for community implementation of preventive interventions is participant retention (Brown and Liao 1992; Ingoldsby 2010). Since 1997, the Nurse-Family Partnership (NFP), a program of prenatal and infancy home visiting by nurses for low-income mothers bearing first children, has been implemented in 44 states, with over 350 local NFP sites, in over 550 counties, serving over 200,000 families in the USA (Nurse-Family Partnership n.d.). International expansion is being conducted in six other high-income countries.

Prev Sci

The NFP is designed to improve the health and development of low-income young women and their children by visiting families in their homes from as early in pregnancy as possible through child age 2. The program accomplishes these goals by improving prenatal health behaviors, parents’ care of their children, and parental health and life course (Olds 2002). Since 1977, it has been tested in a series of randomized controlled trials with different populations and shown considerable promise as a means of improving maternal and child health (Olds et al. 2002; Olds et al. 2004; Kitzman et al. 2010; Olds et al. 2010). It has served as the primary source of evidence in support of the Maternal, Infant, and Early Childhood Home Visitation program funded under the Affordable Care Act (Haskins et al. 2009). As the NFP is replicated in new community settings, every site enters data into a web-based information system that is used to monitor and improve its implementation. From these data, we learned that participant retention and completed home visits in community settings, compared to implementation in the original trials, are lower. Sixty-two percent of the participants in the Denver trial of the NFP completed the program (Korfmacher et al. 1999), for example, compared to 35 % of families in community replication, and NFP nurses in community replication are completing 78 % of the visits completed in the original RCTs (O’Brien et al. 2012). In order to identify likely causes of lower retention in community practice, we conducted a mixed method study of retention and completed home visits in NFP implementation sites. We found that higher risk families dropped out more frequently than their lower risk counterparts and that even after adjusting for participant risks, there were highly significant differences among sites and nurses in participant retention and completed home visits. A qualitative analysis indicated that successful sites employed nurses who adapted the content and dose of the program to the needs of the families they served (O’Brien et al. 2012). While the NFP is structured, it is designed to be adapted to families’ needs. Nurses who adapted the program more extensively to families’ needs had greater success in retaining and engaging them than did nurses who followed the prescribed dosage and content with less attention to participant needs (O’Brien et al. 2012). The results of this mixed method study led to the development of a retention intervention (RI), grounded in motivational interviewing (MI) (Miller et al. 2008), in which NFP nurses were supported in giving even greater control over the content and dosage of the program to the families they visited. In a 15site quasi-experimental study, we found that those sites that received the RI had greater improvements in participant retention and completed home visits from baseline to intervention periods than did matched control sites (Ingoldsby, et al. 2013). Although intervention and control groups were remarkably equivalent on baseline attrition and other characteristics, we

were concerned that our selection of RI sites on the basis of their commitment to helping us develop it may have led to the selection of especially motivated sites, which may have accounted for the improvements in retention and completed home visits over time rather than the intervention itself. We decided to test this retention intervention in a randomized controlled trial before pressing it into practice. In order to increase the RI’s potential for nationwide expansion, we adapted it so that it might be taught to new nurses by nurse mentors who had learned to deliver it in the pilot study (Ingoldsby et al. 2013) and in a subsequent small-scale implementation effort designed in part to develop a corps of nurse mentors. We hypothesized that sites assigned to the RI would have significantly greater gains in participant retention and completed home visits over time than those assigned to the NFP-as-usual condition and that RI effects would be greater for sites with initially lower rates of retention.

Methods Participants and Design As shown in Fig. 1, we recruited 26 sites from two states, focusing on sites that revealed a need and interest in improving their rates of participant retention and completed home visits. They were stratified within states and within high or low retention classifications within those states and then randomized by a study statistician (MM), who had no connection with the sites, to one of three conditions: (1) RI (N=9), in which nurses were mentored by nurses who had learned the RI during pilot phases of RI development; (2) delayed RI (N= 6), in which nurses were mentored to implement the RI with participants enrolled in the NFP during the third cohort year (described below); or a (3) control condition (N=11), NFP as usual. We created the DRI condition to cope with a limited number of nurses available to mentor nurses at the beginning of the trial. Volunteer nurse mentors from the RI condition guided DRI nurses to learn the RI with cohort 3. Randomization was conducted at the level of sites rather than individuals to reduce horizontal diffusion of the RI to control participants. The study was conducted with three cohorts of participants enrolled during 12-month periods: The baseline cohort (1) were those NFP program participants (n=1878) registered between October, 2003 through the end of September, 2004; cohort 2 (n=1942) were registered between October, 2005 through the end of September 2006; and cohort 3 (n=2149) were registered between October, 2006 and the end of September, 2007. The baseline cohort preceded cohort 2 by 2 years in order to reduce the likelihood that the baseline cohort in the RI condition would receive the RI at some point during their 2+ years of enrollment in the NFP, as this would attenuate the estimate of change over time for the RI sites. Note

Prev Sci Fig. 1 Flow of sites* and participants in cluster randomized controlled trial of retention intervention (RI)*. To reduce complexity, the flowchart does not separate the sample by state. The number of sites in state A is RI 4, DRI 4, control 8. The number of sites in state B is RI 5, DRI 2, control 3

26 NFP sites Recruited from 2 States Randomized Disproporonately to 1 of 3 Treatment Condions

RI Sites (N=9) Baseline Cohort 1 Registered 10/20039/2004

Retenon and home visits assessed for newly enrolled parcipants in RI sites (n = 929)

Cohort 2 Registered 10/2005 9/2006

RI sites closed/declined (n=2)

Delayed RI Sites (N=6)

Control Sites (n=11)

Retenon and home visits assessed for newly enrolled parcipants in Delayed RI sites (n=317)

Retenon and home visits assessed for newly enrolled parcipants in Control sites (n=632)

New parcipants receive NFP without RI (n = 854)

RI sites accept (n=7) Nurses received training in RI

New parcipants in declined sites (n= 148)

New parcipants not reached (n = 207)

New parcipants reached (n = 733)

Retenon & home visits assessed for Cohort 2 (n = 854)

Retenon & home visits assessed for Cohort 2 (n = 1,088)

Cohort 3 Registered 10/2006 9/2007

Delayed RI sites declined (n=2)

Delayed RI sites accept (n=4) Nurses received training in RI

New parcipants in declined sites (n= 132)

New parcipants in accepng sites (n= 881). Reach not assessed.

Retenon & home visits assessed for Cohort 3 (n = 1,013)

that nurses in the DRI condition were not mentored to deliver the RI until they began serving cohort 3. The analysis focused on intervention-control differences in change over time in retention and home visits. Examining changes over time by site increases statistical power and controls for site effects. One site assigned to the RI condition closed during the second cohort year and did not participate in the mentoring; a second assigned to the RI declined to participate after originally agreeing. Two sites assigned to the DRI chose not to participate, and at a third DRI site, only two of the four nurse home visitors chose to be mentored in the RI. The analysis was conducted as intention to treat. We assessed retention and completed home visits for the RI group for cohort 3 in addition to cohort 2 in order to determine whether the RI effects were sustained after active mentorship of nurses was completed with cohort 2. Implementation data

New parcipants in declined site (n= 100)

New parcipants not reached (n = 71)

New parcipants reached (n = 164)

Retenon & home visits assessed for Cohort 3 (n = 335)

New parcipants receive NFP without RI (n = 801)

Retenon & home visits assessed for Cohort 3 (n = 801)

(described below) were gathered only for cohort 2 participants assigned to the RI condition and for cohort 3 participants assigned to the DRI condition. Table 1 shows the characteristics of sites in each of the three treatment conditions. As indicated in this table, there were some non-significant differences in organizational auspices by treatment assignment, as might be expected with small numbers of sampling units. Table 2 shows the background characteristics of the participants by treatment condition in each of the three cohorts. To be eligible for the NFP program, mothers must be first-time mothers and low-income (usually indicated by Medicaid eligibility). As noted in this table, there were missing data on some participant background characteristics. NFP participants, as indicated in cohort one, were young (M = 19.60 years), had low education (M =

Prev Sci Table 1

Characteristics of sites by condition

State A State B Organizational auspices Public health department Community-based organization Hospital Duration of program operation Site duration by start of study—mean interval (SD) and range between first visit at each site and October 1, 2003 % Urban

RI sites (N=9)

Delayed RI sites (N=6)

Control sites (N=11)

4 5

4 2

8 3

6 3 0

2 3 1

5 2 4

1644 days (733) 622 to 2439 (range) 62.1 (25.2)

1057 days (938) 200 to 2402 (range) 57.8 (16.5)

1102 days (671) 511 to 2099 (range) 70.7 (17.1)

11.23 years), were unmarried (84 %), and the majority were non-Hispanic White (63 %), Black (17 %), Hispanic (16 %), and Native American (9 %). While there were some differences among treatment conditions for some of these background variables, as noted below, controlling for these differences did not change the results. NFP participants consented at enrollment for their data to be used for evaluation and program improvement purposes. Participants were not informed about, or re-consented, for the

current trial, which consisted of an innocuous intervention. Nurses assigned to the RI and DRI consented to be mentored in the RI after their sites had been assigned to the intervention conditions, as the RI required additional time to learn, and nurses were asked to gather additional data as part of the intervention. Some nurses consented to participate in focus groups to help us understand aspects of intervention delivery. The study was approved by the Colorado Multiple Institutional Review Board.

Table 2 Characteristics of participants by treatment condition and cohort—means and standard deviations (for continuous variables) and percentages (for dichotomous variables) Background Characteristics

Baseline cohort Whole

Number in cohort Maternal age—years

n=1878 19.60 (3.84) n=1878 Education (years. completed) 11.23 (2.05) n=1645 Black 17.3 % n=1656 Non-Hispanic White 62.6 % n=1656 Hispanic 16.3 % n=1656 Native American 9.1 % n=1656 Married 15.6 % n=1646 Employed 44.1 % n=1275 Receiving Medicaid 76.2 % n=1464 Gestational age first visit (weeks) 18.49 (6.62) n=1734

Cohort 2

Cohort 3

RI

DRI

Control Whole

RI

DRI

Control Whole

RI

DRI

Control

n=929 19.73 (3.88) n=929 11.31 (2.18) n=787 20.0 % n=795 53.3 % n=795 19.9 % n=795 12.1 % n=795 19.1 % n=787 43.1 % n=601 81.7 % n=671 18.59 (6.81) n=842

n=317 20.04 (4.40) n=317 11.34 (1.86) n=286 7.3 % n=287 79.8 % n=287 13.6 % n=287 4.9 % n=287 15.7 % n=286 47.2 % n=229 71.5 % n=260 18.11 (5.89) n=297

n=632 19.18 (3.43) n=632 11.06 (1.97) n=572 18.5 % n=574 66.7 % n=574 12.7 % n=574 7.1 % n=574 10.6 % n=573 43.8 % n=445 71.5 % n=533 18.54 (6.70) n=595

n=1088 20.03 (4.01) n=1087 11.50 (2.32) n=1029 24.4 % n=1042 50.1 % n=1042 18.8 % n=1042 10.7 % n=1042 19.4 % n=1032 49.9 % n=819 76.5 % n=886 19.40 (6.92) n=956

n=414 19.87 (3.87) n=414 11.29 (2.01) n=401 11.6 % n=405 71.9 % n=405 19.0 % n=405 6.9 % n=405 17.0 % n=401 44.9 % n=343 80.3 % n=351 17.52 (7.02) n=376

n=854 19.74 (4.00) n=854 11.39 (2.25) n=833 16.8 % n=834 66.3 % n=834 15.2 % n=834 9.4 % n=834 16.1 % n=831 48.7 % n=678 76.6 % n=727 17.40 (6.98) n=755

n=1013 19.96 (3.84) n=1006 11.53 (1.89) n=944 21.5 % n=994 50.8 % n=994 9.9 % n=994 12.2 % n=994 18.1 % n=989 44.0 % n=813 61.2 % n=992 18.25 (6.76) n=923

n=335 19.87 (3.89) n=330 11.33 (1.90) n=308 10.6 % n=321 72.3 % n=321 8.1 % n=321 3.7 % n=321 13.5 % n=319 39.9 % n=263 68.5 % n=321 18.36 (6.90) n=303

n=801 19.22 (3.58) n=797 11.11 (1.89) n=757 19.5 % n=790 59.7 % n=790 13.0 % n=790 8.0 % n=790 13.7 % n=786 48.0 % n=610 67.8 % n=790 18.62 (7.28) n=753

n=2356 19.90 (3.98) n=2355 11.42 (2.24) n=2263 19.3 % n=2281 59.9 % n=2281 17.5 % n=2281 9.5 % n=2281 17.8 % n=2264 48.5 % n=1840 77.2 % n=1964 18.34 (7.02) n=2087

n=2149 19.67 (3.77) n=2133 11.34 (1.90) n=2009 19.1 % n=2105 57.4 % n=2105 10.8 % n=2105 9.3 % n=2105 15.8 % n=2094 44.8 % n=1666 64.8 % n=2103 18.41 (6.98) n=1979

Prev Sci

Retention Intervention (RI)

Measures

The RI has been described earlier (Ingoldsby et al. 2013). It centered on encouraging nurses to have explicit discussions about adjusting the frequency of visits and duration and content of the program to meet the needs of newly enrolled participants. In addition, nurses explained that mothers had the option of switching nurses. Nurses revisited participants’ visit preferences at child age 6 weeks and at 4 months, given that these time periods correspond to periods with relatively higher rates of dropout from the program. To deepen nurses’ skills in client collaboration, addressing clients’ ambivalence about remaining in the program, and adjusting program content to meet clients’ concerns and aspirations, nurses were provided augmented training in motivational interviewing (Miller et al. 2008). RI sites began mentoring and implementing the intervention with cohort 2 families in October 2005; DRI sites began mentoring and implementation with cohort 3 in October 2006. Nurses at the RI sites participated in 12 1-h structured teleconferences, led by NFP nurse mentors who had learned the intervention in the pilot phase (Ingoldsby et al. 2013) and in a small-scale implementation phase that preceded this trial. Nurses in the DRI were mentored by nurses who learned the RI during the second cohort year. Nurses completed readings, watched MI training videotapes between teleconference calls, and were asked to implement the RI with all newly enrolled participants as mentoring proceeded. During mentoring sessions, RI and DRI nurses were guided in learning and delivering the RI components, using written RI guidelines, which included scripts, strategies, and case examples. Nurses from each of the sites presented case examples and challenges they experienced in delivering the RI. After each call, teams were sent detailed summaries for their review. At two points during the 12-month intervention period, nurses were provided quantitative reports on their implementation of the RI, on client choices, and on rates of retention (described below).

Participant Reach Nurses in the RI and DRI conditions completed an implementation data form for cohort 2 (RI only) and cohort 3 (DRI only) indicating the date of each RI discussion and the components of the intervention that were delivered. Each newly enrolled case in the RI/DRI sites was assigned a value which indicated whether the nurse delivered the intervention with participants during the first intervention year, a variable labeled participant reach.

Retention Intervention Fidelity Nurses completed an implementation data form during their first year of RI implementation that captured whether the RI was implemented and clients’ choices for visitation schedule. Feedback to sites regarding their implementation of the RI, rates of retention, and completed home visits was shared during the mentoring sessions.

Control Condition Comparison sites delivered the NFP program as usual.

Participant Choices After each RI discussion with participants enrolled during the first year of implementation (cohort 2 for the RI group and cohort 3 for DRI), nurses asked participants if they preferred the program to be delivered (a) according to the typical schedule, (b) with reduced visit frequency, (c) with increased frequency, (d) with for a temporary leave from the program, (e) ending participation, or (f) by switching nurses. Responses were recorded on RI implementation data form. D e m o g ra p hi cs P a rti ci pa nt a ge , g es ta tio n al a ge , race/ethnicity, marital status, education and employment status, household income, and Medicaid status were assessed at enrollment as part of the standard data gathering for all NFP participants. There are some missing data on participant background characteristics. The exact numbers of cases on which background variables are available for analysis for each cohort and treatment condition are enumerated in Table 2. Site Level Characteristics Some of the site characteristics (e.g., length of NFP program operation prior to the Bpre-intervention cohort^) were calculated using participant data aggregated at the site level. We used the 2000 US Census to determine the percentage of the population in counties served by the program that lived in urban areas (United States Census Bureau n.d). Retention and completed visit outcomes were derived from data entered into the NFP web-based information system, into which NFP sites enter program implementation data. There were no missing data on completed home visits and participant retention. Retention Outcome Participants were considered to have been retained through the last home visit completed prior to the child’s 27th month of life. Note that this definition differs from that employed in earlier studies of participant retention in the NFP, in which participants’ drop status was derived from nurses’ coding participant drop status, with an original focus on amenable attrition (not caused by participant moves from the catchment area) and augmented by visit completion data, i.e., those without a visit within 180 days were considered to have

Prev Sci

been dropped, even if nurses did not code the case as a drop (Korfmacher et al. 1999; O’Brien et al. 2012; Ingoldsby et al. 2013). We invoked this change as we were concerned that the original definition was subject to treatment-related measurement bias, i.e., that nurses in the RI and DRI conditions may code reasons for attrition differently after completing mentoring in the intervention than their control-group counterparts. The operational definition employed in the current trial removes this possible source of error and is conservative in that moving from the catchment area is not amenable to nurses’ efforts to engage participants and is ignored in the current definition. Data on completed visits were derived from the NFP web-based information system. Number-of-Completed-Visits Outcome After every attempted or completed home visit, nurses filled out an encounter form which included information on whether the visit was completed. From data entered on this form, we calculated the number of home visits completed prior to the child’s 27th month of life (or expected 27th month of life calculated from the mothers’ gestational age at registration for those who stopped receiving visits prior to the child’s birth). The count of home visits was based upon every case registered in the NFP in each of the cohorts. While one might present the completed visit outcome as a percent of expected visits, expressing the outcome as number of completed visits allows more direct comparison between implementation metrics in NFP replication with the original trials of the NFP and with other home visiting programs. Number-of-Expected-Visits Covariate Individual participants registered in the NFP at different gestational ages and earlier registration strongly predicts more completed visits. Rather than controlling for gestational age at registration, however, we controlled for expected number of visits given women’s gestational age at registration. Expected visits account for how the standard number of expected visits varies by program phase (4 weekly visits immediately following registration, every other week until delivery, every week for the first 6 weeks following the birth, every other week until the baby is 21 months, and monthly visits until 24 months). The number-ofexpected-visits covariate was constructed taking into account fetal losses and maternal and child deaths, which the retention intervention is highly unlikely to affect.

were conducted (one or two). The first model consisted of the following fixed effects: treatment (two levels: RI+DRI vs control), year (two levels: baseline vs first year in which the intervention was conducted, specifically cohort 2 for RI, cohort 3 for DRI, cohort 2 for control), the interaction of treatment and year, plus expected number of visits and state as covariates. The second model was similar to the first but did not include the six sites in the DRI condition and added a three-level intervention year factor (baseline, cohort 2, cohort 3 for both RI and control groups) to examine the enduring impact of the RI. In the first statistical model, primary analyses contrasted the RI+DRI sites versus control sites, examining treatment differences in the change of retention and completed home visits between the baseline cohort and the cohorts during which the first year of intervention was conducted (cohort 2 for RI and control, and cohort 3 for the DRI). In the second statistical model, we had an additional cohort to test the durability of the RI. We focused the analysis on the contrast of RIcontrol changes in retention and completed visits from the baseline cohort to cohort 3. We examined the hypothesis that RI effects would be more pronounced among sites with initially lower levels of retention but dropped the high-low classification factor from the model after finding no significant results or pattern of changes consistent with this hypothesis. We examined models that included covariates to control for baseline differences in some additional background variables (maternal age, race, Medicaid enrollment, and marital status) but did not include them in the final models because adding the covariates had almost no effect on the estimates and tests and because adding more covariates increased the amount of missing baseline data. The final analyses were conducted as intention to treat, including all NFP participants enrolled in all sites that agreed originally to participate in the trial, irrespective of sites’ and nurses’ actual participation. The Cox proportional hazard model was used to estimate the hazard of attrition (the reciprocal of retention). A robust sandwich estimate aggregated at the level of sites was used to calculate contrasts of interest (Lin and Wei 1989). The continuous dependent variable (number of home visits completed through the child’s 27th month of life) was analyzed with the mixed effect model with sites as random effects. Findings are reported as adjusted hazard ratios or adjusted (least square) means controlling for the effects of other terms in the model, with standard errors, and two-tailed significance tests. Qualitative Data Analysis

Quantitative Data Analysis We examined two separate multilevel statistical models that reflected the two intervention conditions (RI and DRI) and the number of years during which the RI and DRI interventions

Given relatively low rates of RI implementation (reach) at some sites described below, focus groups were conducted by one of the investigators (EI) at eight of the NFP sites in the two intervention groups chosen to represent three levels of reach:

Prev Sci

low (50–64 %; n=3), medium (65–80 %; n=2), and high (>80 %; n=3). A total of 49 participants (41 nurses, 8 supervisors) participated in the focus groups (with a range of three to nine participants at each site). These 60-min sessions were audiotaped for later coding. All nurses and supervisors who participated in the focus groups completed informed consent. Focus group questions inquired about factors affecting implementation of the intervention and nurses’ motivation and participation in the training (e.g., BTell us your thoughts about factors or dynamics that might explain why nurses used the intervention at different levels of intensity^) and challenges or barriers that emerged during the intervention (e.g., Bwhat were the challenges for you and your team in spending team professional development time on the intervention?^). The audiotapes were later reviewed, coded for themes, and categorized using grounded theory (Strauss and Corbin 1990) into the following domains that represented factors affecting implementation: general, nurse, site/team, client, organizational, and intervention-specific factors. Themes/domains were then grouped by site-level implementation rates (low, medium, high) which allowed investigation of patterns of themes and responses across and within implementation groups.

Results Sample Characteristics Table 1 shows the site characteristics in each of the three treatment conditions with respect to organizational auspices, duration of program operations, and the degree to which program catchment areas were Burbanized.^ While none of the differences among treatment conditions were statistically significant, note that four of the five sites administered through hospitals were in the control group. Table 2 shows the participant characteristics for intervention and control groups for all cohorts, and the numbers of cases on which information was available for each background variable. Some of the background characteristics differed by treatment condition, but as noted above, inclusion of these baseline covariates did not alter the estimates and tests of intervention effect, so given missing baseline data, they were not included in the final model. Implementation of the Retention Intervention Reach Among all sites assigned to the RI and DRI conditions (irrespective of participation in the trial), nurses reached 63 % of newly enrolled participants in the first year of implementation. As noted above, four intervention sites did not deliver the RI. The mean participant reach among the sites that participated ranged from 53 to 92 % (mean participation=76 % among

those sites that participated); the one site in which two of the four nurses participated in the intervention reached 53 % of the new enrollees. Participants’ Choices for Program Delivery Among those reached, 86 % of participants chose the recommended visitation schedule, 10 % chose to decrease the frequency of NFP visits, 2 % chose to increase their visits, and the remainder (2 %) took a temporary leave or ended the program early. One participant requested a change in nurse. Intervention Impact on Participant Retention and Completed Home Visits Retention Figure 2 summarizes the hazard analysis (hazard of dropping from the program) following the first completed home visit to the last completed visit in the RI/DRI and control sites, comparing the baseline cohort with the cohorts enrolled in the sites during the first year of intervention implementation (cohort 2 for RI and cohort 3 for DRI), showing each state separately. Note that state A had substantially higher rates of retention overall (p64 % reach) reported that they decided that they would overcome the limitations of the intervention education and mentoring model by working as teams to learn it. These higher reach sites reported that their supervisors were crucial in leading them as a team to learn the RI intervention; their supervisors created a team climate that valued nurses’ lifelong professional development. Those sites with the lowest levels of reach reported that learning the intervention was so

34 32

Control RI+DRI

30

Adjusted # completed visits

Fig. 3 Mean number of completed home visits for control and RI+DRI sites during baseline versus first year of intervention implementation—cohort 2 (for control and RI)+3 (for DRI), shown separately by state

Qualitative Study of High- and Low-Reach Sites

State A (top 2 lines)

28 26 24 22 State B (bottom 2 lines)

20 18 16 14 Baseline

Cohort 2 (Control and RI) or Cohort 3 (DRI)

Prev Sci

Change

Fig. 4 Change in completed home visits among RI and DRI sites between baseline and first intervention year (cohort 2 for RI) and (cohort 3 for DRI) by degree of site reach

11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3 -4 -5

0 %

5 %

1 0 %

1 5 %

2 0 %

2 5 %

3 0 %

3 5 %

4 0 %

4 5 %

5 0 %

5 5 %

6 0 %

6 5 %

7 0 %

7 5 %

8 0 %

8 5 %

9 0 %

9 5 %

1 0 0 %

Reach

challenging, and the time required so significant that they felt overwhelmed. They reported that having to learn the intervention interfered with their time to conduct home visits.

Discussion Sites assigned to the control condition experienced declines in retention and completed home visits over time, while those that received the intervention did not. This pattern of results is consistent with that observed in the pilot study of this intervention (Ingoldsby et al. 2013) and is likely a reflection of participating sites being in need of help with increasing retention and home visits. The limited increases in retention and completed home visits in the intervention conditions is not surprising, given that 4 of the 15 sites assigned to intervention conditions did not participate, and among those that did, the intervention was delivered to 76 % of the possible participants. The durability of RI impact for cohort 3 found in the RI condition, that is after the RI mentoring ended, suggests that once nurses learn this approach to engaging families, it is carried over to new participants. It is revealing that high-reach sites attained the greatest increases in participant retention and completed home visits. It also is revealing that low-reach sites had almost no changes in completed home visits. Whatever gains in retention and home visits found for the RI were produced by the high-reach sites. A central goal of the RI was to reinforce with nurses the self-efficacy and strength-based clinical underpinnings of the program, particularly, that NFP participants are the experts in their own lives and that they know the frequency of visits and duration of participation that will work best for them. The results from this trial, as well as the preceding pilot study, align with other studies demonstrating the positive impact of

motivational strategies to engage families in mental health treatment (e.g., Grote et al. 2008). They also align with recently published findings from the Memphis trial of the NFP that examined latent class trajectories of home visits completed from pregnancy through child aged 2, in which there was a group of lower risk participants that managed to improve their own health and the health of their children with far fewer visits completed after birth than is included in the standard visitation schedule (Holland et al. 2014). The results of the current trial and the findings on completed home visits in Memphis emphasize that NFP program dosage is best adjusted to families’ needs and abilities to participate. In some ways, it is remarkable that intervention-control differences emerged given that four of the 15 intervention sites did not participate in the intervention mentoring and given variability in reaching participants among those sites that did participate. While nurses’ overall reach was relatively high (76 %) among sites that were mentored, three intervention sites implemented the intervention with only half of eligible participants. This observation and the results of the nurse focus groups have reminded us how crucial it is to formulate nurses’ professional development education in learning program innovations in ways that are feasible and align with their clinical practice. This, along with nurses’ feedback from the focus groups indicating that the RI was hard to learn, has led us now to introduce the RI as part of core education in the NFP model and to more explicitly align it with their practice. Given that the three low-reach sites cited lack of teamwork and organizational support for their learning, the NFP National Service Office has taken a more active role in supporting implementing agency administrators’ understanding of the importance of nurses’ professional development as core elements in their education in the NFP model. This lesson has broad implications for the introduction of all new program innovations.

Prev Sci

We also note that the number of sites included in this trial is small, a feature of the design that was exacerbated by our need to divide the intervention group into two groups, delaying the intervention for some sites, and our conducting the trial in two separate states with different levels of success in retaining families and completing home visits. The small number of sites increases the likelihood that random assignment will incur some lack of balance in sampling unit characteristics, as we found with the non-significantly larger proportion of control sites being implemented through hospitals. Analyses of implementing agencies’ auspices in the entire set of NFP sites nationally found that hospitals had higher rates of completed home visits than other agency types, so whatever unbalance exists in the current trial is likely to bias against the hypothesis of intervention impact. Future trials employing clusters of sites should ensure larger numbers of sampling units, especially when designs have these additional levels of complexity. While the pattern of results for the intervention and control conditions was essentially the same for sites in each of the two states, the dramatically different rates of retention and completed home visits by state deserves consideration. All sites in state B were administered through a single state agency, and there was considerable pressure brought to bear on nurses’ completion of specified numbers of visits each week, possibly undermining their motivation to spend time tracking hard-toreach families and taking time to learn new methods of delivering the program. The intervention’s focus on client-directed visit frequency also may have contributed to administrators’ and nurses’ concerns about nurses’ meeting state-directed productivity requirements. An intervention designed to increase the number of home visits by increasing participants’ explicit control of dosage may appear to be counterintuitive to administrators who are driven by requirements that nurses complete a set number of visits per week. We intend to examine this in greater depth in the future. Despite the limitations noted above, these findings align with the findings of a preceding pilot study of this intervention (Ingoldsby et al. 2013). And, as noted above, they have led to alterations in core NFP nurse education and program guidelines to support more explicit participant choice in visit scheduling and program content. This trial, a form or rigorously evaluated quality improvement, was conducted while changes were occurring in retention and completed home visits over time. Nationally, numbers of completed home visits declined over the early years of NFP community replication reaching a low point in 2006 (mean=23.44, SE=0.19) but increased to a mean of 24.02 (SE=0.15) in 2010, which is likely, at least in part, a reflection of the RI intervention findings being communicated to all NFP sites nationally. This diffusion appears to have happened rapidly in state A, which achieved a mean of 31.25 (SE=0.48) completed home visits by 2010. While some gain was observed among all sites in state B as well, the gain was less

dramatic; in 2010, sites in state B completed a mean of 19.99 (SE=0.45). State A sites were completing visits at about the same rate as those in the original randomized controlled trials (~31), while all sites in the USA were completing about 77 % of those in original trials. While more is required to bring NFP community replication sites closer to the mean number of completed home visits achieved in the original trials, the current trial and the preceding research on participant retention (O’Brien et al. 2012; Ingoldsby et al. 2013) appears to be helping improve this aspect of NFP implementation in community replication. Acknowledgments This study was funded by a grant from the W.T. Grant Foundation (6797) to David Olds and an institutional NRSA postdoctoral research training program, 5T32 MH015442, Developmental Psychopathology, Psychobiology, and Behavior (DPRG) for Erin Ingoldsby. We thank the nurses who participated in this study. Conflict of Interest None of the authors has a personal financial interest in the Nurse-Family Partnership. The Prevention Research Center for Family and Child Health, directed by DLO at the University of Colorado School of Medicine, has a contract with the Nurse-Family Partnership© to conduct research to improve the NFP program and its implementation. DLO, PB, EI, DL, and MDK were employed by this center at the time the study was conducted. Olds is the founder of the Nurse Family Partnership. Author Contributions Study concept and design: Olds, Baca, and Ingoldsby. Acquisition of data: McClatchy, Loch, Ramsey. Analysis and interpretation of data: Olds, McClatchey, Luckey, Knudtson. Drafting the manuscript: Olds, Luckey, Knudtson. Critical revision of manuscript for important intellectual content: Olds, Baca, Ingoldsby, Luckey, Knudtson, Loch, and Ramsey. Statistical analysis: McClatchey, Luckey, Knudtson. Obtained funding: Olds. Study supervision: Olds, Baca, Loch, and Ramsey. Additional Contributions We thank Wendy Gehring (UC Denver) for data management and Cheryl Loston-Williams (UC Denver) for help with manuscript preparation. All of these contributors received compensation from research grants for their work.

References Brown, C. H., & Liao, J. (1992). Principles for designing randomized preventive trials in mental health: an emerging developmental epidemiology paradigm. American Journal of Community Psychology, 27(5), 673–710. Grote, N. K., Swartz, H. A., & Zuckoff, A. (2008). Enhancing interpersonal psychotherapy for mothers and expectant mothers on low incomes: adaptations and additions. Journal of Contemporary Psychotherapy, 38, 23–33. Haskins, R., Paxson, C., Brooks-Gunn, J. (2009). Social science rising: a tale of evidence shaping public policy. The Future of Children. Last accessed 11/12/2013 from http://futureofchildren.org/ futureofchildren/publications/docs/19_02_PolicyBrief.pdf Holland, M. L., Xia, Y., Kitzman, H. J., Dozier, A. M., & Olds, D. L. (2014). Patterns of visit attendance in the nurse-family partnership program. American Journal of Public Health, 104, e58–e65.

Prev Sci Ingoldsby, E. M. (2010). Review of interventions to improve engagement and retention in parent and child mental health programs. Journal of Child and Family Studies, 19(5), 629–645. Ingoldsby, E. M., Baca, P., McClatchey, M. W., Luckey, D. W., Ramsey, M. O., Loch, J. M., & Olds, D. L. (2013). Quasi-experimental trial of intervention to increase participant retention and completed home visits in the nurse-family partnership. Prevention Science, 14(6), 525–534. Institute of Medicine of the National Academies. (2009). Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. Retrieved from http://iom.edu/ Reports/2009/Preventing-Mental-Emotional-and-BehavioralDisorders-Among-Young-People-Progress-and-Possibilities.aspx Kitzman, H., Olds, D. L., Cole, R., Hanks, C., Anson, E., Sidora-Arcoleo, K., & Holmberg, J. (2010). Enduring effects of prenatal and infancy home visiting by nurses on children: age-12 follow-up of a randomized trial. Archives of Pediatric and Adolescent Medicine, 164(5), 412–418. Korfmacher, J., O’Brien, R., Hiatt, S., & Olds, D. (1999). Differences in program implementation between nurses and paraprofessionals providing home visits during pregnancy and infancy: a randomized trial. American Journal of Public Health, 89(12), 1847–1851. Lin, D. Y., & Wei, L. J. (1989). The robust inference for the proportional hazards model. Journal of the American Statistical Association, 84, 1074–1078. Miller, W. R., Rollnick, S., & Butler, C. C. (2008). Motivational interviewing in health care: helping patients change behavior (applications of motivational interviewing). New York: Guildford Press.

Nurse-Family Partnership. Available from www.nursefamilypartnership. org last access 11/12/2013. O’Brien, R. A., Moritz, P., Luckey, D. W., McClatchey, M. W., Ingoldsby, E. M., & Olds, D. L. (2012). Mixed methods analysis of participant attrition in the nurse-family partnership. Prevention Science, 13(3), 219–228. Olds, D. L. (2002). Prenatal and infancy home visiting by nurses: from randomized trials to community replication. Prevention Science, 3(3), 153–172. Olds, D. L., Robinson, J., O’Brien, R., Luckey, D. W., Pettitt, L. M., Henderson, C. R., & Talmi, A. (2002). Home visiting by nurses and by paraprofessionals: a randomized controlled trial. Pediatrics, 110(3), 486–496. Olds, D. L., Robinson, J., Pettitt, L., Luckey, D. W., Holmberg, J., Ng, R. K., & Sheff, K. (2004). Effects of home visits by paraprofessionals and by nurses: age-four follow-up of a randomized trial. Pediatrics, 114(6), 1560–1568. Olds, D. L., Kitzman, H., Cole, R., Hanks, C., Sidora-Arcoleo, K., Anson, E., & Stevenson, A. (2010). Enduring effects of prenatal and infancy home visiting by nurses on maternal life-course and government spending: age-12 follow-up of a randomized trial. Archives of Pediatric and Adolescent Medicine, 164(5), 419–424. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: grounded theory procedures and techniques. Newbury Park: Sage Publications, Inc. United States Census Bureau, U.S. Department of Commerce. Available from: http://factfinder2.census.gov/faces/tableservices/jsf/pages/ productview.xhtml?pid=DEC_00_SF1_H002&prodType=table. Last accessed 23 Jan 2014.