New Parent Support Program

A Guide to Program Evaluation In United States Air Force Family Advocacy Program

Edited by: Glenda Kaufman Kantor, PhD Kathy Kendall-Tackett, PhD Written by: Glenda Kaufman Kantor, PhD Kathy Kendall-Tackett, PhD John Landsverk, PhD Steve Barnett, PhD Ellen DeVoe, PhD David Finkelhor, PhD Kristin Pepe, MA Mary Robertson, MUP

November 1999

Acknowledgements

This material is based upon work supported by the Cooperative State Research, Education and Extension Service, United States Department of Agriculture, and the United States Air Force under Special Project #95-EXCA-30414. The contributions of AFMOA/SGOF Family Advocacy, Colonel John Nelson, Lieutenant Colonel Carla Monroe-Posey, and the Program Managers and staff of AFMOA/SGOF are gratefully acknowledged. We are also extremely appreciative of the assistance provided by all the USAF and Family Advocacy staff at the bases, as well as the families who participated in the evaluation.

TABLE OF CONTENTS LITERATURE REVIEW: The Status of Primary Prevention of Family Violence Efforts................... 1 Theoretical Foundations of Child Maltreatment Prevention and Evaluation ............................. 1 Physical Child Abuse.................................................................................................... 1 Aiming Child Abuse Prevention at the Risk Factors........................................ 5 Types of Child Maltreatment ........................................................................... 5 Neglect ......................................................................................................................... 6 Sexual Abuse ............................................................................................................... 6 Prevention Theory ........................................................................................................ 7 Levels of Prevention in Child Maltreatment..................................................... 7 Home Visitation Programs for Child Abuse Prevention.............................................. 10 Overview of Home Visitation to Prevent Child Maltreatment ........................ 10 Overview of Home Visitation Research and Evaluation Efforts .................... 11 1999 Workshop on Revisiting Home Visiting ................................................ 13 Best Practice Recommendations .................................................................. 14 Parent Education and Support Programs .................................................................. 14 Goals of Parenting Education........................................................................ 14 Program Structure ......................................................................................... 15 Components of Parent Education and Support Programs............................ 15 Evaluation of Model Programs ...................................................................... 16 Evaluations of Paternal Effects .................................................................................. 19 Discussion .................................................................................................................. 20 Best Practice Recommendations .................................................................. 20 Theories and Risk Factors for Marital Violence ...................................................................... 25 Theories of Marital Violence....................................................................................... 25 Intra-Individual Theory................................................................................... 25 Socio-Cultural Theory.................................................................................... 25 Social-Learning Theory ................................................................................. 25 Prevention of Partner Violence................................................................................... 26 Marital Enrichment/Couples Programs ......................................................... 26 School-Based Partner Violence Prevention .................................................. 30 Common Concepts of Prevention Programs........................................................................... 34 Three Levels of Protective Factors for Children At-Risk ............................................ 35 Summary of Best Practice Recommendations for Violence Prevention Programs .... 36 Appendix: Resources for Developing a Literature Review...................................................... 38 References .............................................................................................................................. 39

How to Use This Manual ............................................................................. 48

a Cover Design by Jennifer Bunge

TABLE OF CONTENTS Section 1: Evaluability Assessment .......................................................... 52 How to Conduct Evaluability Assessments ............................................................................. 53 Step 1: Determine The Audience For Your Evaluation .............................................. 53 Step 2: Select an Evaluator........................................................................................ 56 Step 3: Get an Overview of Your Program................................................................. 58 Step 4: Determine Types And Amounts Of Data Presently Available........................ 61 Step 5: Construct Plan For Gathering Still-Needed Data........................................... 64 Step 6: Making Sense Of Your EA ............................................................................. 69 Making Decisions: Process or Outcome Evaluation? ................................... 71 Summary .................................................................................................................... 72 Evaluability Assessment Worksheet ....................................................................................... 73 References .............................................................................................................................. 75

Section 2: Logic Model Development........................................................ 78 Determine Who Will Be Involved In the Development Process .............................................. 82 Develop A Common Language ............................................................................................... 83 Identify Program Goals, Objectives, and Outcomes ............................................................... 83 List Services and Activities...................................................................................................... 85 Specify the Prevention Mechanisms ....................................................................................... 89 Linkages Between Outcomes and Program Activities ............................................... 89 Specify the Target Population or Community.......................................................................... 90 Linkages Between Program Activities and Target Population/Community................ 90 The Next Step.......................................................................................................................... 91 Some Final Thoughts .............................................................................................................. 92 Worksheet for Program Goals and Objectives........................................................................ 93 Worksheets for Program Activities .......................................................................................... 94 Worksheet to Specify Details of Program Services................................................................. 95 Worksheet for Linkages Between Program Activities and Program Outcomes ...................... 96 Worksheet for Target Population............................................................................................. 97 Worksheet for Target Communities......................................................................................... 98 References .............................................................................................................................. 99

Technical Appendix: Logic Model Development: USAF Family Advocacy Prevention Programs ...................................................101 Introduction............................................................................................................................ 102 Logic Model for First Time Parents Program (FTP) .............................................................. 102 Figure 1: First Time Parent’sProgram (New Parent Support Program): Elements for a Program Logic Model ....................................................................................... 103 Analysis of the FTP Logic Model.............................................................................. 104 Logic Model for “Home-based Opportunities Make Everyone Successful” .......................... 105 Figure 2: Home-based Opportunities Make Everyone Successful: HOMES: Elements for a Program or Logic Model................................................................... 106 Analysis of the HOMES Logic Model ....................................................................... 107 Process for Development of the New Parent Support Program Logic Model ....................... 107

TABLE OF CONTENTS NPSP Program Rationale...................................................................................................... 109 Figure 3: New Parent Support Program – Logic Model ........................................... 110 Table 1. Research Evidence for the NPSP Logic Model....................................................... 112 References ............................................................................................................................ 114

Section 3: Process Evaluation ................................................................. 117 Target Population .................................................................................................................. 118 Recruitment .............................................................................................................. 118 Retention .................................................................................................................. 122 Tracking Retention ................................................................................................... 125 Program Activities.................................................................................................................. 126 Monitoring Delivery of Services................................................................................ 126 Monitoring Dose Effects ........................................................................................... 130 Forms for Recording Service Activity ....................................................................... 131 Worksheet 1: Characteristics of the Target Population and the Current Participating Population ......................................................................................................................... 133 Worksheet 2: Monitoring Attempts to Contact....................................................................... 134 Worksheet 3: Monitoring Program Retention ........................................................................ 135 Worksheet 4: Monitoring Program Services.......................................................................... 136 Form 1: Recording Service Activities for Educational Programs .......................................... 137 Form 2: Client Activity………………………………………………………………………………138 Form 3: New York’s Home Visiting Program Service Referrals……………………………….139 References ............................................................................................................................ 141

Section 4: Cost Analysis .......................................................................... 143 What Is Cost Analysis?.......................................................................................................... 144 Why Conduct a Cost Analysis? ............................................................................................. 145 A Step-By-Step Approach To Cost Analysis ......................................................................... 146 Defining the Cost Analysis Problem ......................................................................... 147 Identify The Need For Cost Information ...................................................... 148 Identify Internal and External Audiences..................................................... 148 Specify Goals For The Analysis .................................................................. 150 Conducting The Cost Analysis ................................................................................. 150 Identify All Ingredients and the Amounts Required ..................................... 150 Estimate The Costs Per Unit For Each Ingredient And Calculate Total Cost................................................................................. 156 Account For The Effects Of Time ................................................................ 159 Analyze The Cost Data ............................................................................... 160

TABLE OF CONTENTS Describe Distributional Consequences ....................................................... 163 Conduct Sensitivity Analysis ....................................................................... 163 Potential Problems (and Solutions) in Cost Analysis ............................................................ 166 Combining Cost and Outcome Data...................................................................................... 168 Cost-effectiveness Analysis ..................................................................................... 168 Using Cost Analysis To Make and Influence Decisions ........................................................ 170 Beyond This Section ................................................................................................ 173 Worksheet 1: Potential External Audiences for Cost Information ......................................... 174 Worksheet 2: Resource Cost Model...................................................................................... 175 Personnel Costs ....................................................................................................... 176 Nonpersonnel Costs ................................................................................................. 177 Worksheet 3: Cost Analysis .................................................................................................. 178 References ............................................................................................................................ 179

Technical Appendix: Cost Analysis in USAF FAP..............................................180 Introduction............................................................................................................................ 181 Methods................................................................................................................................. 181 Feasibility Pilot Study ............................................................................................................ 182 Overview .............................. .................................................................................... 182 Instruments and Procedures . ................................................................................... 182 Results....................................................................................................................... 183 Conclusion................................................................................................................. 184 Time Diary Study ................................................................................................................... 185 Sample and Procedures............................................................................................ 185 Instrument and Coding .............................................................................................. 186 Statistical Analyses ................................................................................................... 186 Cost Analyses............................................................................................................ 187 Results................................................................................................................................... 189 Time Allocations ........................................................................................................ 189 Cost Estimates .......................................................................................................... 191 Potential Home Visitation and Caseload Scenarios.................................................. 191 Discussion ............................................................................................................................. 193 Recommendations for Follow-Up Study................................................................................ 194 References ............................................................................................................................ 196 Table 1: Day of Interview....................................................................................................... 197 Table 2: Time Allocation – All Staff ....................................................................................... 198 Table 3: Time Allocations – Nurses....................................................................................... 199 Table 4: Time Allocations – FTP Nurses Only ...................................................................... 200 Table 5: Time Allocations – HOMES Nurses Only................................................................ 201 Table 6: Time Allocations – Social Workers.......................................................................... 202 Table 7: Comparison of Time Allocations—CONUS V. OCONUS........................................ 203 Table 8: Distributions of Home Visits by FTP Nurses ........................................................... 204 USAF Cost Analysis Survey.................................................................................................. 205

Section 5: Assessing Program Impact .................................................... 206 Critical Elements of Impact Evaluation.................................................................................. 207 Evaluation Design..................................................................................................... 207

TABLE OF CONTENTS What Are You Trying to Find Out? .............................................................. 207 Will There Be A Comparison Group?.......................................................... 207 What Is The Timing Of Assessments?........................................................ 208 Selecting A Design ................................................................................................... 209 True Experimental Designs ......................................................................... 210 Pros and Cons of True Experimental Designs ............................... 212 Quasi-Experimental Designs....................................................................... 213 Pros and Cons of Quasi-Experimental Designs............................. 215 Non-Experimental Designs.......................................................................... 216 Types of Non-Experimental Designs........................................................... 216 Pros and Cons of Non-Experimental Designs................................ 217 Reviewing Design Options: Example for a Couples Communication Program............................................................ 217 Evaluation Scope...................................................................................................... 218 Who Will Participate In The Study? Identifying Your Sample. .................... 218 Are Sufficient Resources Available? ........................................................... 220 Evaluation Methodology ........................................................................................... 221 Types of Qualitative Approaches ............................................................................. 222 Open-Ended Interviews............................................................................... 222 Focus Groups .............................................................................................. 223 Direct Observation....................................................................................... 225 Pros and Cons of Qualitative Approaches ............................................................... 226 Bottom Line for Qualitative Methods ........................................................................ 226 Worksheet 1: Selecting an Evaluation Design ...................................................................... 227 Worksheet 2: Projecting the Scope of the Evaluation ........................................................... 228 References ............................................................................................................................ 229

Section 6: Designing Data Collection Instruments ................................ 231 Using Existing Data ............................................................................................................... 231 Type of Variables...................................................................................................... 232 Coding Forms ........................................................................................................... 233 The “Coder” of Data ................................................................................................. 234 Client Confidentiality................................................................................................. 234 Item vs. Scale Scores............................................................................................... 235 Collecting New Data .............................................................................................................. 235 Designing Survey Questions .................................................................................... 236 Question Format....................................................................................................... 237 Open-ended Questions ............................................................................... 237 Closed-Ended Questions ............................................................................ 238 Putting Questions Together................................................................................................... 241 Question Order and Transitions .................................................................. 242 Pretesting .............................................................................................................................. 245

TABLE OF CONTENTS Survey Format .......................................................................................................... 245 Self-Administered Questionnaires............................................................... 245 How To Increase Compliance .................................................................................. 246 Interviews ................................................................................................................. 248 References ............................................................................................................................ 252

Section 7: Outcome Measurement .......................................................... 254 The Importance of Measurement .......................................................................................... 256 Reliability and Validity .............................................................................................. 256 Enhancing Outcome Measurement.......................................................................... 257 What Should Be Measured ................................................................................................... 258 Decision Making ........................................................................................................ 259 Key Measures........................................................................................................................ 260 Recommendations for the Measurement Model for the USAF Family Advocacy Program New Parent Support Program........................................................................... 260 Description of NPSP Core Measures ....................................................................... 261 A Catalogue of Recommended Measures for Family Programs........................................... 270 References ............................................................................................................................ 288

Section 7: Appendix: Compendium of Measures ..............................................293 Outcome Measures of Marital Quality & Adjustment: Instruments for Couples .................... 294 Locke-Wallace Marital Adjustment Test (LWMAT) .................................................. 295 Sample Items from the Dyadic Adjustment Scale (DAS) ......................................... 297 Primary Communication Inventory (PCI) ................................................................ 298 Family Status Measures ........................................................................................................ 299 Sample Items from Family Environment Scale: Expressiveness Subscale ............. 300 Couple Violence Measures ................................................................................................... 301 Husband-Wife Conflict Tactics Scale ....................................................................... 302 Abusive Behavior Inventory...................................................................................... 303 Sample Items from Psychological Maltreatment of Women Scale .......................... 305 Parent-Child Interaction Measures........................................................................................ 306 The Child Development Review: Parent Questionnaire............................................ 307 Sample Items from the Adult-Adolescent Parenting Inventory (AAPI-2) ................... 308 Measures of Child Behavior, Social-Emotional Functioning and Mental Health ................... 309 Sample Items from the Infant-Toddler Social and Emotional Assessment (ITSEA) Emotional Negativity Subscale.............................................................. 310 Sample Items from the Child Behavior Checklist for Ages 4-16 ................................ 311 Sample Items from the Conners’ Parent Rating Scale – Revised(S)......................... 312 Sample Items from the Conners’ Teacher Rating Scale – Revised(S) ...................... 313 Sample Items from the Children’s Depression Inventory........................................... 314 Adult Functioning Measures.................................................................................................. 315 Sample Items from the Self-Evaluation Questionnaire: State-Trait Anxiety Inventory............................................................................................................. 316 Sample Items from the State-Trait Anger Expression Inventory (STAXI) ................... 317 Military Stressors ........................................................................................................ 318 Rosenberg Self-Esteem Scale (RSE) ........................................................................ 319 Substance Abuse Measures ................................................................................................. 320 Michigan Alcoholism Screening Test Short Form (S-MAST) .................................... 321

Literature Review The Status Of Primary Prevention Of Family Violence Efforts The following review is divided into three major subsections. The first covers the theoretical foundations of child maltreatment prevention, and how these programs are evaluated. Two types of programs are described: home visitation and parent education and support. The second section provides an overview of theory and risk factors in marital violence. Research on two types of prevention programs are described: marital enrichment and school or community based violence prevention programs. The final subsection discusses common concepts of prevention programs, and includes a description of both risk and protective (resilience) factors. The implications of this research for prevention programs are summarized. At the end of each subsection, we have listed recommendations for “best practice.” For your convenience, we have also summarized these at the end of the chapter. Theoretical Foundations of Child Maltreatment Prevention and Evaluation1 The theoretical foundations behind child maltreatment prevention are both complex and simple. They are complex because there is no single theory used to explain the problem, much of the theory is undeveloped and partial, and a large number of variables and concepts have been proposed as relevant. Moreover, the empirical support for the theories has been ambiguous, adding doubt to the complexity. The theoretical foundations are simple, in that there is a general consensus that certain basic factors play an important role, and these are the factors around which prevention has been organized. Most of the theory regarding the prevention of family violence focuses on the prevention of child physical abuse, so that is where this chapter will start. It will expand its attention to other forms of maltreatment later on. Physical Child Abuse Over two dozen theories have been identified in the literature to explain physical child abuse (Tzeng, Jackson, & Karlson, 1991). Certain of these theories have been particularly influential in guiding thinking about prevention, and will be briefly reviewed here. •

Ecological Model

The ecological model of child abuse has stressed that factors from different levels-the individual, the family, the neighborhood and the community all contribute to 1

By David Finkelhor

1

the problem (Bronfenbrenner, 1977; Garbarino, 1982). Belsky has posited that child abuse results from a combination of three major factors: • an individual's developmental history (and thus predisposition to abuse), • stress producing forces within and beyond the immediate family, and • the values and child-rearing practices that characterize the society or subculture in which the individual and family are imbedded (Belsky, 1980). •

Social Psychological Theory

In his theory, Gelles (1973) stresses six contributors to child abuse: • • • • • • •

the socialization experiences of the perpetrator, psychopathic states of the perpetrator, social position of the parent, situational stress, class and community values and norms, and immediate precipitating situations.

Cognitive/Behavioral Theory

Cognitive behavior theory, typified by Azar, Fantuzzo, & Twentyman (1984), sees child abuse as growing out of five factors: • • • • • •

parent child interaction disturbances, impulse control problems, cognitive disturbances, family stress, and absence of social support.

Attachment theory

Attachment theories hold that children with unresponsive, insensitive and rejecting caretakers develop problematic forms of attachment with serious developmental consequences. Although not a theory about the etiology of abusive behavior, it is a theory that has emphasized the value of early intervention and provided some focus for intervention efforts. Prevention approaches draw on these theories, but essentially focus on two rather simple propositions that are shared by many of the theories: •

Child abuse is much more likely when a parent is confronted by stressful life circumstances and events and lacks the ability and resources to cope with them. 2

•

Child abuse is more likely when a parent has negative or unrealistic cognitions about and inadequate knowledge of, or a distressed emotional orientation toward parenting.

Both of these propositions imply other more detailed etiologic mechanisms that have been subject to considerable research. Following are some of the key variables associated with the theories. •

Corporal Punishment

Corporal Punishment is by far the most proximate and strongly related risk factor. The best estimate is that about 2/3 of all cases of physical abuse are instances of corporal punishment that have escalated out of control. Corporal Punishment is also linked to physical abuse by virtue of parents’ own experiences with physical punishment in the families in which they grew up. Empirical data shows that the more physical punishment a parent experienced as a teenager, the greater the probability of going beyond ordinary corporal punishment with their own children and engaging in severe assaults on the child (Straus & Kaufman Kantor, 1994). •

Stress

A variety of studies have found more life stress among physical abusers (Chan, 1994; Conger, Burgess, & Barrett, 1979). This is an association predicted by most models. A variety of specific mechanisms, though, can be posited to understand the connection. They include the possibility that stress may affect the ability to process information effectively, that stress may affect the selection of parenting behaviors in given circumstances, that stress adds to frustration which leads to violence, and that stress triggers or aggravates underlying mental health or personality problems. •

Coping Skills

Connected to the stress mechanism is the assumption that abusers have deficits in coping with stresses, which exacerbate their impact. This assumption has been confirmed by Hansen, Pallotta, Tishelman, Conaway, & MacMillan (1989). •

Substance Use

One of the main inadequate coping mechanisms associated with child maltreatment is substance use. The disproportionate existence of actual alcohol and drug disorders among physical abusers is widely assumed and has been confirmed in a very good epidemiological study controlling for a variety of other considerations (Kelleher, Chaffin, Hollenberg, & Fischer, 1994). Many mechanisms have been posited connecting alcohol and substance use to violence, including the possibility that substances weaken inhibitions and that they limit the processing of information.

3

•

Isolation

Isolation is generally viewed in models of child abusing as a crucial characteristic of abusing families, something confirmed in most studies (Chan, 1994; Milner & Wimberly, 1979). Social support may reduce the likelihood of child abuse in a variety of ways: decreasing stress, increasing coping, modeling more positive parenting attitudes, and through its informal surveillance which acts as a deterrent to aggressive behavior. There is also some research suggesting, however, that child-abusing families may isolate themselves, perhaps to hide deviant parenting behaviors. •

Negative Views Of Children

Some of the most important theory about child abusers from a prevention point of view has posited that they have negative, hostile, mistaken, unrealistic attitudes and attributions about children in general or their children in particular. Thus they are more likely to see normal acting children as behaving badly, more likely to attribute blame to children in parent-child conflict situations, more likely to see children's bad behavior as motivated and fixed characteristics of the child (internal and stable attribution) (Milner & Dopke, 1997). They are also more likely to believe in punishment and have authoritarian parenting styles (Susman, Trickett, Iannotti, Hollenbeck, & Zahn-Waxler, 1985). Beliefs that their children are bad or provocative combined with a positive view toward the use of physical force may be some of the most proximal mechanisms behind child abuse. •

Emotional Problems

Child abusers are believed to have more problems with depression, anxiety and hostility than other parents, a view confirmed in a variety of studies (Milner & Dopke, 1997). These emotional states result in a lower threshold for perceived child misbehavior and punitive reactions to that behavior. •

Empathy Deficits

Along with increased negativity, child abusers have been found to have deficits in empathy for children or their children in particular (Frodi & Lamb, 1980; Wiehi, 1987). Empathy presumably acts to inhibit aggressive behavior toward children •

Self Esteem

Low self esteem is widely cited as an important characteristic of abusers, confirmed in a variety of studies (Altemeier, O'Connor, Vietze, Sandler, & Sherrod, 1982; Culp, Culp, Soulis, & Letts, 1989; Shorkey & Armendariz, 1985). In various models it is seen as contributing both to negative perceptions of children and to reduced coping ability (Milner & Dopke, 1997) •

Aiming Child Abuse Prevention Strategies at the Risk Factors

4

The characteristics noted above, and their underlying presumed mechanisms have been the ones that have primarily been targeted by prevention strategies aimed at physical abuse. The strategies have tried, on the one hand, to reduce stress and social isolation and increase coping skills and resources and on the other, to teach more positive, realistic, empathetic and non-punitive attitudes and approaches to parenting. While other elements of theory regarding child abuse focus on neurology, personality and psychopathology in the abuser, and still others on social, political, community and historical factors, these have generally been seen as less amenable to direct manipulation and have been less emphasized in prevention program development. There is also an acknowledgment of the role children themselves may play in the instigation of abuse because of difficult temperament or provocative behaviors but relatively little prevention practice has targeted these factors except to use them as possible risk indicators. There has been some recognition that children, particularly older children, may be able to thwart some abuse, by recognizing dangerous situations, by being armed with the knowledge of where to get help and who to tell. This idea has informed some secondary prevention programs for physical abuse and neglect (and spousal assault) as well as sexual abuse prevention programs. Many of the specific strategies adopted by child abuse prevention have also been dictated by practicalities. So, for example, many of the prevention strategies target parents and families in the immediate post-natal period. Although a substantial portion of abuse and neglect (especially the most serious forms) occurs early in a child's life, some early intervention strategy grows out of the ease of identifying families at risk (since almost all appear at hospitals for childbirth) and the willingness of new parents to accept intervention. The targeting of mothers in many prevention programs results from the fact that they tend to be the parents most willing and able to participate. Some child abuse prevention programs try to identify at risk families through other social agency contact (schools, day care) and offer prevention programs. •

Types of Child Maltreatment Prevention Programs

Prevention programs have been organized around a variety of formats. Currently a very popular approach is to provide services directly in the home. This form of intervention has become popular because studies show this approach to be particularly effective. Other approaches have utilized classes and educational formats, group meetings of parents in community institutions and agencies, supervised day care where parents and children interact together, and individual counseling formats.

Neglect

5

Very little theory and research has focused specifically on neglect (Tzeng, Jackson & Karlson, 1991). Polansky and other early theorists emphasized personality deficiencies the apathy-futility syndrome, the impulse ridden character and infantile emotional functioning (1981). More recent theoretical perspectives on neglect rely on multi-factorial and multi-level theories that parallel the theory developed for analyzing physical abuse. These theories tend to emphasize: •

parental competence (Belsky, Rovine, & Taylor, 1984), e.g. sensitivity to a child's needs, and

•

stress and stressful environments, particularly economic disadvantage, broken families, social isolation, unavailable extended family (Wolock & Horowitz, 1984).

Prevention approaches try to provide education and skill development to improve competence and increase parent-child bonding, and specific resources, services and coping strategies to reduce stress and social isolation. Sexual Abuse Sexual abuse tends to be analyzed from a very different frame of reference than physical abuse and neglect. The theory behind sexual abuse has similarities to other child maltreatment theory, but a variety of factors result in very divergent prevention conclusions being drawn from this theory. Finkelhor (1984) has proposed a four preconditions model of sexual abuse that posits the need for the following: • • • •

a motivation to sexually abuse the overcoming of internal inhibitions against abusing The overcoming of external inhibitions (e.g. environmental barriers) against abusing, the overcoming of a child's resistance.

The disturbed cognitive and emotional orientation toward the child that is implicit in the theories on physical abuse and neglect are present in Finkelhor's Precondition 1. Thus instead of the unrealistic expectations or negative attitude toward the child, the cognitions among sexual abusers concern the sexual attractiveness of the child, and the possible appropriateness of acting sexually with the child. The sexual abuse theory actually further parallels the physical abuse theory in its emphasis on the parents' view that the child is meant to serve the parents' needs. The second element of physical abuse and neglect theory, the stressful and unprotective environment are also present in Finkelhor's Preconditions 2 and 3. Some of the same family and economic stresses that may provoke violence toward a child are included in sexual abuse theory as factors that may lower inhibitions and result in sexual acts toward the child.

6

The fact that prevention approaches targeted at physical abuse and neglect do not tend to explicitly encompass sexual abuse stem from a variety of practical factors, rather than theoretical ones. • the perpetrators of sexual abuse tend to be men, who are less accessible to the kind of home visiting and counseling interventions used with physical abuse and neglect. • sexual abuse tends to have its onset after age 6 and especially in preadolescent years (Finkelhor, 1986). Thus the signs and stresses that may be relevant to prevention may not be present or easily affected by the early intervention model adopted in much physical abuse and neglect prevention. • the cognitions and emotional orientations that may underlie sexual abuse are more difficult for intervenors to access. While a parent may readily talk about the burden a child poses for them or the problem they have in getting a child to behave, parents are very reluctant to talk about sexual feelings they may have toward the child. Thus, the primary prevention of sexual abuse has not targeted parents or potential perpetrators or even parent-child interactions, as have the prevention approaches to abuse and neglect. Rather the primary prevention of sexual abuse has focused on educational programs for children on how to recognize sexual abuse, how to resist it and how to report it (Finkelhor & Daro, 1998). The fact that fewer sexual abuse victims are very young, preverbal and extremely helpless infants (as in the case of physical abuse and neglect) has made the empowerment of victims a more plausible strategy for this type of maltreatment. Prevention Theory Prevention approaches are usually classified in two ways: according to when they are applied in the course of the development of the problem, and according to what level of society they take as their target. •

Levels of Prevention in Child Maltreatment

The typical classification of prevention approaches in the course of problem development is designated as primary, secondary and tertiary prevention. Primary prevention efforts are targeted towards a total population to prevent the onset of a problem behavior such as abuse. Secondary prevention efforts are targeted towards a particular population that is believed to be at risk for a problem such as home visitaion parent support programs for new adolescent mothers. Tertiary prevention is directed at those individuals who have already developed a problem, to prevent the recurrence of the problem, in this case working with abuser and abusive families to stop it. Some view tertiary prevention as a way of trying to prevent the problem from having cascading negative outcomes, such as treating child victims of abuse to make sure that they do not go on to develop school problems. An interesting feature of prevention approaches to physical abuse and neglect is that there is a great deal of similarity between the 7

various levels of prevention (Finkelhor & Daro, 1998). Thus home visitation programs, respite care programs, and parent education programs often have similar content for both families where abuse has occurred and where it has not. Such families are sometimes mixed in some programs. This mixing of multiple levels of prevention activities is not true in the case of sexual abuse, however. The primary prevention approaches are generally targeted at children in schools. Once abuse has occurred prevention efforts focus on the abusers and the family. Secondary and tertiary prevention strategies for spousal assault also tend to be different from primary prevention, with a much greater emphasis on treatment of the abuser. Prevention approaches are also differentiated according to the social level of intervention. Kaufman and Zigler (1993) have developed a clear taxonomy of child abuse and neglect interventions according to four levels, and prevention programs may be viewed as being targeted at each of these levels: •

the ontogenetic (or individual) level

•

the microsystem. (or intrafamily) level

•

the exosystem (or community) level

•

the macrosystern (or socio-political) level

8

Prevention Programs For Four Levels Of Societal Organization Ontogenetic Level Microsystem Level Exosystem Level Macrosystem Level Psychotherapeutic Interventions for abusive Parents

Marital counseling (Lutzker, Wesch & Rice, 1984)

Develop community social and health services

Campaigns to increase public awareness

Treatment programs for Abused children

Home safety training (Lutzker, Wesch & Rice, 1984) Health visitors (Olds & Kitzman, 1993)

Crisis hotlines

Formation of NCCAN grants for research

Training Professionals to Identify Abuse

Establishment of a National Commission on Child Abuse and Neglect

Stress management skills Training (Egan, 1983)

Enhancement of parentinfant contact and interactions

Location of foster and adoptive homes

Require states to adopt procedures for the prevention, treatment and identification of maltreatment (Child Abuse Prevention and Treatment Act of 1974)

Job search assistance (Lutzker, Wesch & Rice,1984)

Parent aid programs

Facilitates informal community supports

Legislative efforts to combat poverty

Education for parenthood Programs

Establish family planning centers

Laws against corporal punishment in society & in families

Parenting skills training

Establish coordinating agency for child maltreatment services

Research incidence maltreatment and effectiveness of prevention & treatments

Alcohol and drug Rehabilitation (Lutzker, Wesch, Rice, 1984)

Parents Anonymous groups Respite child care facilities

The reviews that follow consider two of the most widely used prevention approaches to child maltreatment. As will be discussed the two approaches—home visitation programs, and parenting education programs---draw on the theoretical foundations and risk factors elaborated above.

9

Home Visitation Programs For Child Abuse Prevention2 This section will critically review the literature related to research and evaluation of home visitation models for early prevention programs. The review will specifically focus on current and past work in both the military and civilian communities which is related to the home visitation models employed in Air Force Family Advocacy Prevention Programs - First Time Parents (FTP) and Home-Based Opportunities Make Everyone Successful (HOMES). •

Overview of Home Visitation to Prevent Child Maltreatment

Currently, home visitation programs linked to prevention initiatives within Family Advocacy Programs have been implemented in the Air Force, Army, Navy, and Marine Corps, all within the past five to six years. Research and evaluation efforts have been linked to several of these home visitation programs, including the original pilot test evaluation in the Air Force conducted by Mollerstrom, as well as the current evaluation strategies in place with the FTP and HOMES programs. These initiatives within multiple branches of the military are mirrored by recent large-scale initiatives to implement home visitation programs in the civilian world, including the state-wide Healthy Start Program in Hawaii and the Healthy Families America initiative under the auspices of the National Center to Prevent Child Abuse. It should be noted that almost all of these initiatives have targeted the prevention of child maltreatment and have not explicitly included spouse abuse which would be included under the term •family maltreatment”. •

Military Models of Home Visitation Prevention Programs

This rapid promulgation of home visitation in both military and civilian settings as a method to prevent child maltreatment was initially triggered by the 1990 call for universal home visitation as a preferred prevention strategy by the U.S. Advisory Board on Child Abuse and Neglect. The Board’s recommendation was based largely on the promising early results of the early 1980's study of home visitation conducted by David Olds in Elmira, New York. The Olds program of research has used a professional model with RN home visitors, as has the Air Force program FTP. The major effort to implement and test a model which uses para-professional home visitors has generally come from the Hawaii experiment with Healthy Start and the Healthy Families America program. The constituents of the home visitation team have differed across the various branches of the military. Military Models of home visitation teams have included the following:

2

By John Landsverk & Glenda Kaufman Kantor

10

Service provider Models in Military Home Visitation Programs AF Marines Army Navy •

FTP HOMES NPSP NPSP FTP NPSP

Professional Model with RN Professional Model with RN/MSW Professional Model with RN/MSW Professional Model with RN/MSW Para-Professional Model Para-Professional Model as in Hawaii Healthy Start Model

Overview of Home Visitation Research and Evaluation Efforts

Olds and Kitzman’s (1993) comprehensive review of home visitation evaluation research focused on studies which used randomized designs and which evaluated programs to prevent preterm delivery and low birth weight infants, home visiting programs for parents of preterm and low birth weight babies, and home visiting programs for low-income families or families at risk for child maltreatment. It should be noted that no studies of military populations were included in this review. A summary of the studies of programs targeting child maltreatment suggested that: none of the six trials that sought to use home visiting to prevent child abuse and neglect demonstrated overall decreases in maltreatment as evidenced by state CPS records. Three, however, did demonstrate differences for at least some study participants which are suggestive of benefits, either in decreasing abuse and neglect, improving parenting, or decreasing use of medical services associated with abuse and neglect. Following is an overview of current research studies in the civilian and military which are examining the processes and outcomes of home visitation to prevent maltreatment. •

The Olds’ Elmira Studies & Replications

Olds has implemented three successive randomized trials of home visitation over the past 20 years. The original study in Elmira, New York began in 1977 and enrolled 400 women before the 30th week of pregnancy in a comprehensive prenatal and postpartum nurse home visitation program in a small, semi-rural community in Upstate New York. Eighty-five percent of the women were either low-income, unmarried, or teenaged, and none had a previous live birth. The Elmira study cohort has now been followed for 15 years with the home visited group showing significantly fewer reports of child abuse and neglect, fewer subsequent births and greater spacing between birth of the first and second child, significantly lowered use of welfare (AFDC), fewer behavioral impairments due to use of alcohol and drugs, and fewer arrests as indicated by self-reports (Kitzman, Olds, Henderson, Hanks, Cole, Tatelbaum, McConnochie, Sidora, Luckey, Shaver, Engelhardt, James & Barnard, 1997). Olds and colleagues (1998) believe that rigorous studies support the ability of nurse home visitation programs to reduce the risk of early antisocial behavior and prevent numerous 11

problems linked to youth crime and delinquency. In addition, the Elmira study has now included a cost-benefit analysis which has resulted in a published report of recovered costs of the program for the low-income families. Olds and his colleagues have also conducted a replication of the Elmira randomized trial in Memphis with a sample of 1,139 low-income African-American women. The first published report of a two year follow-up in this study has found that fewer nurse visited women had pregnancy induced hypertension, nurse visited families had fewer child health care encounters in which injuries or accidental ingestions were detected, and nurse visited women had significantly fewer second pregnancies (Kitzman, Olds, Henderson, Hanks, Cole, Tatelbaum, McConnochie, Sidora, Luckey, Shaver, Engelhardt, James & Barnard, 1997). The Olds’ group has implemented a third randomized trial in Denver, Colorado (N=736) which is comparing the effects of nurse and paraprofessional home visitors “in serving low-income, at-risk families during pregnancy and the early years of the child’s life. Both types of home visitors have been trained in the same home-visitation model that was used in Elmira and Memphis. No results of this study have been made available to-date. However, we note that Olds and his colleagues regard the use of experienced Registered Nurses as a key component in the program’s success (Olds, Hill & Rumsey, 1998). Finally, the Olds’ group is currently implementing a program to disseminate this home visitation model in high crime neighborhoods in order to reduce risks for conduct disorder and youth violence and to promote families’ economic self-sufficiency. This dissemination plan is based upon the promising effects which have been noted in the previous randomized trials. •

The Healthy Families America Program

A second current vigorous program of home visitation research and evaluation has focused on the para-professional model which has been promulgated in Hawaii (Healthy Start) and in mainland states with the Healthy Families America Program. Two randomized studies of Hawaii Healthy Start have been implemented. Daro has completed a study with a few promising results at one year (NCPCA, 1997). Another randomized trial with a three-year follow-up has been implemented by Duggan and her colleagues at Johns Hopkins University. The Duggan study is being replicated by Landsverk and Carrilio in a randomized design using an enhanced model of Hawaii Healthy Start in San Diego, California. Neither the Duggan nor Landsverk and Carrilio studies have published outcome results as they are early in the follow-up period. •

Military Home Visitation Programs

A number of research and evaluation studies have been implemented for the various military home visitation programs. However, none of them to-date has used the “gold standard” of research evaluation designs, a randomized design. Such designs are 12

regarded as preferable because they allow many alternative explanations of the results to be ruled out, thereby increasing confidence in the program’s effectiveness. •

U.S. Air Force

The Air Force research and evaluation program has used prospective designs without comparison groups. Prior to a current evaluation, the outcome findings in the Air Force have focused primarily on decreases in scores on the Child Abuse Potential Inventory (CAP). Preliminary analyses found decreases in CAP scores for First Time Parent program participants but no comparable data were collected on a comparison group. A current evaluation of the First Time Parents program, being conducted by Glenda Kaufman Kantor, is gathering multi-dimensional measures of family well being. •

U.S. Marines

The Marine Corps research and evaluation effort was modeled after the Air Force and has used similar outcome measures. This research program, under John Landsverk from Children’s Hospital in San Diego, has found significantly decreased CAP scores as well as decreases on a depression measure (CES-D) (Center for Epidemiological Studies-Depression). However, this program has also not used randomized designs, and has not generally employed comparison groups. •

U.S. Army

Thomas from Cornell University has a current evaluation underway of the Army’s First Time Parents program that uses volunteer home visitors. This study also has not used a randomized design to determine program effectiveness, and the results have not yet been published. •

1999 Workshop on Revisiting Home Visiting

In March, 1999, the Board on Children, Youth, and Families of the National Research Council/Institute of Medicine, sponsored a two-day Workshop on Revisiting Home Visiting in Washington, DC. The workshop included presentations from most of the major investigators currently conducting research on home visitation in the civilian sector of the United States. Included in the presentations was a meta-analysis of 60 studies of home visiting for families with young children by Appelbaum from the University of California at San Diego that concluded that no clear conclusions can be drawn about the field at this time. In addition, Gomby from the Center for the Future of Children and the editor of the volume to be published on Revisiting Home Visitation reviewed the findings to be reported in the volume. She concluded that the overall research results led to a sobering view of the home visitation field, indicating that the outcomes have not been as robust as envisioned by the major initiatives, and therefore the studies do not currently support the wide spread dissemination of home visitation as a efficacious intervention. Furthermore, she called for significant research on factors

13

that impact the quality of the programs, citing numerous studies that indicate that most programs are not being implemented with fidelity to the program models. •

Best Practice Recommendations

Recommendations for best practice and program elements found to be critical to effectiveness are best derived from programs with long-term and thorough evaluations. However, as noted above, questions have been raised recently about the efficacy of past home-visitation models. Olds and colleagues (1998) experience of 20 years in the civilian community suggest the importance of the following program components: • • • • • • • • •

A program targeted to low-income, first-time mothers Home visits by experienced, mature nurses with strong interpersonal skills Duration of home visits from pregnancy to two years after birth Frequency of home visits every 1-2 weeks. Focus on-- the mother’s well-being, environment & quality of caregiving Involve family & friends, & utilize community resources Keep caseloads to a maximum of 25 families Supportive guidance to nurse staff & program oversight by Nurse Supervisor Comprehensive records on families, needs, services, progress & outcomes

Parent Education and Support Programs3 Conventional beliefs suggest that mothering is instinctual, leading many to believe that well-functioning families need little formal guidance in parenting. However, the vast majority of new parents are eager for “how-to” manuals and support. Over the years there have been ongoing efforts to perfect the design of parent education programs. Major programming goals have included facilitating parental caretaking and improving both parent and child well being. In some cases the latter goals have been an end by themselves, and in other cases parenting programs have been directed more specifically to the prevention or reduction of child abuse. Although there is some evidence that parenting education, per se, can benefit parental attitudes and behaviors, some would argue that there is less evidence linking such efforts to child benefits from educationally focused parental programs. The present review will examine the goals and components of parenting programs where the major goals have been on education to increase parenting knowledge and skills, and to decrease the likelihood of child maltreatment. We also review key programs where evaluation information is available. •

Goals of Parenting Education

Parent education programs are generally concerned with increasing parents’ awareness and facility with the skills of parenting (Dembo, Sweitzer, & Lauritzen, 1985). But, there is a broad spectrum of programs, which differ on program content and 3

By Mary Robertson & Glenda Kaufman Kantor

14

location, the target client, and goals or outcomes. Programs may be designed to teach parents general education such as child development, disciplinary techniques, and stress and family management. Like other prevention efforts, parenting programs may be directed at primary or secondary prevention levels with primary programs intended to educate parents in order to prevent maltreatment in the first place. •

Program Structure

Parent education and support programs can be home-based, office-based, center-based or a combination. Programs can take on many forms ranging from formal curricula in secondary schools and college-level courses in child development to short courses offered by trained personnel in the home, in mental health centers or in other community agencies, or even to brief pamphlets distributed by mail (Swift, 1986). Parenting classes are generally targeted to couples or mothers, and there is an increasing awareness of the need for parenting programs to focus on fathers. When program components are considered, parent education research shows that experiential approaches (modeling and skill practice) add to treatment effectiveness in the general population (Graziano & Diament, 1992). In addition, higher levels of parental participation, whether in number of times attending class, number of homework assignments completed or both, have been linked to more positive outcomes for parents and children. The level of participation may be related to a number of aspects of client characteristics such as motivation or aspects of program design and implementation such as program content, personnel and location. Also important are findings that longer-term programs aid in the consolidation gains from parenting education (Burch & Mohr, 1980). Guterman (1997), among others, recommends the use of primary prevention programs that aim to prevent new cases of physical child abuse and neglect. Almost all child fatalities caused by child abuse and neglect occur in the first 5 years of life and when parents are expecting a child there seems to be a window of opportunity during which damage prevention and the promotion of long-lasting positive functioning will prove optimal. Therefore, the recommendation is to use this critical period for early education of both expectant mothers and fathers in parenting skills and alternatives to physical punishment. Parenting programs may also center on parenting children of particular age groups or needs. In fact, much of the prevention literature emphasizes programs providing guidance in parenting adolescent children or children with disabilities and illness because these situations may pose special difficulties for parents. •

Components of Parent Education and Support Programs

As discussed above, the key variables associated with child maltreatment theories include stress, coping, substance abuse, parent-child relationships, parental mental health problems, and deficits in parental empathy and self esteem. These variables have provided a focus for programming goals aimed at reducing problem areas and some key program goals merit further elaboration. 15

•

Counter Social Isolation

Parents who abuse or neglect their children are frequently identified as sharing a pattern of social isolation, poor work history and few friendships (Garbarino & Courter, 1978). Therefore programs which include building a family’s social support network are recommended. Olds and Henderson (1989) conclude that parents may lack the skills necessary to take advantage of the social support made available to them. These researchers regard both education and social support as essential. Engaging isolated parents with a high potential for child abuse may be challenging as these parents may wish to avoid discovery because of family secrets such as abuse or alcoholism. Isolated parents may also include those too depressed to reach out for assistance. •

Increase Parental Competence

Parents who have abused their children are believed to overestimate their children’s abilities (Spinetta & Rigler, 1972). A common program inclusion is the provision of information about normal child development and what to expect for children at specific ages. The concept of parental competence has been the focus of many programs. In seven out of nine studies testing gains in parental competence, these types of interventions were found to reduce child neglect (Burch & Mohr, 1980). •

Stress Management

An increased awareness of the effects of stress on families has brought stress management and stress related topics into parent education curricula as well. Pregnancy itself, or in combination with other stressors, may be associated with wife assaults (Amaro, Fried, Cabral, & Zuckerman, 1990; Campbell, Poland, Walder, & Ager, 1992), and if the pregnancy is unplanned and unwanted, the risk for child abuse is greater (Swift, 1986). Many programs focus on teaching parents how to reduce the level of stress associated with pregnancy and parenthood by rehearsing the anticipated sequence of events and planning coping strategies to meet various challenging situations (Swift, 1986). •

Evaluation of Model Programs

Some experts believe that a more comprehensive approach to parent education, targeted to a broader audience, is needed. To appeal to this broader audience, parenting programs need to become more conceptually comprehensive, procedurally interactive, and topically relevant to parental backgrounds and needs (Robertson, 1984; Schroeder & Gordon, 1991). The application of any one particular canned approach no longer appears warranted, and instead an integration of strategies seems necessary (Brems, Baldwin, & Baxter, 1993). Nevertheless, we review some of the program packages and model programs in common usage, and provide evaluative data where this is available. Unfortunately, most prevention programs have not been evaluated comprehensively, and unlike the Olds’ research, discussed above, often we have no 16

formal knowledge of long-term effects of these programs on children and families. We have also summarized below descriptions and information on a number of parenteducation programs that have been evaluated in the United States. •

Common Sense Parenting Program

In a review of the Common Sense Parenting Program, a program designed to teach child-management skills, Thompson, Grow, Ruma, Daly & Burke (1993) found that it is cost effective to do group training as opposed to home-based training. Thirtyfour parents completed an 8-week parenting program, while 27 parents were in a control group. The results of an evaluation indicated that parental reports of child behavior problems, parent attitudes, and parent problem-solving skills improved significantly after parent training. •

C.P.E.P.

The Child Parent Enrichment Project (CPEP), a 12 hour education program over a 6 week period, was evaluated by Barth and associates (Barth, Hacking, & Ash, 1988). They conducted a pretest, and posttests at 6 months, 2 years and 5 years with treatment and control groups. Program participants showed advantages over controls in prenatal care, birth outcomes, and reports of child temperament and indicators of child welfare. Program mothers also were more likely to report better well being. However, both groups demonstrated similar levels of support and there was no significant differences in reports of child abuse. It is important to note that the authors conclude that there were no significant differences between treatment and control groups in the 2 and 5-year self-reported measures. There was some indication of greater success with families with less severe problems. One inference from these findings is that families most in need may not be adequately served by this type of brief program. • Parent Effectiveness Training One time-tested primary prevention model is Parent Effectiveness (previously called Parent Effectiveness Training or PET). This is an eight session program, three hours per sessions, (total of 24 hours) designed by Gordon in the early 1960's. A recent revision to the Parent Effectiveness program involved grouping the eight sessions into two parts so that parents did not have to make a long time commitment to start the program. It allowed parents to take the second part (additional four classes) at a later time. Its primary focus is on developing close and enjoyable relationships with family members, supportive family environment, and developing child rearing skills in parents (Alvy, 1994). Cedar and Levant (1991) conducted an analysis of the Parent Effectiveness program and found the program had the following effects:

17

• • • • • •

•

greatest measurable effect on parental attitudes such as increased understanding of children, democratic ideals, acceptance, and genuineness; measurable positive effects on parent behavior; parents learned the course concepts; effects on child outcomes were greatest for the category of child selfesteem; children perceived their parents as being more accepting; parents were found to improve their degree of acceptance of their children.

Perinatal Positive Parenting

Perinatal Positive Parenting, a child maltreatment Prevention program, offers parent training and information on child care and development, community support groups for new parents, home visits by trained volunteers, and a “warm line” to call for support and information. The model includes weekly contacts with new mothers for two to three weeks after the birth of the baby. Appointments for home visits and invitations to group meetings are made as well.

One Michigan program evaluation of first-time mothers initially found no significant differences between treatment and control on the two measures used, the Bavolek Adult/Adolescent Parenting Inventory or the Broussard Neonatal Perception Inventory (NCCAN, 1983a). However, later posttest results showed increased maternal involvement and child-nurturing environments with younger mothers. This could be attributed to delayed effects for younger mothers (NCCAN, 1983a). • Pride in Parenthood Pride in Parenthood, a first-time parents program, consisted of a model with biweekly support group meetings for young parents in an urban Virginia neighborhood. It targeted Navy and civilian families and young couples on the premise that the isolated young family is at the highest risk for child abuse (NCCAN, 1983b). At the heart of the program was the provision of a family friend for new parents. This person was a volunteer, trained in parent education and support for high-risk families. Contacts were made weekly and home visits and group meetings were encouraged. Evaluation of the program showed that parents in the program had more realistic expectations of their infants than parents in a comparison group. Program parents showed improvements in parenting behavior as measured by the Bavolek Adult/Adolescent Parenting Inventory,l and the Broussard Neonatal Perception Inventory (NCCAN, 1983b).

18

• Project 12-Ways Project 12-Ways is an “ecobehavioral” approach to treating families who have been reported for child abuse and/or neglect. Ecobehavioral programs focus treatment and assessment on changing behavior within the family’s social setting, generally within the home. The model program includes parent-child interaction training, health maintenance and nutrition assistance, home safety training, counseling, job counseling, and referral for treatment of substance abuse (Lutzker, Wesch, & Rice, 1984). Evaluators reviewed reports of child abuse and neglect and found that at year 1 there was a 21 % recidivism rate of child abuse and neglect for the treatment group compared to a 31 % rate for the comparison group. In a second evaluation of Project 12-Ways, both full-program treatment groups and routine-service groups experienced decreases in child abuse and neglect as measured by rates of abuse and neglect charges, rate and severity of recidivism and out-of-home placement (Wesch & Lutzker, 1991). According to Lutzker and colleagues (1998), evaluation outcome data indicated that three of the project components were critical intervention strategies in preventing child abuse. The critical components were: • • • •

infant and child health care, home safety, and stimulation/bonding or parent-child interaction.

S.T.E.P.

Systematic Training for Effective Parenting (STEP) (Dinkmeyer & McKay, 1976) is a nine week (18 hours total) program, and has as its primary goals the strengthening of family life, positive relationships, parental child rearing skills, and self-confident and independent children. A series of studies on the STEP program showed parents to be more accepting of their children, more authoritative in parenting style and that they had a better understanding of their children’s behavior (Alvy, 1994). An important finding was that parents perceived improvements in relationships with their children and spouses (Lifur-Bennet, 1982). In addition, program parents reported that the group setting was the most important program feature contributing to changes seen in family relations. This was due to the comfort they felt knowing that others were facing similar problems. Evaluations of Paternal Effects In spite of the fact that the most severe acts of child maltreatment are committed by father figures, or male caretakers, few prevention programs or evaluations are able to measure the program’s success with fathers. For example, an evaluation of a parent education program known as Parenting Path used a parenting Inventory and family assessment tool to measure stress and family changes. The authors (Peterson & Hawley, 1998) suggest that parent education is especially important for fathers but that 19

their significantly lower scores on the parenting measures may indicate that the instrument does not adequately measure variables more closely associated with fathering than mothering. It may be that measures need to be developed which are sensitive to the father's role in child rearing McBride (1991) examined the effects of a parent support program designed specifically for fathers. The program was based on a two-step experiential model (Klinman, 1986) intended to increase men’s involvement in child rearing activities. The model included a father’s discussion group and father/child playtime. The evaluation results indicated that program participants (27 fathers) showed improved scores as compared to a control group (27 fathers) on the parent domain variables of the Parental Stress Inventory. Improvements were found for scores on the father’s sense of competence, social isolation, and depression. The author concludes that programs such as this may be an effective means of reducing the perceived parental stress fathers experience as they attempt to become more actively involved in raising their children. He suggests that parent education and support programs be designed specifically for fathers in order to meet the needs of today’s families by giving fathers information and support regarding their role in parenting. Discussion This review of parent education and support programs focused primarily on the goals of improved parenting and reducing child maltreatment. The results presented here suggest that program components should include an educational element to improve parental skills and parental competency as well as an element of continued social support. These types of programs are most successful as early intervention and have not yet been demonstrated to have the capacity to reduce or prevent abusive or neglectful behaviors significantly over time for the majority of families who have been reported for child maltreatment. Overall, analyses of comparison studies indicate that the search for the most effective parent education program has been nonproductive. Each program has different goals and objectives and its effectiveness may depend on the specific needs of different parent and child populations (Dembo, Sweitzer, & Lauritzen,1985). Prior reviews of this literature have found some change in parental attitudes, but these changes were not consistent, and varied across assessment tools. Of the programs, on which we presented data, the most significant results were identified for a program with a more comprehensive, ecological model of service provision (Project 12 Ways). It is difficult to say conclusively that a particular program has caused changes in the parents’ behaviors. It is also difficult to identify the “best programs” because many evaluation efforts have never been published, and often, even when a publication exists, more information is provided on the program outcomes than on the program activities. Nevertheless, given the available information, we’ve attempted to identify best practice in parenting education programs. Important program components in parenting education linked to child maltreatment prevention are listed below (also see Guterman, 1997).

20

•

Best Practice Recommendations • • • • • • • • • • • •

Primary/Early Intervention – primary prevention works best to reduce child maltreatment. Social Support--a key component , includes linking families to formal & informal supports. Parental Competency – increasing a parent’s skills in parenting and knowledge of child development helps reduce the risk of child maltreatment. Alternatives to physical punishment--information may reduce child maltreatment. Parental Stress Reduction --parents and children benefit from the parent’s ability to identify and reduce stress. Broad-based Programs --appeal to a general audience but reach and engage higher risk families as well. Experiential Programs – skill practice (role playing and discussion groups) with other educational approaches increases program effectiveness. Participation Levels –high levels of parental participation are associated with the most gains. Program components, such as location, convenience and personnel are important in increasing program participation. Non-Home Based Programs--may build community support and cohesion. Longer programs or follow-up booster sessions—may consolidate gains in parenting knowledge. Longitudinal Evaluations-- are needed to test for positive effects of early intervention in the long run.

The following table displays a synopsis of the parent education programs reviewed.

Study or Author(s) Cedar & Levant (1991)

Program Name

Program Description

Evaluation Description

Evaluation Results

Parent Effectiveness (Parent Effectiveness Training/ PET)

24 hours over 8 weeks. Focus: relationships/ support/child rearing skills.

Meta-analysis of 26 PET program evaluations

+ parent attitudes + parent skills + parent knowledge +parent/child relations +positive effects at 26 weeks

21

Study or Author(s)

Program Name

Alvy (1994) & Lifur-Bennett (1982)

Barth, Hacking & Ash (1988)

Program Description

Evaluation Description

Evaluation Results

Systematic Training for Effective Parenting (STEP)

18 hours over 9 weeks. Focus: family life/relationships/child rearing skills/selfconfidence

Meta-analysis of 31 STEP programs

+ understanding of child behavior + authoritative in parenting style + parent response to group learning

Child Parent Enrichment Project

12 hours over 6 weeks. Focus: after abuse relationships/ parenting skills

Pre and post test (6mos, 2yr, 5yr). Treatment and control groups.

Program participants: + prenatal care + birth outcomes + reports of child temperament and welfare. Posttests at 2 and 5 years. No significant differences.

National Center on Child Abuse and Neglect (1983a)

Perinatal Positive Parenting

Community support groups, home visits, and “warm line” phone support.

First-time mothers/ Michigan. Treatment and control groups.

No significant differences on Bavolek Adult/Adolescent Parenting Inventory (BAAPI) and Broussard Neonatal Perception Inventory (BNPI).

National Center on Child Abuse and Neglect (1983b)

Pride in Parenthood

First-time parents. Biweekly support group.

Treatment and control groups.

Program group: + scores on both BAAPI and BNPI.

22

Study or Author(s)

Program Name

Program Description

Evaluation Description

Evaluation Results

Alvy (1994)

Positive Parenting

For abusing parents. Weekly group meetings.

Treatment and control groups.

Program group: + change in stress factors + change is isolation factors + change in knowledge of child development + change in attitudes/ values

Lutzker, Wesch & Rice (1984)

Project 12 Ways

Parent-child interaction training/ health and nutrition/ home safety/ counseling/ job finding/ referral for treatment and substance abuse.

Treatment and comparison group.

Program group: + outcome for (decreased) rate of child abuse and neglect

Thompson, Grow, Ruma, Daly & Burke (1993)

Common Sense Parenting Program

Teach child management skills.

34 program parents; 27 control parents.

Program group: + outcomes in parental reports of child behavior + parental attitudes + parent problemsolving

Peterson & Hawley (1998)

Parenting Path

Broad parent education program aimed at a large and diverse population on a volunteer basis.

524 parents of newborns (322 female and 220 male).

A negative relationship between the number of stressors perceived by parents and their perceptions of family environment and attitudes toward parenting.

Wesch & Lutzker (1991)

Measure: selfreport stress index

23

Study or Author(s)

Program Name

Program Description

Evaluation Description

Evaluation Results

McBride (1991)

None given.

Parenting education for fathers/ group and father-child playtime.

27 treatment and 27 control fathers.

Parenting Stress Index results show + outcomes for parent domain. All other domains show no significant differences.

Violence in Families: Committee on the Assessment of Family Violence Interventions (1998)

Comparison of studies.

Programs to avert child abuse and neglect.

Evaluation of 10 studies.

7 out of 9 showed improved parental competence reduced child neglect. 1 showed that a program designed to (1) change parental perceptions and expectations; (2) to teach relaxation and mediate stress and anger; and (3) to teach problemsolving skills, produced the largest change in index scores.

24

Theories and Risk Factors for Marital Violence4 Theories of Marital Violence Theories on the etiology of marital violence are similar in many ways to those discussed above for child maltreatment. There is some dispute over the relative importance of one theoretical framework over another. For example, some argue for a social-structural framework (Miller, 1994; Renzetti, 1994) while others favor an emphasis on psychopathology (Dutton, 1994). Early theoretical development on wife abuse (Gelles & Straus, 1979) identified fifteen theories, organized into three broad categories: intra-individual theory, social psychological theory, and socio-cultural theory (see also Bersani & Chen, 1988) that provide guidelines for risk factors that need to be considered. •

Intra-Individual Theory

Intra-individual theory has emphasized alcohol-drug effects (Kaufman Kantor & Straus, 1987; Kaufman Kantor & Straus, 1989), and psychological traits such as self esteem (Hamberger & Hastings, 1986; Hudson & McIntosh, 1981; Roy, 1977) and antisocial personality disorder (Holtzworth-Munroe, 1994). A growing body of research suggests the importance of including personality, neurological and even physiological factors in models of relationship aggression (Miller, 1994). Increasingly, attention is being drawn to variations in psychological pathology among batterers perpetrating more severe abuse against their spouses (Dutton, 1994; Gondolf, 1988; Holtzworth-Munroe, 1994). •

Socio-Cultural Theory

Socio-cultural theories focus on the influence of social location (social class, education, income) on family violence and have attempted to integrate both social structural and family processes (Kaufman Kantor, Jasinski, & Aldarondo, 1994; Straus, 1973). Higher rates of family violence are associated with the stresses and strains of chronic poverty and unemployment. Feminist explanations of women's victimization also underscore social and cultural factors, especially the male-dominated social structure, normative attitudes approving violence and socialization practices (Pagelow, 1984; Smith, 1990; Yllo, 1984). The major constructs in a feminist analysis of wife abuse are the structure of relationships in a male dominated culture, power, and gender inequality (Bograd, 1988). •

Social-Learning Theory

Social-psychological approaches have stressed social-learning through experience and exposure to violence in the family (Kalmuss, 1984; O'Leary, 1988; Straus, Gelles, & Steinmetz, 1980). For example, O'Leary's (1988) social learning 4

By Sarah Avery-Leaf & Glenda Kaufman Kantor

25

model of spousal aggression incorporates five major factors that predict spousal aggression: 1) Violence in the family of origin; 2) Aggressive personality style; 3) Stress; 4) Alcohol use and abuse; and 5) Marital dissatisfaction. While intergenerational transmission of violence has received the most support among risk factors examined in a number of studies, it is evident that marital violence is a complex phenomenon. Any one factor by itself, is likely not sufficient to produce marital violence. Prevention of Partner Violence Compared to programs for the prevention of child abuse, primary prevention efforts in the area of partner violence are in their infancy, and few evaluations of prevention programs have been conducted. Prevention strategies in this area appear to be on the rise and some promising efforts are being made, especially at the community level, and within health care settings. Hamby’s (1998) review of primary prevention of partner violence identified community level programs aimed at reducing societal tolerance for partner violence and programs aimed at augmenting the resources of women and families. Several community campaigns have been implemented to increase awareness of domestic abuse, and to decrease the public tolerance for intimate aggression. Health care programs aimed at screening and identification of woman abuse, particularly among pregnant women, is a major step in addressing the broader dimensions of family violence. Programs aimed at preventing the intergenerational transmission of violence by targeting child witnesses to violence (e.g., the children of battered women in shelters) may be among the most important of prevention strategies. However, many evaluations of primary prevention of partner violence are either still underway or not available in the evaluation research literature. Preliminary evaluations of short-term group work with child witnesses suggest that these programs are most effective with children who demonstrate less severe adjustment problems (Wolak & Finkelhor, 1998). •

Marital Enrichment/Couples Programs

Treatment programs for spouse abusers are commonplace, and many services exist to provide for the needs of the victims of spouse abuse. However, evaluations, or even descriptions, of primary prevention efforts for adults are limited. Rather, primary prevention of spouse abuse has generally been undertaken using one of two approaches: firstly, prevention programs for adolescents, typically school-based and time-limited, and secondly, couples programs, targeting engaged or newlywed couples, and designed to enhance communication and other skills thought to strengthen the relationship. For the most part these types of programs have focused on protective factors (e.g., communication skills) rather than risk factors. The literature describing the couple’s communication approach is reviewed below, and summarized in the table below. Most evaluation efforts in partner violence have been conducted on programs targeting spouses already engaging in violence. Such programs can be categorized as a tertiary prevention effort, and they are exemplified by the Domestic Conflict Containment Program (DCCP) discussed below. School-based violence prevention programs will be reviewed separately, in the following section. 26

•

Couples’ Groups vs. Gender-Specific Treatment Programs

An important distinction must be made at the outset between marital treatment aimed at intact couples, and that which focuses on one or the other partner (e.g. batterers’ treatment, or victims’ support groups). Although a review of the literature investigating the pros and cons of each approach is beyond the scope of this review, suffice it to say that researchers and clinicians tend to agree that gender-specific treatment is indicated in cases in which 1) there is severe aggression reported; 2) one spouse (typically the wife) reports high levels of fear or intimidation; and 3) a separation is being considered, or has been initiated. While still regarded as controversial, some view couples treatment as warranted when the aggression is mild and also reciprocal, and also when spouses report high levels of commitment to the relationship (Brown & O'Leary, 1997). Evaluations of Couples’ Programs Author(s)

Program Title/ Location of Study

Length/Descripti on of Program

Sample and Design (pre-post only unless noted)

Evaluation Results

Markman, Renick, Floyd, Stanley & Clements (1988; 1993)

Prevention and Relationship Enhancement Program (PREP)

6 2-hr weekly sessions of lecture on relationship issues plus private time with consultant to practice communication skills as a couple

N=114 couples (25 received PREP; 47 controls; 42 declined program)

10 weekly 2-hr group sessions covering anger and stress mgmt., communication and conflict resolution

40 husbands; 19 wives

PREP couples showed higher levels of satisfaction, positive communication skills, and lower levels of violence Significant changes in marital adjustment and locus of control

See above

53 abusive males and 29 female partners completed the program ____ ____

Denver, CO Neidig & Friedman, 1984

Rynerson & Fishel, 1993

Domestic Conflict Containment Program (DCCP) Military Bases (unspecified location) DCCP Chapel Hill, NC

HoltzworthMunroe, Markman, O’Leary, Neidig, Leber, Heyman, Hulbert & Smutzler, 1995

(PREP/SAVE)

Combination of the Markman and Neidig programs)

5 year longitudinal study

Participation mandated for abusive service member, partner optional

Significant changes in marital adjustment and locus of control Evaluation results not yet reported

Studies of domestic violence in military populations have been extremely limited but some findings suggest that couple violence tends to be reciprocal, occurs in the 27

context of an escalating argument; and may increase as a function of work stress (Neidig, 1986). However, many of the qualities or risk factors attributed to military couples may also characterize spouse abuse in general population groups (Kaufman Kantor & Asdigian, 1996; Kaufman Kantor & Jasinski, 1998). When Neidig compared military men who assaulted their wives with their nonviolent counterparts, differences were found on stressful events (more for violent group); marital discord; and locus of control (the violent group perceived control as more external to self) (Neidig, 1986). Neidig’s adaptation of his spouse abuse treatment program for military couples (Neidig & Friedman, 1984) was termed the Domestic Conflict Containment Program (DCCP). Based on their prior research, the authors developed a program emphasizing conjoint treatment, communication skills, anger management, issues of personal control, and work-related stress management. This program has been widely used in military settings, although an extensive, controlled evaluation of the effects of DCCP has not been published to date. Care needs to be taken about the population that is targeted for particular programs (i.e., which program for which group?). Emphasizing communication skills with couples, in which the husband is the primary abuser, may focus unduly on the wife’s share of the communication burden. Recent research shows that violent husbands tend to react negatively to the wife’s attempt to raise problematic or conflictual issues with them. Communication training might actually precipitate violent episodes in some problem-solving exercises (Holtzworth-Munroe, 1997). •

An Ounce of Prevention

Couple prevention programs are an efficient and sensible approach to the prevention of future marital problems. Not only is it easier and cheaper to help partners avoid engaging in destructive behavior patterns (rather than helping long-married couples stop using these patterns), but this approach also allows for larger numbers of people to receive treatment, as many couples can attend a single workshop simultaneously. These programs are numerous, but few have been subjected to rigorous evaluation. One exception, the Prevention and Relationship Enhancement Program (PREP), which stems from the work of Howard Markman, (1988; 1993), has been subject to an extensive longitudinal evaluation (Markman, Renick, Floyd, Stanley, & Clements, 1993). It is also important to note that this program’s primary purpose is the prevention of marital dissolution rather than the prevention of partner violence. Evaluation has showed promise for those who completed the program. Program couples scored higher on communication, satisfaction, and reported less violence than a control group at the 4-year follow-up. Some limitations should be noted in drawing conclusions from this study. First, no baseline measures of marital violence had been conducted. Secondly, over half of couples offered PREP either declined or dropped out. The final sample may have shown better outcomes simply because it consisted of couples with more relationship strengths at the start than either non-completers or decliners. Overall, the

28

research indicates that relationship-enhancing programs are more effective when implemented before marriage than with married couples (Bradbury & Fincham, 1990). •

Putting it All Together: Relationship Enhancement plus Treatment for Violence

Although violence researchers agree on the need to prioritize efforts targeting the prevention of domestic violence, there are surprisingly few interventions that are empirically based. A recent literature review (Holtzworth-Munroe, Markman, O'Leary, Neidig, Leber, Heyman, Hulbert & Smutzler, 1995) summarized findings indicating that aggression begins early in the relationship. A surprising 1 in 3 couples experience physical aggression in their relationship prior to getting married (McLaughlin, Leonnard, & Senchak, 1992; O'Leary, Vivian & Malone, 1989). Further, the aggression may reoccur, establishing a pattern of aggressive interaction over time (Feld & Straus, 1989; O'Leary, Vivian & Malone, 1989). Following these principles, the authors suggest that a successful secondary level intervention (directed at an at- risk group) might incorporate the following: • • • •

take place early in the relationship; focus on those who have already engaged in mild aggression, as spouses who have never aggressed are less likely to start; involve couples who are using mild (and NOT severe) forms of aggression; and use conjoint approach for couples who intend to stay together.

Holtzworth-Munroe, Markman, O’Leary, Neidig, Leber, Heyman, Hulbert and Smutzler (1995) used the above findings to guide the development of a prevention program for engaged and newly married couples at risk for violence. The group was limited to those families who had not developed entrenched patterns of severe and frequent violence. This collaborative effort by several spouse abuse researchers has resulted in a new and comprehensive violence prevention program (Holtzworth-Munroe, Markman, O'Leary, Neidig, Leber, Heyman, Hulbert & Smutzler, 1995), combining PREP (Markman & Floyd, 1980) and a program derived from the DCCP, termed SAVE (Stop Anger and Violence Escalation) (Neidig, 1989). Plans to evaluate the impact of the PREP/SAVE program are currently underway. PREP/SAVE is based on a behavioral-cognitive, social learning model. The idea is that violence is a learned behavior, which has been reinforced by the partner’s response (i.e. the victim capitulates, and the aggressor gets what he/she wants). The program objectives, derived from the spouse abuse literature, are to teach communication, problem solving, and anger management skills (alternatives to aggression) to change attitudes accepting of aggression (by raising awareness of negative consequences of aggression), and to address risk factors for spousal violence. Ultimately, it is hoped that accomplishing these objectives will help couples maintain healthy relationships in order to preclude violence.

29

•

Conclusions on Couples Programs

In general, the research discussed herein suggest three important points to bear in mind when designing a “best practice” model for couples’ treatment. First and foremost is the issue of how to deliver services to those who would benefit most from them. This is both a “who” and a “when” question, in that couples at risk for or engaging in violence may be less interested in treatment than couples using a healthier approach to conflict resolution. These couples typically don’t want treatment until their problems with conflict and/or aggression have become quite severe (Bradbury & Fincham, 1990). At this point, problems are more entrenched and thus harder to treat—and often couples don’t seek treatment willingly. One solution to this is to make the implementation of prevention programs a priority, and to appropriately target and screen families for the programs. Secondly, teaching communication, anger management, and problem-solving skills, and allowing in-session time for participants to practice these, is an important priority. In fact, communication problems are the most frequently cited reason for seeking conjoint therapy among marital populations (O'Leary, Vivian, & Malone, 1992). Giving couples the tools to express anger differently may be a deterrent to violence, but more evaluation research is needed in this area. •

Best Practice Recommendations • • • •

•

Screening the level of violence in program families Prevention programs specific to the level of at-risk status Prevention programs which address the risk factors for spousal violence Inclusion of a skills component to the prevention program

School-Based Partner Violence Prevention

Intimate partner abuse is a widespread phenomenon that occurs at all levels of romantic involvement, from early dating relationships in middle school (Foshee, Linder, Bauman, Langwick, Arriaga, Heath, McMahon & Bangdiwala, 1997), throughout adolescence and into adulthood (Arias, Samios, & O'Leary, 1987; Makepeace, 1986; O'Keefe, Brockopp, & Chew, 1986). Although there are many domestic violence treatment programs in existence, there are very few published evaluations of partner violence prevention programs, thus our knowledge as to the effects of such interventions is limited. Overall, these programs tend to be conceptualized as primary prevention efforts and are aimed at adolescents in high school, despite the fact that many teens have already experienced violence in their earliest middle school “dating” relationships). The literature on these programs is reviewed below, and summarized in Table 2.

30

•

Issues of Gender and Feminist Theory

One of the most important, as well as controversial, issues pertaining to schoolbased partner violence prevention lies in the theoretical framework upon which the program is based. Many domestic violence agencies provide services to schools, typically consisting of an in-class or schoolwide presentation featuring a formerly battered woman or speaker from the agency. These presentations tend to frame the issue of partner violence in “gendered” terms, i.e., focusing on males as perpetrators and females as victims. Several of the evaluations published investigate programs based on this feminist model (Jaffe, Sudermann, & Reitzel, 1992a; Jones, 1987; Krajewski, Rybarik, Dosch, & Gilmore, 1997; Lavoie, Vezina, Piche, & Boivin, 1995). For example, two evaluations of Levy's (1984) domestic violence prevention program, Skills for Violence-Free Relationships, have been undertaken (Jones, 1987; Levy, 1984). In its entirety, this program is a multi-session curriculum for adolescents based on a gendered perspective, i.e., that beliefs and adherence to traditional sex roles and acceptance of male dominance within a relationship are at the root of partner violence. Both studies failed to demonstrate change in high school students’ attitudes toward the use of violence. Another evaluation of a presentation by several speakers from the community (including police and domestic violence agency personnel) indicated significant positive attitude changes for most students, but also showed a "backlash" effect (attitude changes showing increased acceptance of violence) for some of the male students. This finding may suggest the use of gender-neutral materials rather than those based on a gendered model of domestic violence (in which males are perpetrators and females are victims), so as to avoid defensiveness from male participants. It is also important to note that research among teens has consistently shown that higher percentages of teenage girls report having used physical aggression against a dating partner than do boys of that age group. Girls also report having sustained more injuries as a result of dating violence (Avery-Leaf, Cascardi, O’Leary & Cano, 1997; Foshee, Linder, Bauman, Langwick, Arriaga, Heath, McMahon & Bangdiwala, 1996). Much debate has arisen from this finding, and there are several possible explanations that have not been tested to date. One potential reason for females’ higher rate of aggression may have to do with reporting styles-- that girls are less likely than boys to minimize their own aggression. This may occur because girls aggression is viewed as less injurious and consequential than boys. While further research is needed to more fully understand this phenomenon, there is a still a need for violence prevention programs to address perpetrators and victims of dating violence without specifically targeting either gender for these categories. •

Timing is Everything…Or is it?

An important aspect of the primary prevention of intimate partner violence lies in the question of when to intervene—that is, what age should the targeted group be? Many researchers and educators favor interventions aimed at adolescents at the point 31

at which they begin having dating relationships. Unfortunately, this in itself is problematic as there is no consensus (and very little research) establishing the age of dating onset. The conception of dating may differ across groups (e.g., “hanging-out “ with a group vs. solo couple dating). Further, there are no studies that address this question by comparing program impact among different age groups. It is also difficult to demonstrate program efficacy when the problem behaviors haven’t yet begun. What is known, however, is that students report using physical aggression in their dating relationships early on—as early as they start dating, which for many teens is at around age 12 (Avery-Leaf, Cascardi, O'Leary, & Cano, 1997; Foshee et al., 1996). Thus the question of whether a violence prevention program is actually accomplishing primary prevention (as opposed to tertiary) is raised. In any event, the fact that many students may have already experienced (and may still be engaging in) violence in their dating relationships highlights the importance of providing a list of resources for those students needing help. •

Issues of Dose and Effects over Time

Despite agreement on the importance of preventing intimate partner violence, surprisingly little time has been devoted to these efforts in the classroom. Programs range in length from a low of 2 ½ hours (Lavoie, Vezina, Piche & Boivin, 1995) to a high of 10 classroom sessions (Foshee, Bauman, Arriaga, Helms, Koch & Linder, 1998; Krajewski, Rybarik, Dosch & Gilmore, 1997), which still represents a relatively low dose. Only two studies specifically reported making comparisons between a longer and a shorter intervention (Jaffe, Sudermann, Reitzel, & Killip, 1992b; Lavoie, Vezina, Piche & Boivin, 1995). In the Jaffe, Sudermann, Reitzel & Killip (1992b) study, students either received a full-day program or one spanning a half-day, but results were reported for all students receiving a program, and so it is not clear whether there were differences based on dose. The other study’s results were complicated by the fact that the groups differed at baseline, and both groups showed changes after receiving the program (Lavoie, Vezina, Piche & Boivin, 1995). This suggests that we still don’t know whether longer programs would show stronger, or more lasting effects on students’ use of aggression and aggression-related behaviors. Another unresolved issue has to do with the effects of violence prevention programs over time. As these programs are cumbersome and extremely costly to evaluate, most do not include a long-term follow up assessment. In the only evaluation reviewed to use a longitudinal design (Krajewski, Rybarik, Dosch & Gilmore, 1997), initial effects were found on both knowledge and attitudes but these washed out at the 5-month follow-up. Thus, the little that is known about the issue is not very promising. On the other hand, several longitudinal program evaluations are currently underway, so there should be some understanding of this issue within the next few years.

32

•

Conclusion

Taken together, these studies demonstrate that prevention programs tend to be effective at changing attitudes accepting of aggression, as these are strongly correlated with aggressive behavior (Cano, Avery-Leaf, Cascardi, & O'Leary, 1998). Behavior change has not been consistently demonstrated, and differential effects by gender appear to be fairly common (more change by girls). In general, program evaluation findings indicate promising effects on attitudes, knowledge, and beliefs associated with partner violence. By itself, this is an important accomplishment. Unfortunately, immediate program effects on violence have not been clearly demonstrated. Although these programs are conceptualized as primary prevention efforts (i.e., stopping problem behaviors from occurring in the first place), they are often interventions, because many adolescents engage in some form of partner aggression in their very earliest intimate relationships. The difficulty in demonstrating program success in decreasing violent behaviors is due partially to the fact that participants vary in their backgrounds and dating experience. Bringing the topic of domestic violence into the classroom, in combination with the various changes in policy and services for victims of violence, augers well for future reduction of this phenomenon. Evaluations on School-Based Programs Author(s)

Jones, 1987

Krajewski, Rybarik, Dosch & Gilmore, 1996

Jaffe, Sudermann, Reitzell & Killip, 1992

Lavoie, Vezina, Piche & Boivin, 1995

Program Title/Location of Study Skills for Violence Free Relationships

Skills for Violence Free Relationships Urban Wisconsin No Title

Urban/rural Ontario

No Title Quebec City

Length/ Description of Program 5 Class Periods Focus on domestic violence and male battering 10 health classes led by teacher and counselor from women’s shelter

1 ½ hour auditorium presentation by community speakers (plus ½ day of student action plans, e.g. play, fundraiser)

2 ½ hours of classroom material (plus 2 supplemental class periods)

Sample and Design (pre-post only unless noted) 566 junior high students 598 high school students quasi-experimental control group design 239 seventh graders quasi-experimental control group design, pre-post and 5 month follow up 637 high school students pre-post (6 week follow up for ½ the sample)

417 10th graders 2 schools; one school received auditorium presentation, other received auditorium plus student action

33

Evaluation Results Knowledge changed but not attitudes (both schools)

Females Better Attitudes? Yes

Effects at post but these washed out by follow up

Yes

Results for full sample (not broken out by full or ½ day) indicated some effects but also a backlash which was sustained at follow up Schools differed significantly at pre and at post; attitude changes for both groups

Yes

Yes

A few issues have emerged as defining “best practice” for young daters in a prevention program. As the goals of a school-based primary prevention program overlap considerably with a couples’ prevention program, the same tenets apply to both formats. We recommend that school-based programs include a skills component, and utilize a group format for maximum efficiency and cost effectiveness. •

Best Practice Recommendations • • • • •

Use of a gender-neutral approach to avoid blaming or backlash effects of defensive responding Focus on the training and practice of skills required for non-violent conflict resolution (e.g. anger management, assertive communication, active listening) Focus on changing attitudes accepting of aggression as a conflict tactic Utilization of booster sessions to maintain program gains Supplementing program interventions with referrals to peer counseling and other mental health services

Common Concepts of Prevention Programs Risk and Protective Factors as a Framework for Prevention Efforts Preventive interventions should be directed towards the common antecedents of the problem behaviors which are being addressed. Antecedent factors are placed in an ecological framework which encompasses demographic, individual, family and community risk factors as central to multiple high risk behaviors. These factors also mirror the four ecological levels discussed above (i.e., ontogenetic, microsystem , exosystem, and macrosystem; Kaufman & Zigler, 1993). Ontogenetic risk factors include history of abuse, alcohol abuse, stress, low IQ, and psychiatric (psychological profile) and physical illness. On the microsystem level there are the aspects of family environment which increase the likelihood of family violence such as marital discord, or single parenthood in the case of child maltreatment. Exosystem risks would reflect the quality of the neighborhood environment , the extent of social isolation, community disorganization, and lack of services. Finally, macrosystem risks include economic conditions, cultural acceptance of corporal punishment, and attitudes, in general, about the appropriateness of violence towards an intimate. It can be seen that the origins of problem behaviors, including all forms of maltreatment, overlap. Marital conflict, couple violence, physical and sexual abuse , and other hardships can be inter-related in their occurrences in families. It would seem to follow logically that the most successful prevention programs would provide multidimensional approaches focused on the four levels of need, and reflecting the inter-relatedness of problems. However, this is a tall order for any one program. In fact, despite recognition of these complexities spanning decades, the majority of prevention programs have been rather singular in their focus and have not 34

cut across disciplinary or “interest group” lines. For example, child maltreatment prevention programs have not had marital violence prevention as a program goal. Until recently, only a minority of family violence prevention programs have attempted a truly ecological focus. Two recent influential reports by scientific bodies (see, Cowan, Cowan & Schulz, 1996) emphasized the need to take into account studies of development and risk for designing and evaluating preventive intervention programs. Although the reports point to the need to identify social context variables such as social class, ethnicity and neighborhood, they recommend that intervention efforts address the proximal contexts of family, schools and peers. These are regarded as linking high levels of risk with the most severe outcomes (Cowan, Cowan & Schulz, 1996). Another paradox in family violence prevention efforts is that despite mounting evidence on the fundamental importance of ending corporal punishment to preventing physical abuse, corporal punishment has been ignored in the literature on prevention of physical abuse. In addition, many prevention programs have not taken a clear or strong stand against all types of corporal punishment. Recent developments in the prevention area include the recognition that programs must build buffers to create resilience in individuals and families. Awareness of the areas of risk and resilience owe much to the work of Rutter (1979). Increasing efforts have been directed at clarifying and refining the concept of resilience. The major shift has been from viewing resilience as the absence of pathology to recognizing it as a manifestation of competence and adaptative behavior in the face of adversity. At the cognitive level, one’s appraisal of a situation could determine the stress invoking potential of the circumstance. One can reduce the impact of risk by changing its meaning. For example, preparation for the transition to parenthood might buffer the transition to this role by altering cognitive beliefs, and reducing stress. An array of protective processes might be identified depending on what the particular risk or stressor is. The table below summarizes major categories of protective factors that researchers have identified (Bogenschneider, 1996). Three Levels of Protective Factors for Children At-Risk Individual Factors Family Factors Temperament Gender, Age, IQ Self Efficacy Social Skills Interpersonal Awareness Empathy Internal locus of control Humor Attractiveness Planfulness & Aspiration

Close relationship with one Person

Extra-Familial Factors Close friend Positive school experiences

Warm, supportive parents Supportive network Parental Harmony Stable care

35

Bonding to family, school, other social networks

The above protective factors identified as relevant to children and youth show a diverse array of resiliencies or strengths that may compensate for stressful and adverse circumstances. They are also a reflection of the focus of risk-resilience research on one part of the picture--the competencies of children over families. Despite some gaps in our knowledge, experts recommend that prevention programs working with families adopt an ecological risk/protective framework to guide the design, delivery and evaluation of prevention programs.(Bogenschneider, 1996). Summary of Best Practice Recommendations for Violence Prevention Programs Type of Program Best Practice Recommendations h Target programs to low-income, first-time mothers. h Have home visits provided by mature and experienced nurses with strong interpersonal skills. h Have home visits take place from pregnancy to two years after birth, with visits every one to two weeks. h Focus on the mother’s well-being, environment and quality of caregiving. h Involve friends, family and community resources. h Limit caseloads to 25 families. h Key References: Olds & h Have a nurse supervisor guide and support the Kitzman (1997), NCPCA nursing staff, and oversee the program. (1997) h Keep comprehensive records on families, needs, services, progress and outcomes. h Link families to formal and informal support. h Parent Education and h Teach parenting skills including alternatives to Support Programs physical punishment. h Teach parents how to identify and reduce stress. h Design programs to reach both a general audience and high-risk parents. h Combine elements of experiential learning (e.g., role playing, discussion groups) with other educational approaches. h Encourage families to participate fully in your programs. h Ensure that your programs are accessible. Consider location, convenience and adequacy of staffing. h Build community support and camaraderie among participants. h Use longer programs, or those with follow-up h Key References: Guterman “boosters,” as these are often more effective. (1997), Lutzker et al. (1998) h Home Visitation

36

h Prevention of Partner Violence

h Screen the level of violence in program families. h Target programs specific to the level of at-risk status. h Address the risk factors for spousal violence. h Include a skills component in the prevention program.

h Key References: Hamby (1998), Kaufman Kantor & Jasinski (1998) h School-Based Partner Violence Prevention

h Key References: AveryLeaf, Cascardi, O’Leary, & Cano (1997), Cano, AveryLeaf, Cascardi, & O’Leary (1998)

h Use a gender-neutral approach to avoid blaming or backlash effects of defensive responding. h Focus on the training and practice of skills required for non-violent conflict resolution (e.g., anger management, assertive communication, active listening). h Focus on changing attitudes accepting of aggression as a conflict tactic. h Use “booster sessions” to maintain program gains. h Supplement program interventions with referrals to peer counseling and other mental health services.

37

Appendix Resources for Developing a Literature Review In addition to the reference provided with this chapter, some excellent sources for beginning a literature review can be found in library and family violence databases, as well as at Web Sites. If you are pressed for time, several resources offer customized search services at no charge or for a nominal fee.

Library Databases: PsychLit, Sociofile, Medline Family Violence References: Clearinghouse on Child Abuse & Neglect Information (NCCAN, 800/FYI-3366) National Victims Resource Center (National Criminal Justice Resource Services) 1-800-627-6872 Fax: 301-251-5212 e-mail: [email protected]

Web Sites:

http://www.apsac.org/index.html

American Professional Society on the Abuse of Children (APSAC)

http://www.childabuse.org/index.html

National Committee to Prevent Child Abuse (NCPCA) National Center on Child Abuse & Neglect (NCCAN)

http://www.acf.dhhs.gov/ACFP/NCCAN/index.html

http://www.tc.umn.edu/nlhome/m206/magd http://www.unh.edu/frl

38

Minnesota Center Against Violence and Abuse Family Research Lab, University of New Hampshire

References

Altemeier, W. A., O'Connor, S., Vietze, P. M., Sandler, H. M., & Sherrod, K. B. (1982). Antecedents of child abuse. Journal of Pediatrics, 100, 823-829. Alvy, K. T. (1994). Parent training today. Studio City, CA: Center for the Improvement of Child Caring. Amaro, H., Fried, L. E., Cabral, H., & Zuckerman, B. (1990). Violence during pregnancy and substance use. American Journal of Public Health, 80, 575-579. Arias, I., Samios, M., & O'Leary, K. D. (1987). Prevalence and correlates of physical aggression during courtship. Journal of Interpersonal Violence, 2, 82-90. Avery-Leaf, S., Cascardi, M., O'Leary, K. D., & Cano, A. (1997). Efficacy of a dating violence prevention program on attitudes justifying aggression. Journal of Adolescent Health, 21, 11-17. Azar, S.T., Fantuzzo, J. & Twentyman, C.T. (1984). An applied behavioral approach to child maltreatment: Back to basics. Advances in Behavioral Research and Therapy. 6, 3-11. Barth, R. P., Hacking, S., & Ash, J. R. (1988). Preventing child abuse: An experimental evaluation of the Child Parent Enrichment Project. Journal of Primary Prevention, 8, 201-217. Belsky, J. (1980). Child maltreatment: An ecological integration. American Psychologist, 35, 320-335. Belsky, J., Rovine, M., & Taylor, D. (1984). The Pennsylvania infant and family development project, III: The origins of individual differences in infant-mother attachment. Child Development, 55, 718-728. Bersani, C. A., & Chen, H. T. (1988). Sociological perspectives in family violence. In V. B. Van Hasselt, R. L. Morrison, A. S. Bellack, & M. Hersen (Eds.), Handbook of family violence (pp. 57-88). New York: Plenum Press. Bogenschneider, K. (1996). Family related prevention programs. Family Relations, 45, 127-138. Bograd, M. (1988). Feminist perspectives on wife abuse: An introduction. In K. Yllo & M. Bograd (Eds.), Feminist perspectives on wife abuse (pp. 11-28). Newbury Park, CA: Sage. Bradbury, T. N., & Fincham, F. D. (1990). Preventing maritial dysfunction:

39

Review and analysis. In F. D. Fincham & T. N. Bradbury (Eds.), The psychology of marriage (pp. 375-401). New York: Guilford. Brems, C., Baldwin, M., & Baxter, S. (1993). Empirical evaluation of a self psychologically oriented parent education program. Family Relations, 42, 26-30. Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513-532. Brown, P. D., & O'Leary, K. D. (1997). Wife abuse in intact couples: A review of couples treatment programs. In G. Kaufman Kantor & J. L. Jasinski (Eds.), Out of the darkness: Contemporary perspectives on family violence (pp. 194-207). Thousand Oaks, CA: Sage Publications. Burch, G., & Mohr, V. (1980). Evaluating a child abuse intervention program. Social Casework, 90-99. Campbell, J., Poland, M., Walder, J., & Ager, J. (1992). Correlates of battering during pregnancy. Research in Nursing and Health, 15(3), 219-226. Cano, A., Avery-Leaf, S., Cascardi, M., & O'Leary, K. D. (1998). Dating violence in two high school samples: Discriminating variables. Journal of Primary Prevention, 18(4), 431-446. Cedar, B., & Levant, R. F. (1991). A meta-analysis of the effects of parent effectiveness. American Journal of Family Therapy, 18, 373-384. Chan, Y. C. (1994). Parenting stress and social support of mothers who physically abuse their children in Hong Kong. Child Abuse & Neglect, 18, 261-269. Committee on the Assessment of Family Violence Interventions (1998). Violence in families: Assessing prevention and treatment programs. Washington, D.C.: National Academy Press. Conger, R.D., Burgess, R.L., & Barrett, C. (1979). Child abuses related to life change and perceptions of illness: Some preliminary findings. Family Coordinator, 28, 73-78. Cowan, P. A., Cowan, C. P., & Schulz, M.S., (1996). Thinking about risk and resilience in families. Hetherington, E.M. & Blechman, E.A., (Eds.) In Stress, coping, and resiliency in children and families. (pp. 1-38), Mahwah, N.J.: Erlbaum. Culp, R. E., Culp, A. M., Soulis, J., & Letts, D. (1989). Self-esteem and depression in abusive, neglecting, and nonmaltreating mothers. Infant Mental Health Journal, 10, 243-251.

40

Dembo, M. H., Sweitzer, M., & Lauritzen, P. (1985). An evaluation of group parent education: Behavioral, P.E.T., and Adlerian programs. Review of Education Research, 55, 155-200. Dinkmeyer, D., & McKay, G. D. (1976). Systematic training for effective parenting: Parents workbook. Circle Pines, MN: American Guidance Service, Inc. Dutton, D. G. (1994). The origin and structure of the abusive personality. Journal of Personality Disorders, 8(3), 181-191. Egan, K.J. (1983). Stress management and child management with abusive parents. Journal of Clinical Child Psychology, 12, 292-299. Feld, S. L., & Straus, M. A. (1989). Escalation and desistance of wife assault in marriage. Criminology, 27, 141-161. Finkelhor, D. (1984). Child sexual abuse. New York: The Free Press. Finkelhor, D. (1986). A sourcebook in child sexual abuse. Beverly Hills: Sage Publications. Finkelhor, D., & Daro, D. (1998). Prevention of child sexual abuse. In M. E. Helfer, R. S. Kempe, & R. D. Krugman (Eds.), The Battered Child (5th ed., pp. 615626). Chicago: University of Chicago. Foshee, V. A., Bauman, K. E., Arriaga, X. B., Helms, R. W., Koch, G. G., & Linder, G. F. (1998). An evaluation of Safe Dates, an adolescent dating violence prevention program. American Journal of Public Health, 88(1), 45-50. Foshee, V. A., Linder, G. F., Bauman, K. E., Langwick, S. A., Arriaga, X. B., Health, J. L., McMahon, P. M., & Bangdiwala, S. (1997). The Safe Dates project: Theoretical basis, evaluation design, and selected baseline findings. American Journal of Preventive Medicine, 12 (Suppl.), 39-47. Frodi, A. M., & Lamb, M. E. (1980). Child abusers' responses to infant smiles and cries. Child Development, 51, 238-241. Garbarino, J. (1982). Children and families in the social environment. New York: Aldine. Garbarino, J. & Courter, K. (1978). Defending the community context of parentchild relations: The correlates of child maltreatment. Child Development, 49, 604-616. Gelles, R. J. (1973). Child abuse as psychopathology: A sociological critique and reformulation. American Journal of Orthopsychiatry, 43(4), 611-621.

41

Gelles, R. J., & Straus, M. A. (1979). Determinants of violence in the family: Towards a theoretical integration. In W. R. Burr, R. Hill, F. I. Nye, & I. L. Reiss (Eds.), Contemporary theories about the family (Vol. 1). New York: Free Press. Gondolf, E. W. (1988). Who are those guys? Toward a behavioral typology of batterers. Violence and Victims, 3(3), 187-203. Graziano, A. M., & Diament, D. M. (1992). Parent behavioral training: An examination of the paradigm. Behavior Modification, 16, 3-38. Guterman, N. B. (1997). Early prevention of physical child abuse and neglect: Existing evidence and future directions. Child Maltreatment, 2(1), 12-34. Hamberger, L. K., & Hastings, J. E. (1986). Characteristics of spouse abusers: Predictors of treatment acceptance. Journal of Interpersonal Violence, 1(3), 363-373. Hamby, S. L. (1998). Partner violence: Prevention and intervention. In J. L. Jasinski & L. M. Williams (Eds.), Partner violence: A comprehensive review of 20 years of research (pp. 210-258). Thousand Oaks, CA: Sage Publications. Hansen, D. J., Pallotta, G. M., Tishelman, A. C., Conaway, L. P., & MacMillan, V. M. (1989). Parental problem-solving skills and child behavior problems: A comparison of physically abusive, neglectful, clinic, and community families. Journal of Family Violence, 4, 353-368. Holtzworth-Munroe, A. (1994). Typologies of male batterers: Three subtypes and the differences among them. Psychological Bulletin, 116, 476-497. Holtzworth-Munroe, A. (1997). Violent versus nonviolent husbands: Differences in attachment patterns, dependency, and jealousy. Journal of Family Psychology, 11, 314-331. Holtzworth-Munroe, A., Markman, H., O'Leary, K. D., Neidig, P., Leber, D., Heyman, R. E., Hulbert, D., & Smutzler, N. (1995). The need for marital violence prevention efforts: A behavioral-cognitive secondary prevention program for engaged and newly married couples. Applied & Preventive Psychology, 4, 77-88. Hudson, W. W., & McIntosh, S. R. (1981). The assessment of spouse abuse: Two quantifiable dimensions. Journal of Marriage and the Family, 43, 873-885. Jaffe, P. G., Sudermann, M., & Reitzel, D. (1992). Child witnesses of marital violence. In R. T. Ammerman & M. Hersen (Eds.), Assessment of family violence: a clinical and legal sourcebook . New York: John Wiley & Sons.

42

Jaffe, P. G., Sudermann, M., Reitzel, D., & Killip, S. M. (1992). An evaluation of a secondary school primary prevention program on violence in intimate relationships. Violence & Victims, 7, 129-146. Jones, L. E. (1987). School curriculum project evaluation report. Minneapolis: Minnesota Coalition for Battered Women. Kalmuss, D. (1984). The intergenerational transmission of marital aggression. Journal of Marriage and the Family, 46, 11-19. Kaufman, J., & Zigler, E. (1993). The intergenerational transmission of child abuse. In D. Cicchetti & V. Carlson (Eds.), The handbook of child maltreatment . Cambridge: Cambridge University Press. Kaufman Kantor, G., & Asdigian, N. (1996). When women are under the influence: Does drinking or drug use by women provoke beatings by men? In M. Galanter (Ed.), Recent developments in alcoholism (Vol. 13). New York: Plenum. Kaufman Kantor, G., Jasinski, J., & Aldarondo, E. (1994). Sociocultural status and incidence of marital violence in Hispanic families. Violence and Victims, 9(3), 207222. Kaufman Kantor, G., & Jasinski, J. L. (1998). Dynamics and risk factors in partner violence. In J. L. Jasinski & L. M. Williams (Eds.), Partner violence (pp. 1-43). Thousand Oaks, CA: SAGE Publications. Kaufman Kantor, G., & Straus, M. A. (1987). The "drunken bum" theory of wife beating. Social Problems, 34(3), 213-230. Kaufman Kantor, G., & Straus, M. A. (1989). Substance abuse as a precipitant of wife abuse victimization. American Journal of Drug and Alcohol Abuse, 15, 173-189. Kelleher, K., Chaffin, M., Hollenberg, J., & Fischer, E. (1994). Alcohol and drug disorders among physically abusive and neglectful parents in a community-based sample. American Journal of Public Health, 84, 1586-1590. Kitzman, H., Olds, D. L., Henderson, C. R., Hanks, C., Cole, R., Tatelbaum, R., McConnochie, K. M., Sidora, K., Luckey, D., Shaver, D., Engelhardt, K., James, D., & Barnard, K. (1997). Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing: A randomized controlled trial. JAMA, 278(8), 644-652. Klinman, D. G. (1986). Fathers and the educational system. In M. E. Lamb (Ed.), The father's role: Applied perspectives (pp. 413-428). New York: Wiley Publications. Krajewski, S. S., Rybarik, M. F., Dosch, M. F., & Gilmore, G. D. (1997). Results

43

of a curriculum intervention with seventh graders regarding violence in relationships. Journal of Family Violence, 11, 93-112. Lavoie, F., Vezina, L., Piche, C., & Boivin, M. (1995). Evaluation of a prevention program for violence in teen dating relationships. Journal of Interpersonal Violence, 10(4), 516-524. Levy, B. (1984). Skills for violence-free relationships . Santa Monica, CA: Southern California Coalition on Battered Women. Lifur-Bennet, L. (1982). The effects of an Adlerian and a behavioral parenteducation program on learning disabled children and their parents. Unpublished Ph.D. Dissertation, California School of Professional Psychology, Los Angeles. Lutzker, J.R., Bigelow, K.M., Doctor, R.M., Gershater, R.M., & Greene, B.F. (1998). An ecobehavioral model for the prevention and treatment of child abuse and neglect. In J.R. Lutzker (Ed.) Handbook of child abuse research and treatment. (pp. 239-266). New York: Plenum. Lutzker, J. R., Wesch, D., & Rice, J. M. (1984). A review of Project 12 Ways: An ecobehavioral approach to the treatment and prevention of child abuse and neglect. Advances in Behavioral Research and Therapy, 6, 63-73. Makepeace, J. M. (1986). Gender differences in courtship violence victimization. Family Relations, 35, 383-388. Markman, H.J., Renick, M.J., Floyd, F.J., Stanley, S.M., & Clements, M. (1993). Preventing marital distress through communication and conflict management training: A 4- and 5-year follow-up. Journal of Consulting and Clinical Psychology, 61(1), 70-77. Markman, H. J., & Floyd, F. (1980). Possibilities for the prevention of marital discord: A behavioral perspective. American Journal of Family Therapy, 8, 29-48. McBride, B. A. (1991). Parental support programs and paternal stress: An exploratory study. Early Childhood Research Quarterly, 6, 137-149. McLaughlin, I. G., Leonnard, K. E., & Senchak, M. (1992). Prevalence and distribution of premarital aggression among couples applying for a marriage license. Journal of Family Violence, 4, 309-319. Miller, S. L. (1994). Expanding the boundaries: Toward a more inclusive and integrated study of intimate violence. Violence and Victims, 9(2), 183-194. Milner, J. S., & Dopke, C. A. (1997). Child physical abuse: Review of offender characteristics. In R. Peters (Ed.), Child abuse: New directions in prevention and treatment across the life span (pp. 25-52). Thousand Oaks, CA: Sage Publications.

44

Milner, J. S., & Wimberly, R. C. (1979). An inventory for the identification of child abusers. Journal of Clinical Psychology, 35, 95-100. National Center on Child Abuse and Neglect. (1983a). Collaborative research of community and minority group action to prevent child abuse and neglect. Volume I: Perinatal Interventions. Part II. Perinatal positive parenting. Washington, DC: US Department of Health and Human Services. National Center on Child Abuse and Neglect. (1983b). Collaborative research of community group action to prevent child abuse and neglect: Pride in parenthood. Washington, DC: US Department of Health and Human Services. National Committee to Prevent Child Abuse (1997). Healthy Families America Credentialing Self-Assessment Tool. Chicago: Author. Neidig, P. H. (1986). The development and evaluation of a spouse abuse treatment program in a military setting. Evaluation and Program Planning, 9, 275-280. Neidig, P. H. (1989). Stop anger and violence escalation (SAVE) instructor's guide. Beaufort, S.C.: Behavioral Science Associates. Neidig, P. H., & Friedman, D. H. (1984). Spouse abuse: A treatment program for couples. Champaign, IL: Research Press. O'Keefe, M. K., Brockopp, K., & Chew, E. (1986). Teen dating violence. Social Work, 31, 463-468. Olds, D., Hill, P., & Rumsey, E. (1998, November). Prenatal and early childhood nurse home visitation. OJJDP Juvenile Justice Bulletin, 1-7. Olds, D.L., & Kitzman, H. (1993). Review of research on home visiting for pregnant women and parents of young children. The Future of Children, 3, 53-92. Olds, D. L., & Henderson, C. R., Jr. (1989). The prevention of maltreatment. In D. Cicchetti & V. Carlson (Eds.), Child maltreatment: Theory and research on the causes and consequences of child abuse and neglect (pp. 722-763). Cambridge: Cambridge University. O'Leary, K. D. (1988). Physical aggression between spouses: A social learning theory perspective. In V. B. Van Hasselt, R. L. Morrison, A. S. Bellack, & M. Hersen (Eds.), Handbook of family violence (pp. 31-56). New York: Plenum. O'Leary, K. D., Barling, J., Arias, I., Rosenbaum, A., Malone, J., & Tyree, A. (1989). Prevalence and stability of physical aggression between spouses: A longitudinal analysis. Journal of Consulting and Clinical Psychology, 57, 263-268.

45

O'Leary, K. D., Vivian, D., & Malone, J. (1992). Assessment of physical aggression against women in marriage: The need for multi-modal assessment. Behavioral Assessment, 14, 5-14. Pagelow, M. D. (1984). Family violence. New York: Praeger. Peterson, J., & Hawley, D. R. (1998). Effects of stressors on parenting attitudes and family functioning in a primary prevention program. Family Relations, 47, 221-227. Polansky, N. A., Chalmers, M. A., Williams, D. P., & Buttenweiser, E. W. (1981). Damaged parents: An anatomy of child neglect. Chicago: University of Chicago. Renzetti, C. M. (1994). On dancing with a bear: Reflections on some of the current debates among domestic violence theorists. Violence & Victims, 9(2), 195-200. Robertson, S. E. (1984). Parent education: Current status. Canadian Counselor, 18, 100-105. Roy, M. (1977). Battered women: A psychosociological study of domestic violence. New York: Van Nostrand, Reinhold. Rutter, M. (1979) Protective factors in children’s responses to stress and disadvantage. In M.W. Kent and J.E. Rolf (Eds.) Primary prevention of psychopathology, Vol. 3 (pp. 49-79) Hanover, NH: University Press of New England. Rynerson, B.C. & Fishel, A.H. (1993). Domestic violence prevention training: Participant characteristics and treatment outcomes. Journal of Family Violence, 8, 253256. Schroeder, C. S., & Gordon, B. N. (1991). Assessment and treatment of childhood problems. New York: Guilford. Shorkey, C. T., & Armendariz, J. (1985). Personal worth, self-esteem, anomia, hostility, and irrational thinking in abusing mothers: A multivariate approach. Journal of Clinical Psychology, 41, 414-421. Smith, M. D. (1990). Sociodemographic risk factors in wife abuse: Results from a survey of Toronto women. Canadian Journal of Sociology, 15(1), 39-58. Spinetta, J. J., & Rigler, D. (1972). The child abusing parent: A psychological review. Psychological Bulletin, 77, 296-304. Straus, M.A. & Kaufman Kantor, G. (1994). Corporal punishment by parents: A risk factor in the epidemiology of depression, suicide, alcohol abuse, child abuse and wife beating. Adolescence, 29; 543-561.

46

Straus, M. A. (1973). A general systems theory approach to a theory of violence between family members. Social Science Information, 12, 105. Straus, M. A., Gelles, R. J., & Steinmetz, S. (1980). Behind closed doors: Violence in the American family. Garden City, NJ: Anchor Press. Susman, E. J., Trickett, P. K., Iannotti, R. J., Hollenbeck, B. E., & Zahn-Waxler, C. (1985). Child-rearing patterns in depressed, abusive, and normal mothers. American Journal of Orthopsychiatry, 55, 237-251. Swift, C. (1986). Preventing family violence: Family-focused programs. In M. Lystaol (Ed.), Violence in the home: Interdisciplinary perspectives (pp. 219-249). New York: Brunner/Mazel. Thompson, R. W., Grow, C. R., Ruma, P. R., Daly, D. L., & Burke, R. V. (1993). Parent education: Evaluation of a practical parenting programs with middle- and lowincome families. Family Relations, 42, 21-25. Tzeng, O. C. S., Jackson, J. W., & Karlson, H. C. (1991). Theories of Child Abuse and Neglect: Differential Perspectives, Summaries, and Evaluations. New York: Praeger. Wesch, D., & Lutzker, J. (1991). A comprehensive 5-year evaluation of Project 12 Ways: An ecobehavioral program for treating and preventing child abuse and neglect. Journal of Family Violence, 6, 17-35. Wiehi, V. R. (1987). Empathy and locus of control in child abusers. Journal of Social Service Research, 9, 17-30. Wolak, J., & Finkelhor, D. (1998). Children exposed to partner violence. In J. L. Jasinski & L. M. Williams (Eds.), Partner violence: A comprehensive review of 20 years of research (pp. 73-112). Thousand Oaks, CA: Sage Publications. Wolock, I., & Horowitz, B. (1984). Child maltreatment as a social problem: The neglect of neglect. American Journal of Orthopsychiatry, 54, 530-545. Yllo, K. (1984). The status of women, marital equality, and violence against wives. Journal of Family Issues, 5(3), 307-320.

47

How To Use This Manual In this guide, we discuss the basics of program evaluation specific to the United States Air Force Family Advocacy Program (FAP). You may be called upon to evaluate many different programs. If you are new to evaluation, it may seem overwhelming. This manual has been designed to guide you through the process. Our goal is to provide you with sufficient information to conduct evaluations in your own programs or participate in an evaluation conducted by others. Throughout this manual, we use examples from the evaluation conducted by the Family Research Laboratory (FRL) at the University of New Hampshire, and the Child and Family Research Group, San Diego. This team used a multi-faceted and dynamic approach in evaluating the USAF Family Advocacy prevention programs, with the ultimate goal of improving services offered to families. In approaching an evaluation, one of your first considerations must be the type of program you are being asked to evaluate. As you know, the Family Advocacy Program (FAP) offers a wide array of violence-prevention programs. These vary substantially in complexity and intensity of service, but can be grouped into four major categories: •

Off-The-Shelf Programs. These are programs that were developed elsewhere and were purchased for implementation (or possible implementation) with Air Force families. Examples of these types of programs include pre-packaged curricula for parent education, marriage enrichment, or stress management. These types of programs are generally developed with civilians in mind. They may or may not be appropriate for a military population.

•

Community-Focus Programs. These are programs aimed at changing attitudes within the community. These may include public service announcements, programs and activities (“week of the military child”), and classes directed at the general public. The focus of these activities is creating awareness of family violence and promoting positive family functioning. These programs may also involve community members in designing programs and taking action on behalf of the community. The goal here is to enhance the social networks of community members and connect them with helping agencies.

•

Field-initiated Programs. These are programs that may have developed at a single site, designed to meet a specific community need. For example, if 48

families in off-base housing live in a neighborhood with a high crime rate, these families might need to establish a neighborhood watch program. This program might be very specific to families at that one base. Evaluation of programs developed to meet a specific community need can cover two issues: the program’s effectiveness, and whether this program can be used by other sites. •

In-depth Prevention Programs. These prevention programs are for families designated as “at-risk,” and have more intensive services than the programs described above. It is possible that you will not evaluate these programs, but will hire outside contractors to evaluate them. This manual provides sufficient information for you to actively participate. Three recently-evaluated FAP family violence prevention programs are used as examples: First Time Parents (FTP), a new-parent home-based support program; Home-based Opportunities Make Everyone Successful (H.O.M.E.S.), a home-based, multi-disciplinary program to promote mission readiness and reduce incidence of child maltreatment; and Prevention and Relationship Enhancement Program (PREP), a marital enrichment program and part of the FAP maltreatment and family-strengthening initiative. Although we frequently use these programs as examples, the material we present can be used to evaluate other programs as well.

Another factor to consider is your unit of analysis. Are you trying to find out something about the program itself or do you want to know about the clients or community it serves? Too often, evaluators tend to focus on client/community outcomes to the exclusion of other important factors. While client or community outcomes are certainly key to understanding the effectiveness of a program, their outcomes are by no means the only way to understand and assess a program. Indeed, clients/communities are the unit of analysis for only one section (Section 5: Assessing Program Impact), where outcomes are the focus. Different aspects of the program itself are the focus of the other chapters. Section 1 deals with the program maturity and whether it is ready to be evaluated. Section 2 helps you focus on the rationale underlying the program. Is there a program model? Is the program model evidence-based? Section 3 has to do with the implementation of the actual program elements. How well does the actual program match the model developed in Section 2? Is the program running smoothly and delivering quality services? Section 4 focuses on the efficiency of the program in terms of financial resources. How much does it cost per participant? What is the cost of participating vs. the cost of not participating? Sections 6 and 7 will help you develop or gather the tools you need for both process and outcome evaluations. If you are in the beginning phases of your project, the section on Logic Models might be the most helpful. If you have concerns about staffing and client satisfaction, you might find Process Evaluation most helpful. If you are considering a full-scale evaluation, but don’t know where to begin, you might find that the section on Evaluability Assessments is most helpful.

49

In this guide, seven different aspects of program evaluation are described. Below is a brief summary of each section. The manual can be read straight through, or on an as-needed basis. Each Section is designed to stand alone, but will also refer you to pertinent information in other sections. We have provided a summary below to help you find the information you need quickly. At the beginning of each section, we have also provided a “key points” box that summarizes the most important concepts. These boxes can provide further information about whether a particular section is relevant. Section 1: Evaluability Assessment In this section, we help you determine whether a program or individual site is ready for a formal evaluation. Indications of readiness, potential pitfalls and the components of evaluability assessment are discussed. Section 2: Logic Model Development What is the theoretical rationale for your program? In this section, we guide you through the steps of logic model development. A logic model will help you make explicit links between your program goals and your specific program activities, or perhaps highlight areas where this information is still needed. Section 3: Process Evaluation Is your program delivering quality services? Are they consistent with the program model? Process evaluation will help you determine this, and provide a means of on-going program monitoring. Section 4: Cost Analysis How much does your program cost per participant? How cost-effective is it? How do you conduct a cost-benefit analysis? In an era of limited resources and funding cutbacks, we must keep our eye on the bottom line. Section 5: Assessing Program Impact What type of design should you use for an outcome evaluation of your program? What is the scope of your evaluation? When is it appropriate to use quantitative methods? Or qualitative methods? The advantages and disadvantages of each approach are summarized in this section, and three qualitative approaches are described. Section 6: Designing Data Collection Instruments Now that you’ve decided on a design, can you use existing data or do you need to collect new data? Coding data from existing records is described. In addition, surveys and interviews are described in this section. Section 7: Outcome Measurement What are appropriate benchmarks for your programs? How should outcomes be measured? This section provides detailed information on how to measure short- and long- term goals. Criteria for measurement selection are presented and domains of measurement are listed. Measures appropriate to USAF FAP prevention programs are 50

provided, including measures of family well-being, parenting, stress, and family conflict and violence. Suggestions for measures of other key family constructs are provided. The measurement package selected for the USAF FAP is presented and discussed along with a rationale for the selection and development of measures.

51

Section 1 Evaluability Assessment

Key Points ¾ Evaluability assessments (EAs) take an overall look at programs to determine whether they are ready to evaluate. ¾ EAs can save time and money by helping you identify problems early, or by preventing premature impact evaluations. EAs will help you determine: whether your program is mature enough to evaluate, the program is functioning as intended, and you can measure outcomes. ¾ One of the first tasks is to determine who will be using the information you gather, and what type of information they need. ¾ You must determine what the target population is, what the essential program activities are, and how program goals link to program activities. ¾ You can determine whether any of the information you need is already available by reviewing the empirical literature, reviewing existing documentation, and conducting site visits, but you may also need to collect additional data. ¾ Typically, the EA phase lasts an average of six to eight months.

Evaluability assessments (EA) are the preparation phase for a program evaluation, and take place before any other formal evaluation activities (Wholey, 1979). Evaluability assessments can save time and money by helping you identify problems early, or by preventing premature impact evaluations. For example, an impact evaluation would be premature if the program model was still undergoing substantial change. In the course of conducting evaluability assessments, evaluators examine all aspects of a program in order to discover whether: •

The program is mature enough to evaluate,

•

The program is functioning as intended, and

•

You can measure outcomes.

Evaluability assessments (EA) enable you to construct a comprehensive description of your program and determine its level of maturity and stability (Smith, 1990). Rather than thinking of EA as a process which determines the success or failure of a program based on its evaluability status, you can approach EA as a way to define 52

the type and scope of evaluation activities. Typically, the EA phase lasts an average of six to eight months (Wholey, 1979). Serious investment in the evaluability assessment phase can prevent wasted resources on impact evaluations that are impossible to carry out, or yield inconclusive results due to program immaturity or poor program implementation. The evaluability assessment should help you decide whether evaluation of the formative type (process) or the outcome type (impact) is appropriate. You will know when it is too early to conduct a formal impact evaluation by watching for these potential pitfalls (Wholey, 1979). •

Lack of agreed-upon and/or measurable objectives.

•

Lack of plausible logic linking intended program activities to intended program outcomes.

•

Discrepancies between actual program activities and those specified in the standards or program model.

•

Inadequate or missing indicators of program performance.

•

Lack of structural support for program implementation (e.g., not enough staff to deliver program services).

Below we describe specific steps for conducting EAs of your programs. We provide lists of questions and issues to consider during an EA. These lists are meant to help you weigh all the information you collect, and make decisions on the future direction of your evaluation.

How to Conduct Evaluability Assessments Evaluability assessments (EAs) can be broken down into six steps. The order in which you execute these steps may vary. While it is important to complete all of these steps, the depth of your investigation may vary as a function of available resources and the type of information that you are trying to collect. Evaluability assessments of community-based or primary prevention programs will generally be less involved than intervention programs that offer intensive services for families at risk. Step 1: Determine the Audience for Your Evaluation One of the first tasks in conducting an evaluability assessment is to determine who will be using the information you gather, and what type of information they need. Because there are usually diverse audiences for evaluation findings, each may be interested in different aspects of your program. Below are some questions to help you determine the audience for your evaluation. 53

•

Who are the stakeholders?

Stakeholders are individuals with a stake (vested interest) in both the program and the results of the evaluation. Stakeholders include a wide variety of people: sponsors of the evaluation, program managers and staff, program clients and their families, other agencies working with the program, interest groups, elected officials, and the general public (Worthen, Sanders, & Fitzpatrick, 1997). In the U.S. Air Force, stakeholders of note include members of the military command structure such as first sergeants and commanders, and senior medical leaders. Example: Community Website One proposed project sought to establish a website with information about available resources on a wide range of subjects including domestic violence, child abuse, stress management, and parenting information. The goal of this program is community empowerment, and increasing the safety net of resources available to families. Some stakeholders could include the AFMOA/SGOF liaison, the base commander, the developers of the website, representatives of the community organizations, and local families.

•

Who will use the results of the evaluation?

Among the stakeholders, only a few may be the actual audience for your evaluation results. Determining who will actually use the results will help determine what information they need. Knowing who will use the results will also help you prepare written summaries of your findings. Many stakeholders have the predominant role of advisor. They are not involved in the day-to-day activities of the program. These stakeholders will most likely need only an executive summary of your findings. A few of your stakeholders, however, will play a more active role. These stakeholders will probably require a complete report of findings as they concern themselves with the hands-on aspects of your program. •

Why do the sponsors of an evaluation want one? What are they trying to accomplish?

As Worthen and colleagues describe (Worthen, et al., 1997), decisions to evaluate stem from someone’s need to know. You must determine what types of information they need, and what they plan to do with this information. •

Should representatives from other groups be interviewed while planning the evaluation?

Another important planning step is to determine whether members from any 54

other groups need to be interviewed. While it may not be possible to interview “everyone,” it is frequently helpful to gather information from those outside the program who work with it on a regular basis, or who may be affected if changes in the program might result from the outcome of the evaluation. Examples include first sergeants, who rely on the services provided by the program, and Family Support, who might be called to fill in the “gaps”. As in the example of the community website (see above), community agencies can also be included. The results of an EA may be used “in-house” to help in program development. They may also be used in progress reports to stakeholders and funding agencies, and later in articles describing program development. An evaluability assessment can also facilitate communication between evaluators and multiple stakeholders (Worthen, et al., 1997). For an evaluability assessment to be most effective, stakeholder interests must be an integral part of the process. When stakeholder interests are well integrated into the assessment and evaluation, your chances of reaching consensus on mutually beneficial evaluation tasks and options should be greatly enhanced. In turn, this process should increase the overall usefulness of the evaluation (Rossi & Freeman, 1993). Furthermore, discussing results of the EA with stakeholders affords them the opportunity to be engaged in follow-up decisions regarding results of the assessment. This would include all stakeholders in appropriate ways, from clients to program staff, administrators, support services, higher base command and policy makers. Below is an example of how comments and concerns of stakeholders were integrated into the evaluation of FAP programs. Application: USAF Program Example The perspectives of multiple stakeholders have been incorporated into every stage of the Family Research Laboratory (FRL), University of New Hampshire evaluation project. The project was launched by conducting focus groups with field staff from all three USAF Family Advocacy prevention programs selected for EA. The goal of the focus groups was to get a sense of FAP staff perceptions of their program models and procedures, and of barriers and challenges in implementing the First Time Parents (FTP), H.O.M.E.S., and outreach programs. Next, the FRL evaluators reviewed relevant FAP documents, and then conducted interviews with key informants in order to gain an understanding of FAP Division leadership views about Family Advocacy and the mission and goals of FAP. The FRL evaluators also interviewed program administrators and staff during base site visits. (Also see, Kaufman Kantor, et al., 1998; Evaluability Assessment Report, Section I). During the implementation phase of the evaluation, the evaluation team expanded its contacts to a wider range of important stakeholders including fathers, whose wives were the primary recipients of services from the First Time Parents Program, and military leadership such as commanders from various squadrons, first sergeants, chief nurses, and a USAF chaplain (also a recipient of FAP services). A USAF-wide survey of FAP teams was administered as one of the last Phase I

55

EA activities conducted by the FRL evaluators. This was done because field staff are key stakeholders and their input was considered very important. The base survey was administered at the end of Phase I so that survey questions could be grounded by information gathered from FAP staff during earlier activities (focus groups, keyinformant interviews, and site visits). The survey elicited more detailed information from staff members concerning their perspectives on the goals, barriers, risk assessment and outcome measurement strategies utilized at the bases. In addition, target group information was developed by asking staff to describe family profiles characteristic of client groups currently utilizing each program (Kaufman Kantor, Landsverk, DeVoe, Finkelhor, & Straus, 1999). Finally, input from consultant partners was facilitated by a feedback system which included written and e-mail communication as well as periodic meetings with the entire evaluation team. Step 2: Select an Evaluator One of the first decisions that you have to make is who will conduct the evaluation. Should you ask current program staff (including you) to evaluate the program, or should you hire an outside evaluation consultant? Your choice depends on the needs of the program and the objectives of the evaluation. A short-term evaluation may be best handled internally while an evaluation of large programs might require the assistance of outside evaluators. It is one of the many facets of your program that you must make decisions about. We have listed some of the advantages and disadvantages of external vs. internal evaluators below. •

External Evaluators: An external evaluator is someone who is not an employee of the program under evaluation. Advantages •

Objectivity: An external evaluator may be less likely to be influenced by knowledge of a program’s history, operation, and objectives at the outset of evaluation. External evaluators may also be more likely to be free of personal interest in the findings, increasing their objectivity. However, many internal evaluators will also be able to function objectively.

•

Expertise in a Particular Field: External evaluators come from a variety of fields and may have expertise that members of the in-house staff do not. However, you might find that internal evaluators also have the necessary expertise.

•

Staff May Be More Honest with the External Evaluator: Sometimes program staff divide into “warring” factions. In this situation, an outsider may be able to speak to all of these groups without taking sides in the dispute. On the other hand, an internal evaluator who has established relationships with the staff may also be able to accomplish this.

56

•

Better Able to Present Unpopular Information: Because external evaluators have some distance from the program, it might be easier for them to present negative or unpopular information. On the other hand, an internal evaluator who knows and has a good working relationship with the staff may be even more effective in this role.

•

More Credibility with High-Level Stakeholders: An external evaluator may have more credibility with high-level stakeholders (e.g., Department of Defense, Congress), particularly if the evaluator is someone with stature in his or her field, and the evaluation is a rigorous, scientific one.

Disadvantages

•

•

May Need More Start-Up Time: Because external evaluators may be unfamiliar with a program, they are likely to need more start-up time. However, even an inside evaluator may encounter this is they are relatively new to the program or at a distant site.

•

Outsider to the Organization: External evaluators may have difficulties gaining staff trust and cooperation during the evaluation process. Similarly, a new or disliked internal evaluator may encounter the same problem.

•

Increased Cost: Evaluations by outsiders frequently cost more because of travel expenses and greater lead time, but an experienced evaluator may be cheaper in the long run since they may need less time at the site than someone new to evaluation recruited from among the staff. However, an internal evaluator who is responsible for multiple sites may also have to travel and require more lead time.

Internal Evaluators: An internal evaluator is a staff member of the program under evaluation, or the agency in which the program is housed. For example, a program manager may function as an internal evaluator. Advantages •

More Familiar with Stakeholder Interests: Internal evaluators may have a better grasp of stakeholders’ needs. However, a skilled external evaluator may also be able to gather this information relatively quickly. A study of internal and external evaluations in the Netherlands suggests that there is a higher rate of utilization of internal researchers’ findings in evaluation studies. This may be due to higher rates of communication between internal researchers and program policy makers (Rossi & Freeman, 1993). Internal evaluators may be better able to use evaluation results to effect actual change in a program because of their familiarity with the program and its administration. 57

•

Increased Efficiency: Internal evaluators may know more about the program and its staff, history, operation, and objectives which means less time spent becoming acquainted with the program during the evaluability assessment and evaluation. However, an internal evaluator who is new or not familiar with a particular program, may not have this advantage.

•

Better Rapport with Members of the Program/Organization: Internal evaluators may be better able to use evaluation results to effect actual change in a program because of their familiarity with the program and its administration. On the other hand, program staff may resist changes initiated from among their ranks, and may be more willing to “hear” these suggestions from an outsider.

Disadvantages •

Reduced Objectivity: Internal evaluators concerned about threats to their jobs due to evaluation results may not be able to conduct an objective evaluation. This includes susceptibility to pressure from other program or organizational staff. However, external evaluators are not immune from these pressures, especially if they fear that their contract will be terminated if the findings are not to the liking of persons funding the evaluation.

•

Insufficient Time: If staff or program managers are expected to conduct evaluations in addition to other job responsibilities, then either daily responsibilities or the evaluation tasks may be shortchanged. Resource considerations should include a pragmatic calculation of staff availability, and staff burden.

The advantages and disadvantages of internal and external evaluators listed above are not absolutes; they are merely suggestions for issues to consider in deciding whom to choose to conduct an evaluation. All evaluators, internal and external, must monitor themselves for objectivity throughout the evaluation process. Ultimately, the decision about who should evaluate your programs is up to you, but it is important to be aware of the costs and benefits of both approaches. Decision making about the appropriate selection of an evaluator is a prelude to the conduct of an Evaluability Assessment (EA). Step 3: Get an Overview of Your Program Once you have a sense of your audience and their needs, you can begin to learn more about the program you want to evaluate. Step 2 focuses on a general overall view. You will continue to learn about your program through Steps 4 and 5. In considering the program, you must determine what the essential program activities are, the mechanism or theory in which the program is grounded, and how 58

program goals link to program activities. One method of getting the big picture is to develop a logic model or graphic representation of linkages between program objectives and outcomes, which is described in detail in Section 2 (“Specify the Prevention Mechanisms”). You must determine whether the entire program or only a portion of it is to be included in the evaluation. Further, you need to know the settings in which the evaluation is conducted (e.g., clinic, home, or other community sites). Another practical consideration is the availability of resources for the evaluation (Worthen, et al., 1997). The EA conducted by the FRL identified “lack of resources” as one of the staffperceived barriers to program implementation of FAP prevention programs. If program staff members are already over-burdened by their daily responsibilities, they may be much less willing to participate in an evaluation, or cooperate with evaluators. Below are two issues that you might address with regard to conducting an EA: •

Are there sufficient personnel and funding to conduct an evaluation?

•

Are the deadlines reasonable? Is there sufficient time to conduct a thorough evaluation?

It is important to continue an EA until you have enough information to make sound decisions about the evaluation. If you haven’t been able to answer the important questions about the plausibility of various evaluation options, keep going until those questions are answered (Wholey, 1979). •

What is the program trying to accomplish? Are the goals and objectives explicitly spelled out? Are they plausible?

•

Is there a program blueprint?

•

How is this operationalized? Are program activities occurring as planned?

•

Has the program been around long enough so that operational glitches have been resolved?

•

Is there a mechanism for receiving rapid feedback on program components that aren’t working?

59

Below is an example from the FRL EA specifying the sources of information they used to get an overview of the programs they were evaluating. Program Information Type • Program goals and objectives

•

Program implementation

•

Program model

FRL Information Sources • Meetings with FAP Division personnel; key informant interviews; FAP documents; site visits; mailed survey • Key informant interviews; focus groups with FAP staff; FAP documents; site visits; mailed survey • Literature review; FAP documents; site visits; mailed survey

If a program blueprint exists, one way to detect areas that need improvement is to compare the program blueprint (intended program model) with the actual program model (Pecora, Seelig, Zirps, & Davis, 1996). During the EA, you should explore the blueprint as well as actual program activities to determine whether the program is operating as planned. When stakeholders involved in the evaluation reach consensus regarding the actual and intended program models (e.g., program goals, activities, and outcomes), you have reached an important milestone. When the actual program deviates from its blueprint model, program improvements can be made, and/or the underlying model can be modified. Thus, EA results can be used to build and refine program models and program implementation so that there are clear relationships between program resources, activities, and expected outcomes (Wholey, 1987). In the following example, a program blueprint and program activities were compared during EA conducted by FRL. Application: USAF Program Example The assessment of the FAP Prevention and Relationship Enhancement Program (PREP) couples communication program (Holtzworth-Munroe, Markman, O'Leary, Neidig, Leber, Heyman, Hulburt, & Smutzler, 1995; Markman, Renick, Floyd, Stanley, & Clements, 1993a) indicated that the program as implemented deviated significantly from the PREP blueprint. The PREP model was designed to target couples in early phase relationships who had not yet engaged in destructive types of conflicts. In actuality, at the sites visited, and sites from which critical focus-group members were drawn, the majority of clients were attending PREP as part of their post-batterer or post-anger management group treatment plan (i.e., tertiary level prevention). In addition, one key informant indicated a belief that couples early in the cycle of domestic violence were prime candidates for PREP. A critical issue for FAP is to decide whether this program is the most appropriate choice for USAF couples as a prevention strategy to deter further spousal violence, especially since PREP has not been previously evaluated with couples who have established patterns of physical conflict. 60

Step 4: Determine Types and Amounts of Data Presently Available In conducting an evaluation, it is wise to consider whether any of the information you need is already available. There are several ways for you to do this: reviewing the empirical literature, reviewing existing documentation, and conducting site visits. Each of these steps is reviewed below. • Review relevant literature At some point early in the process, you will want to familiarize yourself with the literature relevant to the program you are evaluating. The literature will help you: •

Evaluate whether the program is grounded theoretically.

•

Assess measures and outcomes, and their appropriateness.

•

Get up-to-date information on the latest evaluation efforts.

Example: Literature Review In preparing a literature review on family violence prevention, there are several different topics to consider. These include the following: •models of prevention evaluation, •child maltreatment prevention, •home visitation, •child and family measurement, •community strengths, and •partner violence prevention. Two examples of how this literature might pertain to a current evaluation are described below. •Results from the home visitation literature indicated the importance of service intensity to program outcomes (Olds & Kitzman, 1993) •Reviews of successful child abuse prevention approaches underscored the essential nature of parenting education support, providing health education, and providing linkages to formal and informal supports in effective prevention models (Guterman, 1997).

61

• Review program documentation You might be able to use information that already exists in your evaluation. Reviewing program documentation is an important early step. You can start by collecting and reviewing documents which describe the program mission, goals, objectives, and intended outcomes (Smith, 1989). You want information and background on the program, and information about individual clients and families. Descriptive information on a program may come from administrative regulations, funding proposals, administrative manuals, annual reports, reports from prior evaluations, or those of similar programs in the same organization. Try to unearth any previous evaluations or reports of the program. If none are available, find out if field staff or program administrators have conducted informal surveys of client groups or received client satisfaction feedback. Since the purpose of the EA is to obtain real insight into the program theory and practice, it is useful to look for consistent themes and patterns in data sources. Below is an example of sources and types of information the FRL team led by Kaufman Kantor gathered from existing records.

Example: Use of Program Documentation The FRL team reviewed program documentation for First Time Parents, H.O.M.E.S., and couples communications programs. First, a number of program documents were requested from division staff, including reports, briefings, previous evaluations, publications, program manuals and standards. Second, data from FAP Division on prevention programs were obtained and analyzed. Third, documentation at the base level was obtained during site visits. Base staff provided intake and referral forms, hand-outs used in the FTP and Outreach programs, instructional materials, community information forms, risk assessment check lists and other measures. (Also see, Kaufman Kantor, et al., 1998; Evaluability Assessment Report, Summary & Conclusions, Section II; & Section IV.)

From your review of the records, you have learned about the history of the program, how long it’s been in existence, changes that may have taken place, and its staffing patterns and interactions with other organizations. Some of this information may become important in the evaluation. For example, suppose that an element of an intervention program was dropped due to funding constraints. You may have enough information available to determine whether this missing element has affected clients. For families who completed classes in parenting, or stress and anger management, there may be pre- and post-class measures of these variables. Other types of program information could include articulation of program goals, description of curriculum, and whether program providers can document how many classes each family attended (an indication of “dose”). For families in more intense secondary-prevention programs, data available 62

could include prevention files, intake records, and scores or profiles on standardized instruments. The quality of this information can range from poor to excellent. As was the case for the USAF FTP and H.O.M.E.S. programs, information about basic demographics (e.g., age, marital status, sex, number of children, and ethnicity) is helpful in describing the client population. You may have measures taken at intake that can be used for comparison upon completion of the program (e.g., the Conflict Tactics Scale, Family Needs Assessment Screener). You may also be able to use case reports to document and code “intensity of service,” by noting how many times clients had contact with program staff. If documentation on families is available, you need to determine if it will be useful to include in the evaluation. When considering these types of data, you will want to consider several issues. •

Do the records appear to cover the variables of interest?

•

Has the information been collected in a non-biased way (e.g., with all eligible program families vs. only with more compliant families)?

• Are there records for most of the families in the program? •

Are the existing records complete or is there a great deal of missing information?

• Conduct Site Visits If the EA is not conducted in-house or by internal staff, site visits allow the evaluator to observe first-hand how the program operates. You then are able to integrate your observations from site visits with documentation you have received about the program and with interviews to develop an accurate description of the program (Rossi & Freeman, 1993). During your site visits, interview program personnel about their perspectives on program goals, objectives and implementation (Rossi & Freeman, 1993). In order to maximize the usefulness of your site visits, try to interview a variety of people including key staff, board members, clients, and stakeholders. You can also observe family contacts and home visits. Site visits frequently have a “best-foot-forward” quality to them. To moderate the latter, you will want to allow enough time to observe more typical operations, and conduct individual interviews, in addition to the special activities and meetings that can occur during site visits. Shadowing multiple staff across repeated key activities is an important means to overcome the tendency not to discuss difficulties with site visitors. Multiple visits and observations at different program sites give the truest picture of the program. Also one might assume that problems exist in a program (or the provider) 63

when problems are highly visible or if staff present questionable interventions as their “best practice.” Your visits will also give you an opportunity to get a sense of the political climate at the site and whether personnel will cooperate with (or resent) the evaluation. Since many sites may be involved in your program, you may need to visit a variety to fully understand how your program is working in the field. In selecting bases to visit, you should strive for diversity of geography, mission and major command, and program maturity. Many differences in program implementation will be apparent at various sites. Below is an example of how bases were chosen by FRL to participate in the EA. Example: Selection of Locations for Site Visits The FRL team conducted five site visits during the Phase I of the evaluation project. Base selection was made in conjunction with AFMOA FAP division staff. The collection of bases selected reflected the diversity of geography, mission and major command, and program maturity across the USAF. Bases with more high-risk families (based on CAP scores) were chosen. In addition, there were logistic considerations; the evaluators chose bases that were relatively close to where they lived. The visits involved extensive interviews with key prevention staff, and observations of home visits for H.O.M.E.S. and FTP programs. These visits enhanced the team’s understanding of all three Family Advocacy Programs and supplied data critical to the evaluability assessment (Also see Kaufman Kantor, et al., 1998; Evaluability Assessment Report, Section II).

Step 5: Construct Plan for Gathering Still-Needed Data Once you have determined what information is already available, the next step is to develop a list of information you still need, and the best way to collect it. This is relevant not only to the EA, but also to future process and outcome evaluations. Step 5 includes lists of questions about five aspects of your program. In the boxes, are illustrations about how these types of questions might pertain to your program. You need to decide how much of this information is relevant to you, as well as any additional information you want to include. Questions about Clients These questions help determine who are the actual and intended program participants (refer also to Section 3, “Target Population,” of this guide). •

Who is the target audience of your the program?

64

•

Is the program designed to have community-wide penetration or do families enroll?

•

How do families come to participate?

•

Do participants differ in systematic ways from non-participants?

One purpose of an EA is to determine whether the program is reaching its target population. In conducting an evaluation, you need to first determine whether there is agreement about who is in this population. All first-time parents? Only families who are identified as being at-risk? Only families of enlisted personnel vs. families of officers? Second, you need to determine whether prospective clients are being approached in a systematic way. Are there specified inclusion and exclusion criteria? Is there a plan for client recruitment (e.g., all families with young children, all first births that occur at the base during a specified recruitment period)? Finally, you might consider whether a systematic plan exists for handling refusals. Is there a protocol for “passive refusals” (i.e., those who never return telephone call or other attempts to contact)? How many failed attempts to contact are necessary before the family is excluded? What are the demographic characteristics of refusals and do they differ from study participants? If such data are unavailable then this type of monitoring might be considered as a type of process evaluation to be undertaken. This analysis would permit you to adjust your recruitment techniques if you determine that you are failing to reach an important sub-group of your target population. Questions about Program Model Questions in this section inform you about intended program model or blueprint including goals, objectives, activities and linkages among them. In order to develop evaluation questions for both process and outcome evaluations, program goals and objectives must have been identified. In the evaluation context, the terms “goal” and “objective” are not always clearly differentiated, and are often used interchangeably. We, however, make a distinction between these terms as follows: Goal: A general statement summarizing what you hope to accomplish by implementing your program. Program goals encompass cognitive, behavioral and attitudinal changes in clients (Herman, Morris, & Fitz-Gibbon, 1987). A program may have multiple goals, and each goal is likely to have multiple objectives. Objective: Objectives are the operationalization of goals, and the intended measurable results that clients strive to achieve. Objectives are more specific 65

than goals, and they describe strategies to accomplish goals. Objectives are often stated in terms of increase/decrease by a specific number or percent within a given time. For example: “Objective is to reduce child maltreatment reports by 30% in the next 5 years.” •

What is the program intended to do for the people it serves?

•

Have formal goals and objectives been identified?

•

What are they?

•

Are there clearly defined linkages between the program’s theoretical framework, activities, and objectives?

Example: Review of FAP Prevention Program Goals & Objectives The FRL team’s analysis of the H.O.M.E.S. program did not find clear linkages between theory, activities and objectives. Objectives were far-reaching given a low intensity of services (e.g., decrease incidence of negative teenage behaviors: suicide, chemical dependency, physical fighting, arrests, runaways, truancy, assaults, and related health contacts). In addition, there was lack of agreement in the field that the primary goal of H.O.M.E.S. is prevention of child and spouse maltreatment. Objectives for the FTP Program were clearly stated (though not in numerical terms), consistent with practice, and potentially achievable (e.g., “increase parental knowledge”, “decrease abuse potential”, “decrease levels of stress”).

Questions about Process Process questions include program activities and services, as well as intensity and length of services. Process evaluation is also the focus of Section 3. •

What types of activities are there?

•

What is the schedule of activities?

•

Are the actual program activities the same as those specified initially in the program plan?

•

Is there a prescribed number of contacts with families? Are the projected goals being met?

•

What is the length of intervention period for each program?

66

•

What are the perceived barriers and challenges to intervention that are faced by staff?

During a process evaluation, you evaluate the actual delivery of services. What you are concerned about is that families are receiving roughly the same treatment. Issues to consider include: •

How are assessment tools being used?

•

Are many forms incomplete? If incomplete forms are a problem, perhaps a meeting with field personnel is in order to make the paperwork more userfriendly.

•

Is the program realistic in terms of the number of contacts that must take place?

•

Is there a procedure for handling missed visits?

•

Are there mechanisms for tracking “dose” effects due to different levels of participation?

•

Do field personnel have a clear understanding of what to do and how to administer the prescribed program in a consistent way?

•

Are there regular staff trainings and spot-checks on inter-rater reliability? These questions are important in early EA, and can head off trouble. They will also become part of an on-going process evaluation (discussed in detail in Section 3, “Monitoring Delivery of Services”).

Example: Evaluating a Parenting Class As part of your primary-prevention effort, you may decide to offer one or more parenting classes for all parents of infants and/or young children. A process evaluation of a primary-prevention program is similar to that of a secondaryprevention program. You would want to know who this program is reaching and whether the curriculum is being delivered consistently. You would need to know that this program is being marketed well, what the specific learning objectives are, and if the course that is being evaluated matches the model. Perhaps most importantly, you want to know whether an already developed (“canned”) course translates well to a military population. When sites modify a program, these modifications could indicate that the program, as originally developed, did not translate well to a military population (see Section 3, “Program activities”).

67

Questions about the Organization of Services In this section, we list questions about organizational structure of the program and program variability across sites. •

Where are the services provided?

•

Who provides the services?

•

How large is the staff and is there adequate coverage?

•

How is the program resourced? By FAP exclusively? By Integrated Delivery System (IDS) partners?

•

How is the program administered?

• What is the organization hierarchy? • How do programs vary across sites in implementation of the model? • At what point are families becoming involved in the programs?

Example: Organization of Services Questions under this heading are perhaps the most straightforward. Much of this information can be gathered from existing records, and is related to assessment of inter-site differences. Differences between sites are almost inevitable, but good data collection at this point will help explain why these differences exist. One crucial aspect is whether there is adequate staffing to deliver the intended program. You may encounter a situation where field workers are assigned too many cases to adequately provide the services specified in the design (e.g., a certain number of home visits). In this situation, more staff may need to be hired or the model may have to be modified. Also, it is important to determine whether staff are supported in other ways including adequate training, and feedback when they encounter situations that call for extra care. For example, is there a mechanism for group case-management or are all decisions made from the top down?

Questions about Outcomes Outcome questions include intended program effects, observable program effects and available measures of programs effects (See also Section 5, “Evaluation methodology.”)

68

•

Is the program’s treatment clearly identifiable and consistent?

•

Are the outcomes clear, specific and measurable?

•

What types of outcome measures are being used across programs and bases?

•

What is the availability of outcome data?

At this early phase, the goal is to make sure that outcomes can be assessed. Are there good theoretical reasons for the selection of variables? Does everyone agree on what those important variables are? What steps were taken to limit bias in data collection? Are data on critical variables even being collected? Below is an example of some limitations the FRL team identified during the EA of the FTP program.

Example: Outcomes The FRL EA team identified some limitations in the collection of outcome measures used for FTP families. For example, many sites were not collecting post-program Child Abuse Potential (CAP) assessments. Because military families are very mobile, on quite short notice, many of the families were lost to follow-up: an issue to be addressed during a process evaluation. In addition, there were no measures of family strengths or family gains. Further details are found in the EA report of this program.

Step 6: Making Sense of Your EA Once you have gathered all the information described above, you create a formal program model description. This allows you to compare the intended program with the program that is actually being used. Furthermore, the program description assists you in determining whether the program is likely to achieve its goals and objectives as stated. You now need to make a decision about whether this program is ready for a formal evaluation. Worthen and colleagues (Worthen et al., 1997) have described four steps that will help you determine whether or not you should proceed with your evaluation. •

Clarify the theory or model that underlies your program.

This is something that has been described previously and will also be described in Section 2 (“Identify program goals, objectives and outcomes”). But before you proceed to the next step, you should have a firm grasp of what the model is and why it is supposed to work. •

Examine the program currently in use, and determine whether it matches the 69

model. In addition, you need to determine whether it is plausible for the elements of the program to achieve the specified goals. Are the program objectives well defined? Does your program match its model? Could the activities of the program reasonably be expected to help clients attain the program goals? •

Consider whether the proposed evaluation approaches are feasible and will meet the information needs of the stakeholders.

Is the proposed evaluation going to be too burdensome or difficult for the staff to implement? Have the stakeholders reached some type of consensus on the information they need and will the proposed evaluation meet that need? Are the intended uses for the evaluation results well defined? Example: Influence of Stakeholder Information Needs on FRL EA Decisions The FRL team’s recommendations for the implementation phase of the evaluation took into account the needs of key stakeholders for information, and their priorities. FRL weighed these needs and priorities along with their findings on program status in the decision-making process. For example, interviews with key FAP Division stakeholders revealed unanimity on a need to know whether prevention programs were working, and whether they were cost-effective. Key stakeholders indicated that there was a need to be able to show some return on the long-term financial investment in USAF family maltreatment prevention programs. FRL considered these interests along with the feasibility of being able to demonstrate dependably the benefits of FAP programs to families.

•

Determine the evaluation priorities and how the information will be used.

Who will receive a copy of the evaluation findings? Will this information be used to implement changes in the program? In personnel? How much input will stakeholders have as to the uses of the study? Several factors will indicate when it is inappropriate to proceed with an evaluation. The EA may reveal that the stakeholders cannot agree on the program model, or an evaluation plan. The program itself may differ significantly from its original model. The goals of the program may not be consistent with the current program actions. Or there may be simply not enough resources to evaluate the program (Worthen, et al., 1997). There may be some additional factors that may also indicate that the program is not ready to be evaluated. For example, the program administrator or stakeholders may have already made a decision about a program that will not be changed with evaluation results. In this case, an evaluation would waste everyone’s time and is not worth 70

pursuing. Another possible scenario is that the sponsors of the evaluation may be asking for something that is unrealistic in terms of both scale and timeline. For example, they may ask for results within the first month, which would not give you sufficient time for data collection. Any data you do collect on such short notice is likely to misrepresent the program you plan to evaluate. These would be legitimate reasons for not going forward with an evaluation (Worthen, et al., 1997). In Figure 1 at the end of this section , we have provided you with a worksheet to help you assemble all the information you have collected and decide what to do next. This worksheet refers you to other sections of this manual for additional information. It is designed to help you pinpoint which parts of your program may need to be corrected before an evaluation can proceed. Example: When do you know that you have enough information to proceed with decision-making? As a general rule, you know that you have enough information when you are able to answer the key questions with a fair amount of confidence. One simple rule is that the time is right when you have closure on your major questions. By closure, we mean that the same answers to questions emerge consistently across multiple observations or sources of data. For example, the First Time Parent Program was viewed affirmatively across all sources of information. Concerns over stigmatization as a barrier to measurement and assessment were also a constant. Data obtained from the AFMOA FAP Division on families, data collected from a mailed survey, and interviews and observations at site visits revealed consistent profiles of families for FTP, and a consistent picture of what H.O.M.E.S. families were like. Interviews conducted at site visits, and analysis of FAP databases provided a consistent picture of targeting, assessment and service delivery patterns. At the close of six months, the evaluation team felt ready to draw conclusions on prevention program status in the USAF.

Making Decisions: Process or Outcome Evaluation? Once you have your EA findings in hand, and feel that sufficient information has been obtained, you must decide on the next step. Assuming you have decided to move forward on an evaluation, you need to decide if you want to pursue a process or outcome evaluation, or some combination of both. If your program has an ongoing education or treatment component, you may also want to start a process evaluation to monitor delivery of services and ensure that information is collected in a systematic way. If outcome evaluation is your ultimate goal, you have to determine whether your program is sufficiently mature, and whether the outcomes you are interested in can be measured in some reliable and meaningful way. Hopefully, the target program has been planned with well-defined and clearly measurable objectives which have been agreed to by program planners and administration. One should also be able to identify intended 71

activities designed to meet the objectives, and causal hypotheses linking means and ends. Example: Evaluability Assessment of the First Time Parents Program The results of the First Time Parents program indicated that it is, in fact, evaluable. The Phase I EA of FTP showed that it is mature, has heavy investment by stakeholders, and its program model content is well-articulated. Therefore, it is possible to conduct an evaluation of FTP which explores possible relationships between program activities, outcomes and other factors such as client and environmental characteristics. If you discover that the program you want to evaluate needs improvements in its design or implementation, or that the program does not operate as it was designed, it is best to postpone an outcome evaluation. If you attempt to conduct an outcome evaluation while your program is still establishing stability, and before it has had time to accomplish longer-term objectives, you will not give the program a fair chance to demonstrate the desired outcome (Smith, 1990). In any of these circumstances, a process evaluation may be more appropriate and can provide useful feedback to improve your program. Example: Evaluability Assessment of H.O.M.E.S. The evaluability assessment findings of the FAP H.O.M.E.S. program strongly indicated that a process evaluation would be most beneficial. A number of concerns surfaced during site visits and consultations with the AFMOA FAP Division and base staff. Some of the problems included insufficiently specific guidelines for the target population, the types of services to be delivered and the major goals of the program. There were personnel issues concerning the roles of nurses and social workers (and areas where their roles overlapped). Further, the program significantly differed in implementation from its original model. The FRL staff recommended these concerns be addressed and that the model be changed to reflect the actual program being used.

Summary In this section, we have discussed the purposes and products of evaluability assessment, and outlined the steps involved in the EA process. We have presented a variety of questions to help you design your own evaluability assessment, and have presented EA as a part of the evaluation process and not something that determines program success or failure in its own right. In Section 2, we discuss the program logic model as a tool used in evaluation. We delineate the steps for building a logic model, and describe how to use the logic model in evaluation work. 72

Figure 1 Evaluability Assessment Worksheet Answer the following questions about each area of your program. One or more “No’s” in each area may indicate a problem. Questions about Target Yes Population • Has a target population been specified? • Is the program reaching its target population? • Are participants similar to non-participants?

No

Decision Criteria Problems in this area indicate that your program is not yet ready for an outcome evaluation. A process evaluation, however, could be helpful. Review Section 2 (“Specify the target population”) for help in defining your target population, and Section 3 (“Recruitment”) for help in reaching them and monitoring your success.

Questions about the Program Model • Have formal goals and objectives been specified? • Is there consensus among stakeholders about the objectives of the program? • Are there clearly defined linkages between the program’s theoretical framework, activities and objectives? Questions about Process • Are program activities clearly specified? • Is there a specific schedule of activities? • Is there consistency among providers? • Is the program in the field consistent with the program model? • Are projected goals being met?

This area may also mean that your program is not yet ready for an outcome evaluation. Refer to Section 2 (“List services and activities”).

A process evaluation would be very helpful with process questions, and may indicate areas in which your program could improve. It is important to address these issues before proceeding to an outcome evaluation. See Section 3 (“Monitoring delivery of services”).

73

Questions about Organization of Services • Is there adequate staff coverage? • Does the program have sufficient resources? • Are the qualifications for providers specified? • Is the organizational hierarchy and administration of the program clear? • Does implementation of the program vary across sites? Questions about Outcomes • Is the program’s treatment clearly identifiable and consistent? • Are the outcomes clear, specific and measurable? • Are there outcome data available? Questions about Stakeholders • Do you know who the stakeholders are and what type of information they need? • Do you know who will use the results of the study and how they will use them?

A program will not work well without the supporting infrastructure. Programs with these types of issues may appear ineffective in an outcome evaluation. Better to address these concerns and make any needed program or documentation changes before proceeding.

These issues also need to be addressed before proceeding to an outcome evaluation. Refer to Section 2 (“Identify program goals, objectives and outcomes”) for information on how to define outcomes that are measurable.

Before gathering any information, you must consider the needs of the enduser of this information. Refer to Section 1 (“Step 1: Determine the audience for your evaluation”) for information on how to determine the needs of stakeholders.

74

Key References Pecora, P.J., Seeling, W.R., Zirps, F.A., & Davis, S.M. (1996). Quality improvement and evaluation in child and family services: Managing into the next century. Washington, DC: CWLA Press. Rossi, P.H., & Freeman, H.E. (1993). Evaluation: A systematic approach. (5th Ed.). Newbury Park, CA: Sage. Smith, M.F. (1989). Evaluability assessment: A practical approach. Boston: Kluwer Academic. Smith, M.F. (1990). Evaluability assessment: Reflections on the process. Evaluation and Program Planning, 13, 359-364. Worthen, B.R., Sanders, J.R., Fitzpatrick, J.L. (1997). Program evaluation: Alternative approaches and practical guidelines. (2nd Ed.). New York: Longman.

References CSR Incorporated. (1996). Preventing child abuse and neglect: A case study of I care (Contract No. 105-92-1808). Washington, DC: U.S. Department of Health and Human Services. Fink, A. (1993). Evaluation Fundamentals: Guiding Health Programs, Research, and Policy. Newbury Park, CA: Sage. Guterman, N. B. (1997). Early prevention of physical child abuse and neglect: Existing evidence and future directions. Child Maltreatment, 2(1), 12-34. nd

(2

Herman, J. L., Morris, L. L., & Fitz-Gibbon, C. T. (1987). Evaluator's Handbook. Ed.). Newbury Park, CA: Sage.

Holtzworth-Munroe, A., Markman, H., O'Leary, K. D., Neidig, P., Leber, D., Heyman, R. E., Hulbert, D., & Smutzler, N. (1995). The need for marital violence prevention efforts: A behavioral-cognitive secondary prevention program for engaged and newly married couples. Applied & Preventive Psychology, 4, 77-88.

75

Joint Committee on Standards for Educational Evaluation (1994). The Program Evaluation Standards. (2nd Ed.). Thousand Oaks, CA: Sage. Kaufman Kantor, G., Landsverk, J., DeVoe, E., Finkehor, D., & Straus, M.A. (1999) Evaluability Assessment Report: The USAF Family Advocacy Program Family Violence Prevention Initiative. Durham, NH: University of New Hampshire Family Research Laboratory. Markman, H., J., Renick, M. J., Floyd, F., J., Stanley, S., M., & Clements, M. (1993a). Preventing marital distress through communication and conflict management training: A 4- and 5-year follow-up. Journal of Consulting and Clinical Psychology, 61(1), 70-77. Markman, H. J., Blumberg, S. L., Stanley, S. M., Leber, D., Neidig, P. H., & Holtzworth-Munroe, A. (1993b). PREP/SAVE Consultant's Manual : PREP Educational Products, Inc. Behavioral Sciences Associates, Inc. Olds, D. L., & Kitzman, H. (1993). Review of research on home visiting for pregnant women and parents of young children. The Future of Children, 3 (3), 54-92. Patton, M. Q. (1987). How to use qualitative methods in evaluation. (2nd Ed.). Newbury Park, CA: Sage. Pecora, P. J., Seelig, W. R., Zirps, F. A., & Davis, S. M. (1996). Quality Improvement and Evaluation in Child and family Services: Managing Into the Next Century. Washington, DC: CWLA Press. Rossi, P. H., & Freeman, H. E. (1993). Evaluation a systematic approach. (5th Ed.). Newbury Park, CA: Sage. Smith, M. F. (1989). Evaluability assessment: A practical approach. Boston: Kluwer Academic. Smith, M. F. (1990). Evaluability Assessment: Reflections on the process. Evaluation and program planning, 13, 359-364. Stecher, B. M., & Davis, W. A. (1987). How to focus an evaluation. (2nd Ed.). Newbury Park, CA: Sage. Weiss, H. B., & Jacobs, F. H. (Eds.). (1988). Evaluating family programs. New York: Aldine De Gruyter. Wholey, J. S. (1979). Evaluation: Promise and Performance. Washington, DC: The Urban Institute. Wholey, J. S. (1987). Evaluability Assessment: Developing program theory. New

76

Directions for Program Evaluation, 33, 77-92. Worthen, B. R., Sanders, J.R., Fitzpatrick, J.L. (1997). Program Evaluation: Alternative Approaches and Practical Guidelines. (2nd ed.). New York: Longman.

77

Section 2 Logic Model Development

Key Points ¾ Logic models are graphical representations of the theory underlying a program. They help you identify program objectives and activities, and the linkages between them. ¾ Logic models can be used for both primary and secondary prevention programs. They can be helpful not only in the beginning phases of an evaluation, but also in developing a program. ¾ To develop a logic model, you must identify desired program outcomes, specify program activities, and specify characteristics of the population you want your program to reach. In this section, we describe logic models, what they are and how to build them. A logic model is a graphic representation of the theory that underlies a program’s goals. Logic models are used in identifying program objectives, activities, and expected outcomes, and finding the linkages between them. They can be used in both program development and the early phase of program evaluation. Throughout this section, we refer to three figures. Figure 1 is an example of a logic model that was developed for an existing prevention program: the New Parent Support Program. The graphic model was developed primarily on the basis of program documentation, and then elaborated on by program staff. You might find that logic models are most useful for these complex, multi-step programs with outcomes that are measured at several different points. However, logic models can also be used in primary prevention programs. In Figure 2, we provide a preliminary and hypothetical logic model that could be applied to a primary prevention program, such as a class for parents of young children. While it is not as complex as Figure 1, Figure 2 demonstrates that logic models are useful even with less complex programs. Indeed, they are often quite useful in planning stages when trying to operationalize program goals and activities. Figure 3 is a blank model that you can photocopy and use to describe your own program. For each of the major components, we provide sample worksheets that are designed to help you identify program specifics. There are copies of these in the appendix of this section so that you can photocopy them and use them while developing a model. The worksheets provide more detail than you will actually need for your graphic model. But this level of detail will help you conceptualize your program more fully. Once you have summarized this information on your worksheets, you can transfer it to the blank logic model in Figure 3. 78

Figure 1 New Parent Support Program- Logic Model Population Served

Prenatal 0 - 3 One Parent Eligible for Military Care No Open or Pending Maltreatment Case

Service Delivery Standards: Individual Practice Objectives Evidence-based Activities

Program Elements Provider: Professional / ParaProfessional Family Screening Individualized Plans Home Visits Provider Client Relationship Community Linkages Education Assessment, Referral, and Management Multi-disciplinary Team Approach

Intermediate Outcomes

Knowledge of Child Growth & Development Enhance Family Member Role Adaptation Problem Solving Skills Potential for Maltreatment

Proximal Outcomes Physical, Sexual, and Emotional Child Abuse and Neglect Partner/ Spouse Abuse Positive Child Growth and Development Community Resilience

Distal Outcomes Mission Readiness Build Healthy C iti

79

Figure 2 First Time Parent’s Program Elements for a Program Logic Model Target Population

Program Characteristics

Intermediate Outcomes

Eligibility:

Service Delivery

•Birth/adoption of a child is expected, or recently occurred

•Family Advocacy Nurse delivers services

Reduce Family Maltreatment & Preserve Families

•One of the parents is elligible for military medical care

•Nursing assessment and diagnosis provided

•No open maltreatment cases •No allegation of maltreatment pending

Home Visits

Population Characteristics:

•Maternal, prenatal, postpartum issues

•New Parents identified at-risk for maltreatment

(Primary Service Modality): Focus: Education, Supp’t & Guidance

•Newborn, child behavior, care, growth & development •Family healthrelated behavior & practices

&Increase parentchild attachment behaviors and emotional bonding &Increase healthy parenting behaviors &Increase perception of newborn behaviors, child growth and development