Development and Initial Evaluation of a Cost-effective, Internet-based Program to Assist Professionals in Reporting Suspected Child Maltreatment Maureen C. Kenny, Angel M. LopezGriman & Brad Donohue
Journal of Child & Adolescent Trauma ISSN 1936-1521 Journ Child Adol Trauma DOI 10.1007/s40653-016-0110-3
1 23
Your article is protected by copyright and all rights are held exclusively by Springer International Publishing. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com”.
1 23
Author's personal copy Journ Child Adol Trauma DOI 10.1007/s40653-016-0110-3
ORIGINAL ARTICLE
Development and Initial Evaluation of a Cost-effective, Internet-based Program to Assist Professionals in Reporting Suspected Child Maltreatment Maureen C. Kenny 1,2 & Angel M. Lopez-Griman 3 & Brad Donohue 4
# Springer International Publishing 2016
Abstract Many professionals in the U.S. are deficient in the identification and federally-mandated reporting of suspected child maltreatment, and there is a dearth of evidencesupported training programs in child maltreatment reporting. The Internet offers great promise for delivering such training because it is cost-effective and permits a vast audience. In this feasibility study, we describe the development and initial evaluation of an Internet-based training module specific to the identification and reporting of suspected child maltreatment. Participants who completed this training program gained a greater knowledge of child maltreatment reporting practices and were able to identify signs and symptoms of child abuse better than participants in a control condition (p < .001). There was also a significant change in participants’ self-reported knowledge (p < .001) after taking the tutorial. Finally, qualitative analyses indicated that participants reported that their experience with the training program was very positive. The results of this preliminary study suggest positive training outcomes in child maltreatment reporting can be achieved in a relatively short amount of time and without substantial cost. This study was funded by Florida International University Division of Information Technology (Grant number 09–047) to Maureen C. Kenny. * Maureen C. Kenny
[email protected]
1
Department of Leadership and Professional Studies, Florida International University, Miami, FL, USA
2
College of Arts, Sciences and Education, Florida International University, 11200 SW 8th Street, ZEB 245A, Miami, FL 33199, USA
3
The Village South, Miami, FL, USA
4
Department of Psychology, University of Nevada, Las Vegas, NV, USA
The examined training program may offer educational administrators in pre-professional child service delivery systems a method of training future professionals to manage child maltreatment. The study results also provide a base in which investigators can enhance training in child maltreatment reporting. Keywords Child maltreatment training . Internet . Child abuse reporting . Training module . Knowledge . Skills . Reporting behavior More than 3.6 million referrals for child maltreatment were made in 2014 to child protective service agents in the United States (U.S. Department of Health & Human Services, Administration of Children and Families 2016). This finding is particularly impressive when considering the vast majority of child maltreatment incidents are not reported to child protective service agencies (Putnam 2006). Alvarez et al. (2004) suggested that professionals often fail to report child abuse due to a lack of training in identification of the signs and symptoms of child maltreatment, and scarcity of empiricallysupported training programs to assist in the process of making child maltreatment reports. Indeed, the development of training programs in this area is desperately needed (Gilbert et al. 2009). In the United States, professionals, including educators, physicians, and mental health providers, are mandated to report suspected child maltreatment to the appropriate authorities (U. S. Department of Health & Human Services, Administration of Children & Families Administration on Children, Youth and Families 2016). However, there remains no national standard governing the amount and content of child abuse training required by mandated professionals. Post-secondary educational settings, such as colleges, nursing
Author's personal copy Journ Child Adol Trauma
and medical schools, may be optimal environments to assist in the development of child maltreatment reporting skills because students in these programs are likely to experience professional situations in which they are legally-mandated to report suspected child maltreatment during their clinical experiences and after their graduation (Mathews 2011). Further, studies indicate collegiate students desire training in child maltreatment reporting (Goldman and Grimbeek 2015). However, professional development curricula at universities rarely include such training, which increases risk of child abuse victimization by leaving victims unreported (Champion et al. 2003).
Need for Training Internet Training As the Internet becomes more accessible, computer-based training programs are increasingly utilized to teach various skill-sets and have been evaluated as good or better than traditional class lectures (see meta-analysis by Sitzmann et al. 2006). For instance, Sitzmann et al. (2006) found Internet training to be slightly more effective in teaching declarative knowledge (i.e., facts), and as good as face-to-face training in teaching procedural knowledge (e.g., skills). Indeed, the use of the Internet to assist professionals in understanding and appropriately responding to child maltreatment scenarios offers many advantages over face-to-face lecture-oriented curriculums (e.g., easy to update programming, standardized formatting, capable of reaching a considerable audience). Educational institutions, which are increasingly using Internet based education, provide an ideal backdrop in which to teach burgeoning professionals to effectively manage child maltreatment situations. However, training in mandated child maltreatment reporting appears to be grossly absent or lacking in university settings (Arnold and Maio-Taddeo 2007; Rossato and Brackenridge 2009).' Advantages of Internet Training There are several advantages of utilizing Internet-based training. In agencies in which there is a need to train many professionals and there is a high staff turnover rate, the use of the Internet allows for ongoing training. Further, in academic institutions in which faculty may lack expertise in the area of child maltreatment, Internet training provides student access to the necessary content related to child maltreatment. The flexibility of allowing participants to complete training at their own pace and location is another benefit. In addition, given that a full curriculum is often cited as a reason to omit information on child maltreatment (Baginsky and Macpherson 2005; Crettenden and Zerk 2012), utilizing time outside the classroom would enable faculty to focus on other concepts in face-to-face discussions.
Finally, an Internet- based system simplifies storage of data/responses and, thus, evaluation can be performed to optimize the effectiveness of the program. Internet-based training eliminates many logistical issues of cost, location, and staffing problems, although it does require high front-end development costs and specially-trained content developers (Coppola and Myre 2002). Using the Internet for Child Maltreatment Training To date, the Internet has been neglected as a training tool for child abuse reporting, but it has been used to teach other aspects of child maltreatment. For instance, Paranal et al. (2012) evaluated the Internet version of Stewards of Children, a childhood sexual abuse (CSA) prevention program. Participants rated the training format favorably and reported that a training facilitator was unnecessary. However, the investigators noted several program development needs (e.g., be more user friendly, fix registration issues). Rheingold et al. (2012) examined the feasibility and acceptability of the same program for child care professionals provided in two formats (i.e., in-person, Internet). While both formats were determined to be feasible to implement and acceptable to professionals, the in-person format was judged by participants to be more comfortable and to lead to a greater likelihood of sharing the learned information with others. Furthering this research, Rheingold et al. (2015) conducted an independent multi-site controlled evaluation of the same program. Results indicated that the Stewards of Children program improved CSA knowledge, attitudes, and preventive behaviors in participants. However, no statistical differences were found between training modalities (i.e., in-person versus Internet) on knowledge and preventive behaviors, suggesting that these modes of training were equally effective. In a randomized control trial, Delaney et al. (2012) compared in-person child abuse training with a comparable Internet-based training program for prospective foster parents, and found that the Internet-based online training was more effective than live training at increasing child maltreatment knowledge. There was also a trend for the Internet training group to be more empathic (toward birth parents) than the in-person group. Participant feedback indicated high satisfaction with the Internet program. While both Rheingold et al. (2012) and Delaney et al. (2012) showed improved participant satisfaction with the Internet training approaches to abuse prevention, neither examined the effects of the training on child maltreatment reporting behavior. The School Community Professional Development Act (Florida Statute Section 1012.98(12)) mandates that the Department of Education requires teachers in grades K-12 to participate in continuing education training provided by the Department of Children and Families on identifying and reporting child abuse and neglect. The 1 h Internet-based training targets school personnel and focuses on identifying
Author's personal copy Journ Child Adol Trauma
and reporting child abuse; it is housed on the Department of Children and Families website (the child protection agency in Florida). Despite the necessity of requiring this for education professionals, there is no known evaluation of this training. In an uncontrolled evaluation involving 105 participants, Kenny (2007) found that future professionals who participated in an Internet-based child maltreatment reporting training improved their knowledge regarding signs and symptoms of abuse as well as reporting procedures. This study provided some preliminary evidence specific to the feasibility and consumer satisfaction of an Internet program focused on child abuse identification and reporting. However, more outcome studies are needed to demonstrate improvements in child maltreatment reporting behavior subsequent to Internet-based training. Rationale for Current Study The purpose of this study was to augment the aforementioned Internet training program that was originated in Kenny (2007), while improving the study methodology to incorporate a control group and a psychometrically-validated outcome measure. The primary changes to the training content included updating the statistics on child abuse to include the most current available, and incorporating a 67-min video entitled, BMandated Reporting^ (Kid Safety of America 2012; e.g., interviews with victims and child protection workers, examples demonstrating how to call the hotline). Also, the training program was converted to enhanced software in order to better assist participant viewing. It was hypothesized that participants who were assigned to the experimental child maltreatment reporting condition as compared with participants who were assigned to a control condition from pre to posttraining would evidence (a) significantly greater improvements in knowledge of child maltreatment types and symptoms; and (b) significantly greater improvement in knowledge of reporting procedures and laws related to child maltreatment. This study also employed mixed-methods research in order to more fully understand the results of the training and use both quantitative (test scores) and qualitative (participant feedback) data.
Method Participants Over 1000 students (N = 1690) in a large, urban, Hispanic Serving Institution (HSI), with a mean age of 22.52 years (SD = 5.01) participated in the study. A majority (77 %) of the students were female. About level of education, 21 % (n = 357) were freshmen, 16 % (n = 272) were sophomores, 38 % (n = 645) were juniors, 23 % (n = 390) were seniors,
and 2 % (n = 26) were graduate students. Self-reported ethnicity was 71 % Hispanic (n = 1199), 12 % White Non-Hispanic (n = 202), 8 % African American (n = 135), 3 % African Caribbean (n = 50), 3 % Asian (n = 50), and 3 % reported being Bother^ (n = 54). On average, participants reported spending 17.48 h per week on the computer (with a range of .5 to 170 h). Table 1 shows demographic characteristics of the study sample. Procedure The university Institutional Review Board (IRB) gave approval for the study prior to data collection. The study occurred across 24 months in two academic years (2011–2013) at the HSI. Participants were recruited through several courses at the university. They were notified about the study through emails, course announcements and, correspondence from professors. Potential participants were informed of the nature and purpose of the study as well as the expected time commitment. Participants were informed of the training (i.e., given the URL) and were able to complete the on-line training at any point during the semester. The training was self-paced and could be returned to at any time or completed in multiple sittings. However, the tests required completion during a single log on. All participants completed a consent form which was the first question on the pretest of the tutorial, prior to beginning the training. No adverse events were reported to occur. Only participants that completed both the pre and posttest were included in the analysis so that there were no missing test results. However, there were variable rates of responding to the Training Satisfaction survey, which was optional. Training Conditions Child Abuse Reporting Training Students were required to complete the child abuse reporting training as part of their senior teaching seminar at the university (N = 298), as a requirement in the counselor education ethics course (N = 194), or fulfillment of an undergraduate psychology course (N =805). The Child Abuse Training URL is www.childabuse. fiu.edu. The training site was built using Articulate Storyline (a course authoring, interactive on line learning system) and displayed in Flash. The testing site was built on the Moodle learning management system (LMS) and contained approximately 70 slides. The training curriculum was updated from its previous version (Kenny 2007) to include current child abuse research findings, Florida state policies and procedures, and inclusion of a video on mandated reporting. Content from the site was compiled from the current research literature on child maltreatment. Information was gathered from recent journal articles, books, media accounts, the Child Maltreatment report (U.S. Department of Health and Human Services, Administration on Children, Youth, and Families
Author's personal copy Journ Child Adol Trauma Table 1 Demographic and knowledge of child maltreatment types and reporting laws scores information across experimental conditions (N = 1690)
Variable
Maltreatment reporting (n = 1297)
Control group (n = 393)
Mean Age %Female
22.8 (5.3) 79 %
21.7 (4.0) 86 %
Hispanic White-Non Hispanic
70.8 9.7
83 6.7
African American
5.1
3.3
African Caribbean Asian
5.4 4.9
6.7 0
%Ethnicity
Other Student Level Freshman
19.0
21.5
Sophomore
14.0
15.6
Junior Senior Graduate Student
24.0 40.0 3.3
38.3 23.0 1.5
Mean (SD) Pretest
15.4 (2.4)
13.2 (3.6)
Mean (SD) Posttest
19.8 (1.8)
15.4 (2.2)
2012), and Florida statutes (Florida Statutes, 5 § 39.201 2011). The definition of child maltreatment was derived from the Child Maltreatment Report (U.S. Department of Health and Human Services, Administration on Children, Youth, and Families 2012): An act or failure to act by a parent, caregiver, or other person as defined under State law that results in physical abuse, neglect, medical neglect, sexual abuse, emotional abuse, or an act or failure to act which presents an imminent risk of serious harm to a child (p. 124). The following topics were covered in the training module: incidence and prevalence of child maltreatment; cases of abuse and neglect from the media; descriptions/indicators of types of abuse; emotional and behavioral consequences experienced by victims of child maltreatment; Florida law and statistics related to reporting child maltreatment; and child maltreatment reporting procedures for mandated reporters. For each type of child maltreatment (neglect, sexual abuse, physical abuse, emotional/psychological abuse), the following was included: definitions; signs and symptoms; examples; and possible sequelae. The tutorial contained a section specific to frequently asked questions and national resources on child maltreatment. The web site also included a 67-min video entitled, BMandated Reporting^ (Kid Safety of America 2012), which consists of three parts; Reporting Advocacy (explaining why it is important to report child maltreatment, case examples of children on whom reports were made); Reporting Process (how to make a report and to whom); and Reporting Aftermath: Myths and Challenges (what happens after a report
is made; and myths and challenges to the reporting process). The film explains the why, when, how, and to whom of reporting. The digital copyrights to this film were purchased so that it could be incorporated into the website. It took approximately 2 h for the participants to complete the tutorial, including film viewing. Control Training Module Control participants (n = 393) were not randomly assigned but, rather, chose to complete the tutorial as part of the aforementioned undergraduate psychology class requirement. Control group participants were not exposed to any of the child abuse training materials; rather, in an attempt to control for time on the computer, an Internetbased training module on mood disorders of similar length was created. The training site was built on the LMS, and contained approximately 70 slides related to various mood disorders, identification and diagnosis, as well as treatment options. Participants in this condition also watched two films specific to Bipolar Disorder and antidepressants (each film lasted approximately 30 min). Measures Knowledge of Child Maltreatment Types and Reporting Laws Because there are currently no empirically validated measures specifically developed to assess knowledge of child abuse reporting, the first author created a measure. A test bank of 46 multiple-choice (4 response) questions was initially generated by the first author to assist in evaluating the training curriculum. The questions were created to assess information that was addressed in the tutorial, and included reporting
Author's personal copy Journ Child Adol Trauma
procedures, signs of abuse and neglect, and penalties for not reporting. Three experts in child maltreatment (one academic lawyer specializing in child abuse, one psychologist who is clinical director at a child maltreatment treatment center, and one early childhood educator specializing in child abuse-all doctoral level academics) reviewed the test bank and were asked to select items that were deemed important knowledge for a future mandated reporter. Items were selected if two or more of the experts chose it for inclusion. Three items were selected by all three experts, while 18 items were chosen by at least two of the experts. Thus, the test resulted in 21 items for which two experts agreed. The order of questions in the test was randomized in both the pre- and posttest versions. Sample items include: BThe failure to report child abuse is…^; BWhen making a report of abuse to the proper authorities, you must include all of the following information…^; BMaking a report of suspected abuse can be done by…^; and BIf a parent fails to provide adequate shelter and clothing for a child, this may be considered…^. The test also included non-scored items requesting demographic information from participants. This measure was administered to all participants immediately before and after completion of the trainings. To ensure participants completed the entire tutorial before the post-test, the password to enter the posttest was generated after the tutorial was completed. Thus, participants could not gain access to the posttest until at least viewing the entire tutorial. All items on this measure were scored as either correct (1 point) or incorrect (0 points) for a maximum potential score of 21 points; raw scores were used in the analyses. Two week test-retest reliability for this measure was .84. Training Satisfaction Upon completion of the posttest, participants who had completed the child abuse training were directed to an optional Qualtrics survey that assessed the participants’ experience (e.g., What is the most important piece of information you learned; How can this tutorial be improved; What did you like best about this tutorial). The survey also asked participants to rate their knowledge prior to and after completing the training. In addition, there were 16 statements about specific aspects of the website (i.e., was the posttest difficult; would this material be better served in an in person presentation; I watched the film; I read all the information in the training). Participants responded to these statements utilizing a 4-point Likert scale that assessed the extent to which they agreed with each statement.
Results Chi square tests were conducted to determine potential differences between the experimental conditions (i.e., child abuse reporting training condition, control condition) on demographic variables (i.e., age, gender, ethnicity). Chi square
analysis on ethnicity (χ2 (1, N = 214) = 2.31, p > .05) and gender (χ2 (1, N = 1629) = .364, p > .05) revealed no significant differences between participants in the experimental and control groups. However, a t-test revealed a significant difference on age such that the experimental group participants were older than the control group participants (t = 4.218, p < .001). Therefore, age was included in the primary analysis below (see Knowledge of Child Maltreatment Types and Reporting Laws) as a covariate to assist in managing age between the experimental conditions. See Table 2 for these results. Knowledge of Child Maltreatment Types and Reporting Laws An analysis of covariance (ANCOVA) was conducted, utilizing group condition (child abuse reporting training vs. control) as the independent variable, age and Knowledge of Child Maltreatment Types and Reporting Laws pretest scores as covariates, and the Knowledge of Child Maltreatment Types and Reporting Laws posttest score as the dependent variable. The results indicated that participants who were in the child abuse reporting training condition improved more than control group participants on Knowledge of Child Maltreatment Types and Reporting Laws from pre- to posttest. The mean posttest score for the participant group was 19.81 (SD = 1.77) and the mean posttest score for the control group was 15.44 (SD = 2.15). Self- Reported Knowledge of Child Maltreatment Most of the participants (88 %) completed the Training Satisfaction survey, which was optional. Prior to taking the tutorial, participants rated their knowledge of child maltreatment as 17 % minimal, 72 % moderate and 11 % advanced. After taking the tutorial, participants rated their knowledge as 2 % minimal, 26 % moderate and 73 % advanced. A Wilcoxon signed ranks test demonstrated a significant change in participants’ selfreported knowledge (z = −26.72, p < .001), with the majority of participants (n = 781) reporting improved knowledge levels. Six (
