Educational Research Review 1 (2006) 148–156
Evaluating character education programs and missing the target: A critique of existing research Christopher A. Was a,∗ , Dan J. Woltz b , Clif Drew b a
Kent State University, Educational Foundations and Special Services, 405 White Hall, Kent, OH 44242, United States b University of Utah,United States Received 20 January 2006; received in revised form 14 August 2006; accepted 16 August 2006
Abstract The need for character education is apparent in the statistics regarding school violence, absenteeism, drop out rates, and achievement. The directive has been sent out by the government to get the job done. This manuscript examines a number of studies that claim to provide evidence that character education programs work, and others that provide evidence that is less supportive. These mixed conclusions are particularly difficult to synthesize because of conceptual and methodological weaknesses common in this area of research. In this review, rather than make a claim as to the effectiveness of character education programs, we point out that the research regarding character education has not examined evaluation questions with an empirically rigorous eye. We draw particular attention to the lack of behavioral outcomes reported in the existing research. © 2006 Elsevier Ltd. All rights reserved. Keywords: Character education; Moral education; Character education programs; Character education evaluation
Repeated calls for character education throughout school districts, cities, states, and the U.S. federal government have fueled the creation of a plethora of character education programs. Not only have these programs been designed, many have been implemented in U.S. schools. Many of these programs are used without rigorous scholarly research to support the designers’ claims of successful impact on student character. While research regarding character education programs does exist, the vast majority lacks the scientific rigor to support conclusions about character education whether favorable or not. The current review examines the research regarding the efficacy of character education and analyzes the research methodologies utilized. The need for empirical research investigating the outcomes of character education and comparisons of character education approaches is apparent in the number of ways that character education programs are distinguished. A basic review of character education literature uncovers a multitude of ways in which character education programs are distinguished from one another: Curricular versus instructional, traditional versus contemporary, formal versus informal, and many combinations of these. We begin by distinguishing between traditional and contemporary approaches to character education. Traditional approaches to character education are exemplified by education in ancient India. In this education system, moral education was not a separate topic for study, but was the underlying theme of the entire curriculum.
∗
Corresponding author. Tel.: +1 330 672 2929; fax: +1 330 672 3407. E-mail address:
[email protected] (C.A. Was).
1747-938X/$ – see front matter © 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.edurev.2006.08.001
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
149
This reflected the society’s belief that every act and thought had a moral quality. Content areas were taught by similar methods employed today, while the teacher provided constant modeling of moral thought, speech, and actions (Altekar, 1944; Keay, 1959). This approach to moral education, as an integrated part of the entire curriculum, was prominent in many ancient cultures and is recognized in some modern character education approaches. Character education has been a part of education in the U.S. beginning early in the nation’s history. Character education in the U.S. started in the middle of the 1600s (Vardin, 2003) and by the late 1800s two approaches had developed (Howard, Berkowitz, & Schaeffer, 2004). The traditional approach reflected the curriculum described above, in which the goal of education was to teach traditional values and indoctrinate students to these values. The second approach to develop in the U.S. was a relativist approach. This form of moral education is less about teaching absolutes in terms of right and wrong, and more about teaching students to think critically about the decisions they make when faced with an ethical dilemma. Most contemporary approaches to character education are derived from this relativist approach and are more content specific in the form of moral lessons, vocabulary words, and words of the month. Another common distinction among character education programs is curricular versus instructional. Curricular approaches use separate lessons to teach the specified virtues, but they can also use existing curriculum to teach the virtues. For example, teachers using historical or mythological figures to teach about virtues are using a curricular approach. Curricular approaches also include teaching virtues through “virtue of the month” techniques, in which a specific virtue is the focus for a determined period. Instructional approaches are based on an implicit curriculum, which includes how teachers teach, the school culture, and school traditions (see Wren, 1999). Instructional approaches are founded in the belief that how a teacher is being will have an impact on student behavior. Teachers model the specific behaviors that they wish for their students to adopt. An example of an instructional approach to character education is a teacher who models listening skills and the ability to empathize with an opposing point of view. The teacher requires the students take a position on a topic for debate. Before a student can argue their point of view, they must paraphrase the last speaker from the other side of the argument. This encourages empathy and understanding for others as well as better listening skills. The instruction is similar to the traditional approach in that character is an underlying theme throughout the curriculum. It is important to note that curricular and instructional approaches to character education are not mutually exclusive, but can coexist and support one another. Yet another distinction found in the character education literature is formal versus informal programs. Formal character education is based on direct instruction and intentional lessons designed to affect character development.1 Students are challenged to reflect on moral issues, values, and principles. Lickona (1989) described three parts to formal character education knowing the right, valuing the right, and making the right choices. Informal character education is a form of character education that takes advantage of the environment, teachable moments, and the immediate communities. Hanging posters with moral lessons, using word of the month, and moral scenarios are examples of informal character education techniques. It is clear that traditional and contemporary, formal and informal, and curricular and instructional approaches have considerable overlap. Moreover, a mix of these approaches is typically utilized in many character education programs. 1. The resurgence of character education The need for character education programs is expressed repeatedly in the literature through statistics regarding school violence, truancy rates, and number of dropouts. The reported level of violence in U.S. public schools is one variable leading to the call for character education. In the academic year of 1999–2000 approximately 1.5 million violent incidents occurred in U.S. public schools with 71% of public schools experiencing one or more violent incidents in the same time frame (National Center for Educational Statistics, 2003). The National Center for Educational Statistics (2003) also reported that 54% of U.S. public schools took at least one serious disciplinary action with 84% of the actions taken being a five-day or greater suspension. More than 1,100,000 actions were taken with greater than 10% of those actions being removal from the schools. The United States Department of Education (2005) reports that, 1
Direct instruction is an approach to teaching in which the teacher disseminates information directly to the students.
150
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
“Between 1989 and 1998, the rate of petitioned truancy status offense cases handled by juvenile courts increased by 85% (from 22,200 to 41,000), representing a 61% increase in the rate of truancy cases.” Although character education in schools throughout the U.S. has been a point of contention and debate for many decades, character education is making a strong comeback in response to these figures. Currently, in the U.S. there is a push for character education at the level of primary and secondary education. During the fiscal years of 1995–2001, 45 states had received grants from the U.S. federal government under the Character Education Pilot Project Grant Program (United States Department of Education, 2006a). Character education was included as a feature of the No Child Left Behind Act (United States Department of Education, 2006b), leading to a compulsory agenda to develop character education curricula. Given the current thrust for incorporating character education into the educational curriculum, efforts to understand the effectiveness of specific character education programs are essential. Questions that need to be asked include: Does character education make a difference? What outcomes are reasonable to expect from common approaches to character education? Are some approaches more effective than others at achieving targeted outcomes? Such questions are difficult to answer in any area of social program evaluation and they are particularly difficult to address in the area of character education. The state of California’s education code states that, Each teacher shall endeavor to impress upon the minds of the pupils the principles of morality, truth, justice, patriotism, and a true comprehension of the rights, duties, and dignity of American citizenship, and the meaning of equality and human dignity, including the promotion of harmonious relations, kindness toward domestic pets and the humane treatment of living creatures, to teach them to avoid idleness, profanity, and falsehood, and to instruct them in manners and morals and the principles of a free government (Section 233.5a). This section of the California Code of Education demonstrates the enormous range of variability in definitions of character. Equal to the range of abstract values which might be considered to define character, there are a multitude of approaches to character education and nearly as many structured programs have been developed. This complexity creates an enormous challenge for researchers wishing to address the questions about program effectiveness posed earlier. Within this article, we describe the difficulties inherent in evaluating character education using select examples to describe the range of research methodology employed. Because of government mandates, both federal and state, we focus on research in the U.S. 2. Starting point for evaluating character education: what are program goals? Character education is often touted as the pathway to instilling virtues in students, and the resounding virtues expressed in the literature are honesty, integrity, respect, and self-discipline. The first difficulty facing those evaluating the impact of character education programs stems from the nature of the specific program goals. Berkowitz and Bier (2004) stated that it is difficult to discuss the effectiveness of character education without considering its goals. The vast majority or researchers and writers working in the area of character education come to the same conclusions regarding what the intended outcome of a character education program should be. Splittgerber and Allen (1996) stated that the aims and objectives of learning and caring communities in the middle schools are to promote personal development, social development, and academic development of young adolescents. Others have proclaimed that the outcomes of character education should be based on personal virtues, relationship virtues, and community virtues (Kagan, 2001). These basic sets of virtues have also been categorized as moral virtues and social virtues (Beller, 2002). Although the nomenclature may vary, the underlying virtues are repeated through out the literature. The virtues, which are repeatedly declared to represent the intended outcomes of character education include: honesty, loyalty, dedication, citizenship, integrity, courage, perseverance, and self-motivation. The goals of character education programs are often stated in abstract terms that rely on constructs such as integrity and respect. How does one measure changes in integrity that might result from educational programs? These abstractions could be operationalized in a variety of outcome measures, but it seems unlikely that any self-report, interview, or observation measures could do justice to such broad and abstract constructs. Without adequate measures of the target virtues, how can we hope to evaluate which programs lead to positive outcomes? Even if we concede that the virtues that represent program goals are universal and measurable, there are several questions that remain regarding the evaluation of character education programs. The question is not whether character education is necessary. In the U.S. there are government mandates for character education. Nor are we currently
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
151
interested in determining how to approach character education. Multitudes of character education programs currently exist. The question we are interested in answering is “How do we determine if specific character education programs are obtaining the stated goals?” 3. Outcome measures used to assess progress toward the goals When one reviews the literature regarding character education, it is easy to develop a sense that the effectiveness of various character education programs has been demonstrated with a variety of outcome measures. Character education has been reported to improve academic motivation and aspirations, academic achievement, pro-social behavior, bonding to school, pro-social and democratic values, conflict-resolution skills, moral-reasoning maturity, responsibility, respect, self-efficacy, self-control, self-esteem, social skills, and trust in, and respect for teachers. Furthermore, character education has reportedly reduced absenteeism, discipline referrals, school failure, suspensions, school anxiety, pregnancy, and substance use (e.g, Brooks & Kann, 1993; Cassell, 1995; Hogan, 1996). A cursory view of these outcomes seems to correspond well to the common goals of character education. In reading this literature, one is often left with the impression that character education programs have broad positive impact, and that research simply needs to focus on determining which components of which programs are most effective at producing these outcomes. However, a closer look at the outcome measures used to arrive at these conclusions suggests that a conclusion of general effectiveness is not so clear cut. We suggest that in many cases, the outcomes described above are indirectly or poorly measured. The most common outcome measures assess student, teacher, and administrator attitude toward the program or school environment. Although attitude change is often a program goal and important to assess, it is too often used to the exclusion of other outcome measures. This is problematic because attitude measures are often easily influenced by social pressure to support program initiatives. Moreover, expressed attitudes can be quite inconsistent with actual behavior (De Los Reyes & Kazdin, 2005; Koppelman & Goodhart, 2005). Sometimes, the evaluation of student change is assessed by teacher or administrator perceptions of the students. Admittedly, measuring most psychological and social constructs that have broad, real-world meaning in student populations is difficult. However, this difficulty is increased dramatically when researchers assess outcomes in one population (e.g., students) using the perceptions of others who might have different goals, motives, and experiences (e.g., parents, teachers, administrators). This is particularly problematic when attempting to measure difficult-to-observe constructs such as motivations, attitudes and values. Finally, too often research on character education uses measures of perception for behaviors that can be objectively measured (e.g., absenteeism, academic success, etc.). We examine these issues more closely within a sample of representative character education evaluation studies. It is important to note that the extent of peer reviewed research regarding the efficacy of character education programs is quite limited. Howard et al. (2004) stated that the research in this area is extremely vague and limited when one considers the number of character education programs in place. In general, the studies included in this review were chosen based on the date of the study, the author’s claim that the implementation of a specific character education program was evaluated in their study, and the methodology employed (see Table 1). Every attempt was made to gather empirical research published in peer reviewed, scholarly journals. However, this proved to be a difficult task. Leming (1993) stated that two approaches exist for measuring the effectiveness of modern character education programs. The first consists of informal evaluations based on anecdotal evidence or surveys from administrators and teachers regarding their perception of the program. Of the studies reviewed for this manuscript, the first six measured only the perceptions of character education participants (students, faculty, or staff). However, there is variability in these studies as to the research questions asked and the outcome measures applied. A typical example of a study representing this approach to evaluation is Davidson and Stokes (2001). Davidson and Stokes explicitly stated that the purpose of the study was to determine if character education reduced discipline problems, sexual harassment, and resulted in better behaved students. Instead of measuring the incidence of discipline problems from records kept over time, Davidson and Stokes asked teachers and administrators if the students were better citizens after the character education program and if the teacher and administrators wanted to continue implementing the program. Teacher and administrator attitude toward the program may be important, but it is difficult to defend this as an accurate measure of the incidence of specific discipline problems. Another concern with the Davidson and Stokes (2001) study was the lack of control regarding the character education programs studied. The researchers distributed questionnaires to educators pursuing a Master’s degree in secondary
152
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
Table 1 Author(s) (year)
Research questions CEa
Davidson and Stokes (2001)
Does
Perry (2002)
Do students, parents, faculty, and staff feel a sense of community in a Community of Caring school? How has the CE impacted the climate and culture of the school, and the behavior of the students?
Stoppleworth (2001)
Revell (2002) Harrington et al. (2001)
reduce discipline problems?
Does the social and political environment of the school determine outcome of CE program? Is the All-star program an effective CE program?
Muscott and Talis-O’Brien (1999)
Are CE programs effective for students behavioral and learning disabilities?
Corley (2000)
Does CE affect student’s standard’s of right and wrong? Does the Heartwood Character Education program impact student behavior Does the Child Development Program change student attitudes about school and academic progress, and change their social attitudes, values, and skills? Will the intervention of a literature-based program decrease disciplinary infractions and increase in grade point averages?
Leming (2000) Battistich (2001)
Fink et al. (2003)
Duer et al. (2002) a
Will a character education program improve behavior related to respect and responsibility?
Methodology Asked teachers and administrators if the students were better citizens after the CE program. Assumed a sense of community leads to a decrease in misbehavior and an increase in good behavior. Stated a change in student behavior based on one student’s claim that he/she had decreased the number fights in which he/she was involved. Interviewed 700 students involved in CE programs regarding their views of citizenship. Results were based on no measures of behavior changes, only self-report of the students. Interviewed 19 students with behavioral disorders and learning or language disabilities regarding whether the student benefited from CE program. Measured student self-reported standards of right and wrong, not changes in student’s behavior. Compared program teachers and comparison teachers ratings of student behavior. Measured a significant reduction in alcohol and marijuana use, and violent behaviors, including gang fighting. Found a decrease in disciplinary infractions, an increase in grade point averages, and an improvement in attendance and cafeteria behavior. Found a reduction in the number of tardies, truancies, insubordination and fighting.
CE: character education.
education and asked these educators to distribute the questionnaires to faculty and administration in their respective schools in the state of Alabama. Davidson and Stokes (2001) reported that in 1995 the state Alabama mandated that all schools in the state “teach character education 10 min per day using the teaching approach, virtue of the week” (p. 3). The authors made no attempt to control for differences in how the various schools implemented the mandated character education. This lack of control in the distribution of questionnaires results in ambiguity regarding the type of character education programs evaluated, which greatly limits the specificity of interpretations that can be drawn from the evidence. In another study, Perry (2002) asked if students, parents, faculty, and staff felt a sense of community in a Community of Caring school, and if the community of caring program was seen as the cause of the sense of community. Community of Caring is a comprehensive approach to character education which incorporates five core values into all aspects of the Pre K-12 curriculum, and is specifically aimed at students with disabilities (National Center for Community of Caring, 2006). Teachers incorporate the five core values (caring, respect, responsibility, trust, and family) into all aspect of the classroom and course content. Students also work with each other on problem solving and reflection on the core values. Families, school, and community work on building relationships, and students participate in service learning. Perry (2002) assumed that if there is a sense of community then there is a decrease in misbehavior and an increase in good behavior based on Battistich’s (2001) report to the American Educational Research Association (AERA) national convention. Although there is evidence suggesting that a sense of community is associated with positive interpersonal behavior, it is not a foregone conclusion that a sense of belonging or community does lead to positive interpersonal behavior. Again, easily obtained perceptions of the program are substituted for more objective measures of behaviors that are expected to change. Stoppleworth (2001) also studied the perception of students and teachers in a character education program. Stoppleworth defined character education as “any class, program, club, or initiative that directly promotes or encourages
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
153
values. . .” (p. 71). According to Stoppleworth, in the school chosen for the case study, 75% of the students had been exposed to some variant of character education. Again, we see a lack of control regarding the character education program examined by the researcher, which makes focused interpretations impossible. Stoppleworth (2001) stated that one of the underlying research questions was “How has the character education impacted the climate and culture of the school, and the behavior of the students?” (p. 70) Evaluating changes in climate and culture of a school might be best accomplished with qualitative methods, but behavior change in students would be best accomplished with objective measures across all students. Again, it is erroneous to claim that measuring beliefs about the impact of a character education program is the same as measuring the behavioral changes resulting from the program. Although Stoppleworth interviewed many students in the case study school, the claim that behaviors changed in the school was based an individual student’s self-report that he/she had decreased the number fights in which he/she was involved. There are too many contextual demands on students to respond in a desirable manner to such inquiries to be able to equate the responses with actual behavioral change. Again, discipline records, if kept in a consistent manner, would be a more direct measure of the target outcome. In a much broader qualitative study, Revell (2002) interviewed 700 students involved in character education programs in Chicago, Illinois regarding their views of citizenship and their perceptions of the character education program. She found that the type of school (magnet versus non-magnet) was correlated to their responses.2 She concluded that the social and political background of the school influences the way students receive and understand the character education program. Although Revell (2002) chose 12 public schools for this project based on their participation in the same character education project, there are characteristics of the schools in her study that may have influenced her conclusions. Revell stated that another factor for choosing schools for this study was the schools had been involved in the character education program for a substantial amount of time. However, she does not provide detailed information regarding the range of time which the schools had participated in the program. Revell also reported that some of the schools were in areas that were “associated with particular demanding social environments,” (p. 423) and that one was for children who were older, but not yet progressed through the 9th grade. When discussing the differences between schools, these schools were not distinguished from other non-magnet schools. The conclusions regarding the differences in the way students in the magnet schools responded compared to responses in the non-magnet schools are at best difficult to interpret without distinguishing these confounding variables. Harrington, Giles, Hoyle, Feeney, and Yungbluth (2001) conducted a study of the All-Stars program. This program is specifically designed as a prevention program to reduce sexual behavior, drug abuse, and violence by targeting mediated variables. The program includes whole-classroom sessions, smaller group sessions, and one-on-one sessions. Harrington et al. (2001) designed a large-scale field study of 1655 students. Six hundred twenty nine of the students were in a specialist condition (a specialist in the All-Star program led the character education curriculum), 287 students in the teacher condition (teachers were trained in the All-star program), and 739 students in the control condition (no All-star program). Overall, they reported no consistent effects of the All-Star program on problem behaviors. Interestingly, as with many other studies, the results were not based on direct measures of behavior, but instead on the self-report of the students about their behavior change. It is evident that although some of these studies resulted in statistically significant findings, there is often questionable validity to the conclusions that are drawn from them. When researchers set out to determine if behavior change is an outcome of character education and only measure attitudes about the program, or attitudes regarding outcomes, and often the attitudes of a third party, the conclusions regarding behavior are at best inferential. In addition, when using qualitative measures of attitude change it is important to measure attitudes prior to the intervention as well as after the intervention. Alternatively, some progress has been made in measuring outcomes that are consistent with the research questions. The following studies approached the research questions with what we view as appropriate methodology, although often with inconsistent results. Muscott and Talis-O’Brien (1999) were specifically interested in the effects of character education on students with behavioral and learning disabilities. The character education program in the study, SO (Service-Learning Opportunities) Prepared for Citizenship, is an after school program designed to increase character development in students by teaching specified character traits. The authors used an interview format with 19 students with behavioral disorders and learning 2
Magnet schools were defined by Revell (2002) as schools where children are accepted based on examination results and other schools were defined as “less academic.” (p. 423).
154
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
or language disabilities. Results indicated that these students felt they learned from the character education program and found that the students believed they were taking responsibility for their actions, responded to ideas of teamwork and cooperation, learned to make new friends, and found learning about character education fun and rewarding. Although they do not provide records-based measures that the character education program employed made a difference in the behavior of these students, the qualitative research methods employed in this study were more rigorous then previously reviewed qualitative studies. However, as Muscott and Talis-O’Brien (1999) stated in their conclusion the use of experimental and quasi-experimental designs would enhance the rigor and expand the potential of research in this area. Although the study resulted in null results, Corley (2000) used a quasi-experimental, pre/post test design using a treatment group and control group, and a character education program as the treatment. Corley measured student self-reported standards of right and wrong. After completion of the 3-week character education program, there were no significant differences in changes student’s standards of right and wrong between the two groups. However, a portion of the measure designed by Corley measured change in student decision making provided with morals based scenarios prior to and following the character education, the students were asked questions regarding how one might respond in particular situations. The questions regarding moral actions may simulate how one might respond in an actual situation. In their stages of Change model, Prochaska and DiClimente (see Prochaska, DiClemente, & Norcross, 1992) argued that in the contemplation stage addicted individuals have realized that a problem exists. Although one may stay in this stage indefinitely, the individual may also move into the preparation stage in which the intention to change behaviors is present which is often accompanied by a reduction in problem behaviors. Corley (2000) also gathered change score data of discipline referrals for the control and experimental groups. The results were not significant, but given direct measurement of the target construct and the sound methodology of the study, this administratively less desirable finding may be more accurate than the findings from other studies previously cited. Although Corley (2000) did not find significant changes in student’s standards of right and wrong, or in discipline referrals the pretest/posttest, quasi-experimental design employed provides sound methodology for answering the research questions regarding the program’s efficacy. Leming (2000) conducted a study of the impact of the Heartwood Ethics Curriculum for Children character education program on student behavior. Lemming reported that ethical conduct data from teacher’s ratings of student conduct provided differing perspectives on the impact of the program. The Heartwood curriculum is a read-aloud literature based curriculum in which lessons and home based assignments are organized around multicultural stories. In a pretest/posttest design conducted by Leming (2000) each student’s score on a teacher questionnaire regarding student behavior ranged from 16 to 80. Scores at the high end of the scale indicated a teacher’s judgment of the presence of good character. In this quasi-experimental study, there were two program (treatment) and two comparison (control) schools. Teachers in the program schools were trained in the Heartwood curriculum. Pretests questionnaires were given to all teachers at the beginning of the school year and posttest questionnaires were given to all teachers at the end of the school year. Examination of the results of the ANCOVA at grades one through three indicated that the difference between program and comparison teachers’ ratings was statistically significant. An examination of the adjusted posttest means revealed that the Heartwood teachers rated their students’ behavior lower than the comparison teachers’ rated their students’ behavior. It should be noted, however, that both Heartwood teachers and comparison teachers noted improved student conduct on the posttest measure. Leming (2000) pointed out that it has been noted in other evaluations of character education programs that teachers in character education programs tend to develop higher expectations for student behavior and, as a result, after implementation of a character education program ratings of student conduct may be made using higher standards (e.g. Dunn & Wilson, 1997). If Dunn and Wilson (1997) are correct, this points to the problems of validity with using only indirect measures of behavior (self-report and others-report) and not including quantitative measures of behavior when evaluating character education programs. Of the studies covered in this review most are supportive of the character education programs they covered. Many of the studies have provided evidence that teachers and administrators involved in the programs believed the program made a difference. Others provide evidence that the students enjoyed the character education program, feel their attitudes changed, and even stated they are more likely to be cooperative and incorporate teamwork. What’s missing is direct measurement of the stated outcomes of the character education programs covered in these studies. If the intended outcomes of character education include reduced absenteeism, discipline referrals, pregnancy, school failure, suspensions, and substance abuse, then why does the majority of the research regarding character education programs focus on teacher, staff, administration, and student perception of the programs and not the behavioral outcomes?
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
155
One study that measured behavioral records as well as perception outcomes is that of Battistich (2001). Battistich conducted a study of the Child Development Program (CDP), which is a whole-school approach intervention program designed to foster elementary students’ social, ethical, and academic development by helping schools become communities of caring learners. In the 2001 study, Battistich compared 12 CDP schools to 12 comparable schools from six school districts across the U.S. in a four-year study. Results indicated that when the CDP was implemented throughout the schools, not only did student attitudes about school and academic progress change, but there was significant change in their social attitudes, values, and skills. The contention being made in this manuscript is that measures in behavioral changes and measures of the attitudes of participants are equally essential to support the effectiveness of a character education program. Consistent with this assertion, Battistich (2001) found that students reported significant reduction in alcohol and marijuana use, and that school records indicated a decrease in violent behaviors, including gang fighting, and misconduct in school. A follow up study conducted by Battistich (2001) which included 334 of the CDP students and 191 comparison students found that in middle school the CDP students were significantly less likely to engage in misconduct and delinquency. This follow up study provides further evidence that the character education program made a significant difference in the behavioral outcomes intended for such a program. It is interesting and necessary to point out that in the initial study; Battistich (2001) completed a structural equation model (SEM) in which changes of the students’ perception of the school as a community mediated the relation of the CDP program and behavioral outcomes. Battistich does not present the SEM for the reader in this article. Although the mediated relationship supports the view that changes in attitude and perception leads to changes in behavior, it is difficult to argue for cause and effect when using correlational data, even when analyzed with SEM. However, Battistich (2001) did provide evidence of behavioral change directly measured from behavior records. This is the type of data necessary to make a solid argument that character education programs have a significant impact on the type of behaviors that are targeted as objectives of these programs. Similarly, Fink, Hansen, and Jensen (2003) found that the intervention of a literature-based program in a middle school in a northern suburb of Chicago led to a decrease in disciplinary infractions, an increase in grade point averages, and an improvement in attendance and cafeteria behavior. Duer, Parisi, and Valintis (2002) implemented a character education program specifically designed to improve behavior related to respect and responsibility. Administrative data gathered after the implementation of the program revealed that there was a reduction in tardiness, the number of truancies, insubordination and fighting. The three preceding studies each reported significant results based on direct records-based measures following the implementation of a character education program. However, the behavioral changes reported are subject to a variety of influences, and the implementation of the specific character education program is only one such influence. Battistich (2001) begins to approach the complexity of influences on student behavior by including the mediating variable of student’s perception of changes in the school community. Many other factors could be included when modeling the variety of effects on student behaviors. Factors such as teacher expectations, parent involvement, and student attitudes about the character education program certainly play a role in the efficacy of the program. It is also important to note that records-based measures are subject to threats to validity. Record keeping systems often lack quality control and may contain incomplete or even inaccurate data. A study’s findings may be may inadvertently affected by data collection activities if data collectors are aware of which students took part in the treatment and control conditions. A researcher’s or data reporter’s expectations may unintentionally influence scoring practices. As previously mentioned, Dunn and Wilson (1997) contended that teachers expectations of student behavior may change following participation in a character education program. To eliminate these potentials for bias, researchers should practice data collection procedures that minimize these threats to the study’s internal validity. 4. Conclusions The question of whether or not character education should or even will be implemented in U.S. schools is becoming less relevant. Federal government mandates and regulations have set forth directives that will place character education in the schools. Instead, questions regarding the effectiveness of the multitude of different character education programs are essential to address. However, if, educators, administrators, and government officials wish to effectively promote the use of character education in schools, it is necessary to continue to collect the type of data that will answer these questions. Qualitative data and analyses based on sound techniques are certainly an important part of the research process. However, given the stated goals of most programs, consistent use of direct, quantitative measures designed to
156
C.A. Was et al. / Educational Research Review 1 (2006) 148–156
assess the intended behavioral outcomes of character education must also be part of the research process. Pretest/posttest designs allow researchers to measure actual changes in school statistics and quasi-experimental and experimental designs allow researchers to control for a number of confounding variables. The use of such techniques by objective researchers will ensure that progress in the character education endeavor is made. We urge researchers to stay true to rigorous qualitative and quantitative methods in pursuit of these questions. References Altekar, A. S. (1944). Education in ancient India. Benares: Nand Kishore & Bros. Battistich, V. (2001, April). Effects of an elementary school intervention on students’ “connectedness” to school and social adjustment during middle school. In J. Brown (Chair), Resilience education: Theoretical, interactive and empirical applications. Symposium conducted at the annual meeting of the American Educational Research Association, Seattle, Washington. Beller, J. (2002). Positive character development in school sport programs. Washington, DC: ERIC Digest. ERIC clearinghouse on teaching and teacher education. Berkowitz, M. W., & Bier, M. C. (2004). Research-based character education. The Annals of the American Academy, 591, 72–85. Brooks, B. D., & Kann, M. E. (1993). What makes character education programs work? Educational Leadership, 51(3), 19–21. Cassell, J. R. (1995). Improving self-control in upper elementary students through a program of character, civic, and social education. Unpublished manuscript Retrieved May 10, 2006 from http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content storage 01/0000000b/80/22/58/6f.pdf. Corley, S. H. (2000). The effects of character education of ninth grade students at an urban high school. Dissertation abstracts international (UMI No. 9969485). Davidson, L., & Stokes, L. C. (2001, November). Educators’ perceptions of character education. In Paper presented at the Mid-south educational research association regional meeting, Little Rock, AR. De Los Reyes, A., & Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483–509. Duer, M., Parisi, A., & Valintis. (2002). Character education effectiveness. Washington, DC: ERIC digest. ERIC clearinghouse on teaching and teacher education. Dunn, L. T., & Wilson, D. (1997). A research report—moral classrooms: The development of character and integrity in the elementary school. Kansas City, MO: The Teel Institute. Fink, C., Hansen, C., & Jensen, J. (2003). Improving student achievement through character education. ERIC Digest. Washington, DC: ERIC clearinghouse on teaching and teacher education, 2002. Harrington, N. G., Giles, S. M., Hoyle, R. H., Feeney, G. J., & Yungbluth, S. C. (2001). Evaluation of the All Stars character education and problem behavior prevention program: Effects on mediator and outcome variables for middle school students. Health Education and Behavior, 28(5), 533–546. Hogan, M. G. (1996). Increasing the responsibility levels of fourth grade gifted children by promoting positive character traits and caring behaviors. Retrieved May 10, 2006 from http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content storage 01/0000000b/80/22/92/a2.pdf. Howard, R. W., Berkowitz, M. W., & Schaeffer, E. F. (2004). Politics of character education. Educational Policy, 18(1), 188–215. Kagan, S. (2001). Teaching for character and community. Educational Leadership, 59(2), 50–55. Keay, F. E. (1959). A history of education in India and Pakistan. Calcutta: Oxford University Press. Koppelman, K. L., & Goodhart, R. L. (2005). Understanding human differences: Multicultural education for a diverse America. Boston: Pearson/Allyn & Bacon. Leming, J. S. (1993). Character education: Lessons from the past, models for the future. Camden, ME: The Institute for Global Ethics. Leming, J. S. (2000). Tell me a story: An evaluation of a literature-based character education programme. Journal of Moral Education, 29(4), 413–428. Lickona, T. (1989). Educating the moral child. Education Digest, 55(1), 45–48. Muscott, H. S., & Talis-O’Brien, S. T. (1999). Teaching character education to students with behavioral and learning disabilities through mentoring. Education and Treatment of Children, 22(3), 373–390. National Center for the Community of Caring (n.d.) Retrieved May 31, 2006 from http://www.communityofcaring.org/AboutCofC/overview.html. The National Center for Educational Statistics (2003). Retrieved May 30, 2006 from http://nces.ed.gov/pubs2005/crime safe04/. Perry, C. M. (2002). Snapshot of a community of caring elementary school. School Community Journal, 12(2), 79–102. Prochska, J. O., DiClemente, C. C., & Norcros, J. C. (1992). In search of how people change: Applications to addictive behaviors. American Psychologist, 47(9), 1102–1114. Revell, L. (2002). Children’s responses to character education. Educational Studies, 28(4), 421–422. Splittgerber, F., & Allen, H. (1996). Learning and caring communities: Meeting the challenge of at-risk youth. Clearing House, 69(4), 214–216. Stoppleworth, L. (2001). An ethnographic study of participants’ perceptions of character education including students, parents, teachers, club sponsors, administers, and community support people. Dissertation Abstracts International,. UMI No. 3025154. U.S. Department of Education. (n.d.). Retrieved May 31, 2006a from http://www.ed.gov/admins/lead/safety/training/truancy/problem pg17.html. U.S. Department of Education. (n.d.). Retrieved June 1, 2006b http://www.ed.gov/programs/charactered/awards.html. Vardin, P. (2003). Character education in America. Montessori Life, 15(2), 32–34. Wren, D. J. (1999). School culture: Exploring the hidden curriculum. Adolescence, 34(135), 593–596.
