The Effects of a STEM Professional Development Intervention on ...

Copyright 2011 by the Mid-South Educational Research Association

RESEARCH IN THE SCHOOLS

2011,VoL18,No. 2, 16-25

The Effects of a STEM Professional Development Intervention on Elementary Teachers' Science Process Skills

Alicia Cotabish, Deborah Dailey, Gail D. Hughes, and Ann Robinson University of Arkansas at Little Rock

In order to increase the quality and quantity of science instruction, elementary teachers must receive professional development in science learning processes. The current study was part of a larger randomized field study of teacher and student learning in science. In two districts in a southern state, researchers randomly assigned teacher participants to the experimental and control conditions. The current study reported the effects of 2 years of sustained, embedded professional development on Grade 2 through Grade 5 elementary teachers' science-process skills, as defined by their ability to design experiments, and teacher perceptions of their capacity to lead students in scientific explorations. The results revealed a statistically significant difference between the adjusted post-test scores for the two groups, with the experimental group scoring higher than did the control group, indicating a significant gain in teachers ' science process skills. The results for teacher perceptions about their own science process skills revealed a statistically significant increase between pre- and post-test scores, with a large effect size reflecting a gain in teacher views of more than 2 points on a 4-point scale. The results for teacher perceptions about students' science process skills also revealed a statistically significant increase between pre- and post-test scores with a large effect size. The results of this study document the effects of sustained and targeted teacher professional development focused on improving content-specific science instruction for students in the elementary classroom.

Elementary teachers are the beginning of the Science Technology Engineering Mathematics (STEM) pipeline. To develop the gifts and talents of friture STEM innovators, elementary classrooms should provide rich, inquiry-based science insfruction (National Science Board, 2010). Unfortunately, research suggests that science is virtually ignored in the elementary grades. Fulp (2002) reported that Grade K-5 teachers spent only an average of 25 minutes on science instruction each day. Furthermore, Fulp (2002) indicated that science class usually involved reading about science instead of engaging in authentic hands-on science. In a national observation study, Banilower, Smith, Weiss, and Pasley (2006) documented that 83% of K-Grade 5 science lessons were considered poor quality.

Banilower et al. (2006) indicated that these lessons lacked appropriate teacher questioning to guide students toward an understanding of science concepts. The dismal state of elementary science education has prompted policy makers to issue a clarion call to increase the talent pool in the United States by improving K-12 science education (Committee on Science, Engineering, and Public Policy, 2007). Duschl, Schweingruber, and Shouse (2007) suggested that a teacher's understanding of science as a discipline affects instruction and student achievement more than does a teacher's factual recall. The quality of understanding science is not stressed in many undergraduate preparation programs where science is often seen as a body of facts and investigation is merely following sequential steps in the scientific method with no real emphasis on inquiry or evidence-based conclusions (Duschl et al., 2007). To compound the problem, few elementary teachers engage in professional development activities to improve their science teaching after

Correspondence concerning this article should be addressed to Alicia Cotabish, Center for Gifted Education, 2801 South University Avenue, University of Arkansas at Little Rock, Little Rock, AR 72204-1099 E-mail: [email protected] Fall 2011

16


ALICIA COTABISH, DEBORAH DAILEY, GAIL D. HUGHES, AND ANN ROBINSON

receiving their undergraduate degrees (Rice, 2005). Fulp (2002) reported that a majority of elementary science teachers have had fewer than 15 hours of science-specific professional development. In order for teachers to lead effective science instruction, they must know how to teach science, and they must know how students leam science. Teachers need to be familiar with the language of science and they need to have a deep understanding of the processes of science. Specifically, Michaels, Shouse, and Schweingruber (2008) maintained that teachers must be able to create science leaming opportunities for their students and be able to relate science language and leaming to real-life events. In a recent case study of six elementary teachers by Goodnough and Nolan (2008), teachers reported concems about their ability to teach science. One teacher in particular commented that her lack of grounding in science resulted in her discomfort in leading science lessons. Eshach (2003) noted that when an elementary teacher is concemed about her/his ability to teach science, she/he tends to develop a dislike for the science discipline and will avoid teaching it, if possible. Eshach (2003) reported that significant changes in teachers' attitudes toward science can be improved in a relatively short time, as indicated by a study on inquiry-based science instruction. In a survey of 300 primary teachers from the United Kingdom, Murphy, Neil, and Beggs (2007) found strong relationships between science professional development opportunities and increased confidence in teaching science, thereby indicating that elementary science instruction can be improved by providing high-quality, sustained, professional development opportunities. In Taking Science to School, Duschl et al. (2007) called for an increase in empirical research that targets "building expertise in science teaching" (p. 353). In particular, Duschl et al. recommended research that includes studying the effects of sustained professional development practices such as mentoring. In addition, Duschl et al. reported limited evidence regarding teacher perceptions of science teaching and student leaming. Thus, the current study sought to contribute empirical research to address these gaps.

through eighth-grade teachers experience sustained science-specific professional development in teacherpreparation and in-service programs (Duschl et al., 2007). As recommended by the NRC, teacher leaming should focus on (a) developing science content knowledge using the classroom-adopted science curriculum, (b) providing demonsfrations of how students leam science, (c) providing examples of integrating technology into the science curriculum, and (d) modeling what the teacher will actually teach in the classroom. Moreover, professional development for elementary science teachers should be embedded in the context of a real-world classroom, and take into account teachers' concems, their need for resources, and their current knowledge and skills (Bitan-Friedlander, Dreyfus, & Milgrom 2004; Buczynksi & Hansen, 2010). Developing Teachers Science Process Skills Professional development. High-quality professional development has the power to affect teacher instruction and student leaming. The Council of Chief State School Officers (CCSSO, 2008) targeted five indicators of high-quality professional development in mathematics and science. These areas included "content focus, active leaming by teachers and coherence with curriculum/standards, collective participation, and sufficient time" (CCSSO, 2008, p. 8). Yoon, Duncan, Lee, and Shapley (2008) noted that professional development can improve student achievement by enhancing teacher knowledge, skills, and motivation. Science professional development. When examining science-specific professional development, Kennedy (1998) reported increased student achievement in science when teacher inservice programs modeled scientific processes in developing content knowledge. Duschl et al. (2007) described science processes as "making observations and measurements of natural phenomena, articulating hypotheses, and designing and carrying out experiments" (p. 14). Beyer and Davis (2009) suggested that professional development that affords teachers opportunities to conduct evidence-based scientific practices, such as those described in science processes, helps to increase teachers' science content knowledge and conceptual understanding. Their research suggested that teachers need practice in inquiry-based instmction in order to provide effective feedback to students when the opportunity arises. Such professional development has the potential to improve science instruction because teachers must leam the science content that they need to teach and they must experience inquiry-based instruction themselves before they are able to implement it. Gerard, Varma, Corliss, and Linn (2011) reported

Theoretical Framework The theoretical framework of this study was guided by the principle that teacher instruction in science can be improved with sustained, embedded professional development that mirrors what teachers are expected to do in an inquiry-based curriculum grounded in real-world problems. The National Research Council (NRC) recommended that all K-

Fall 2011

17


THE EFFECTS OF A STEM PROFESSIONAL DEVELOPMENT INTERVENTION ON ELEMENTARY TEACHERS' SCIENCE PROCESS SKILLS that "professional development programs that support teachers to engage in comprehensive, consfructivistoriented leaming processes can improve students' inquiry-science leaming experiences" (p. 438). Duration. Research suggested that both quality and quantity of professional development affects science instmctional practices. In an analysis of teacher professional development programs in mathematics and science, the Council of Chief State School Officers (CCSSO) (2008) documented that most effects were reported by programs providing a minimum of 45 hours of professional development annually. Moreover, to increase the usage of inquirybased practices, at least 80 hours of professional development are needed for teachers to establish an investigative classroom culture (Supovitz & Tumer, 2000). These professional development experiences usually involved summer institutes plus jobembedded activities throughout the year. In a review of literature, Gerard et al. (2011) reported that professional development that exceeded 1 year in duration prepared teachers to incorporate curriculum that increased students' inquiry-based leaming experiences.

coaching in science should focus on classroom support to enhance pedagogical content knowledge. Such support is vital in an elementary classroom where materials and content knowledge are limited. Another science study was focused on a limited mentoring partnership between new, inexperienced elementary teachers and experienced teachers (Gustafson, Guilbert, & MacDonald, 2002). The beginning teachers explained that their most beneficial leaming came from their conversations with and observations of experienced teachers. As a result of these interactions, new teachers were able to gain professional knowledge and self-confidence in their own science instmctional practices. Koch and Appleton (2007) described a case study of two university science educators and two elementary teachers who were interested in improving their science teaching. The teachers received initial fraining on the science curriculum followed by mentoring support from the university science educators. Both elementary teachers indicated they were initially uncomfortable with anything less than a controlled environment, but after the mentoring experience, teachers were willing to lead inquiry-based activities in their classrooms. The teachers also indicated that their questioning skills had improved, and they would encourage students to think scientifically before answering questions. It is evident that sustained, embedded professional development is needed to change instmctional practices of elementary science teachers. Peer coaching and mentoring is a means to extend teacher leaming and provide real-world support (Little, 2005; Showers & Joyce, 1996).

Peer coaching. Sustained, embedded professional development can involve a collaborative program utilizing a mentor or a peer coach. This type of professional development allows the teacher to apply leaming in the real world of the classroom while being supported by a peer coach. Showers and Joyce (1996) defined peer coaching as a means of helping teachers fransfer newly acquired leaming and skills from the workshop to the classroom. Peer coaching is a mode of support for teachers to ensure students reap the benefits of science teaching and leaming. Furthermore, Chase and Wolfe (1989) described peer coaching as a means by which teachers can experiment with new ideas in a safe environment free from the fear of failure. Little (2005) commented that peer coaching can provide teachers a natural support system that can enhance teacher performance by sharing knowledge and expertise through collaboration. Providing teachers with support and expert knowledge can lead to positive changes in the classroom.

When quality professional development is combined with a coaching or mentoring program, optimal changes in teacher insfruction occur (Neuman & Cunningham, 2009). Neuman and Cunningham reported a moderate effect size (d = 0.77) when peer coaching was paired with professional development in comparison with implementation of a professional development component only. When measuring the effect of peer coaching on student leaming. Showers (1984) found coaching significantly confributed to higher student achievement scores on a concept attainment measure. Results of these studies indicated that peer coaching enhances the ability of teachers to fransfer their leaming to the classroom, leading to greater student achievement. In summary, research supports the need for sustained, embedded, quality science professional development for elementary teachers. According to Buczynski and Hansen (2010), professional development must provide the fraining and support needed by teachers to implement an inquiry-based curriculum. In order to increase the

Although peer coaching as a form of professional development is not a cure-all for weak science content knowledge, research does suggest that the residual benefits are evident for teachers. In a mentoring study, involving two middle school science teachers and a university professor, Appleton (2008) found that after peer coaching, teachers had not become experts in science content, but they could access the science content needed for the lessons they wanted to teach. Appleton concluded that in order to generate change in the science classroom, peer

Fall 2011

18



had no intervention from STEM Starters during these 2 years, but received duplicate services at the conclusion of the study to frjlfill an agreement with the two school disfricts.

quality and quantity of science instmction, elementary teachers must receive professional development in science leaming processes. Few randomized field studies have focused on the combined effects of sustained, embedded professional development (peer coaching) and professional development institutes when examining elementary-teachers' science-process and perceptions about leading student investigations. To address these concems, a STEM initiative, STEM Starters, was developed and subsequently funded through the U.S. Department of Education. STEM Starter program components, goals, objectives, and activities focus on increased science leaming for all students in Grades 2 through Grade 5, and increased knowledge and skills in the STEM disciplines for their educators (Cotabish, Dailey, Robinson, & Hughes, in press; Cotabish, Robinson, MacFarlane, & Wood, 2010; Dailey, Cotabish, Robinson, & Hughes, 2011, 2012).

Participants Randomly selected from two disfricts in a southem state, 70 teachers from Grades 2 through Grade 5 were assigned to the freatment and confrol groups. Funding consfraints resulted in the selection of two districts. The two districts were specifically selected because they were geographically located in the proximity of the researcher, and because their demographics were representative of the state. Nine percent of the student population in Disfrict 1 was culturally diverse and 40% of the students participated in the free- and reduced-lunch program. Thirty percent of the student population in Disfrict 2 was culturally diverse and 69% of the students participated in the free- and reduced-lunch program. Due to attrition, data were analyzed from 30 teachers in the experimental group and 29 teachers in the control group. There were also casewise missing data when individual teachers missed adminisfrations of some instruments; the missing data resulted in smaller numbers for some analyses. Although random selection was utilized, all participants were female with the exception of two males. Years in teaching ranged from 0 to 34, with an average of 12.8 years (SD = 7.48). Fifty seven of the 59 teachers in the study reported less than 1 year of experience in teaching science. This finding was expected because literacy and mathematics are the foci of instruction; science is often ignored in the elementary grades (Milner, Sondergeld, Demir, Johnson, & Czemiak, 2011). Table 1 summarizes teachers' grade level disaggregated by disfrict (#1, #2) and condition (experimental, confrol).

Purpose of the Study The purpose of the current study was to assess the effects of 2 years of sustained, embedded professional development on Grade 2 through Grade 5 elementary teachers' science-process skills, as defined by their ability to design experiments, and their perceptions of their capacity to lead students in scientific explorations. Specifically, the research questions were as follows: 1. To what degree are elementary teachers' understandings of science process skills impacted by 2 years of sustained professional development? 2. To what degree are elementary teachers' perceptions about science process skills impacted by 2 years of sustained professional development? Method Design The current study was part of a larger randomized field study of teacher and student leaming in science (Cotabish et al., 2010; Dailey et al., 2012). Only teacher data are reported in the current study. After 2 years of intervention consisting of 60 hours of summer institutes across two consecutive summers, and 60 hours of one-toone peer coaching across 2 school years for a total of 120 hours of professional development for each experimental teacher, data for both experimental and confrol teachers were compared. Confrol teachers

Fall 2011

19


THE EFFECTS OF A STEM PROFESSIONAL DEVELOPMENT INTERVENTION ON ELEMENTARY TEACHERS' SCIENCE PROCESS SKILLS

Table 1 Number of Experimental and Control Teachers by Grade Level and District

Disfrict

Group

Grade 2

Grade 3

Disfrict #1

E 5 5 C 6 6 Disfrict #2 E 4 4 C 4 4 Note. E = Experimental group. C = Control group. Intervention. STEM Starter teachers participated in two, week-long summer institutes focused on science content and delivery, specific curriculum units, technological applications, and differentiation of instruction. The two-summer institutes provided 60 hours of professional development necessary for the implementation of the science-curriculum units. Professional development involved teachers taking the role of students in the implementation of the curriculum units. An experienced science instructor led the teachers through the problem-solving units by modeling effective science instmction. Emphasis was placed on overarching concepts, such as change and systems, higher-order thinking skills, inquiry-based instruction, experimental design, and the use of technology, as recommended by VanTassel-Baska (1998). Professional development also involved the use of technology in the classroom to enhance leaming. Teachers were exposed to multiple Intemet resources aligned with their specific units. These Intemet resources provided content information for the student and teacher as well as multiple activities and educational games to motivate students. STEM Starters provided peer coaching on a weekly basis to the participant teachers. The peer coach was a former secondary chemistry/physics and gifted and talented teacher. Once school began, the peer coach was in each of the schools two to three times per week. She visited each class at least twice a month to provide support to the teacher. In the classroom, the peer coach modeled the lesson, assisted the teacher with instmction, and monitored and encouraged student involvement. Outside of the classroom, the peer coach made certain that all necessary science activity materials were in the schools and maintained contact with all teachers by phone or email to ensure their needs were being met.

Fall 2011

Grade 4

Grade 5

Gifted and Talented

2 5 2 2

1 1 1 1

3 0 1 0

Elementary Science Specialist 0 0 2 0

Instrumentation Teacher science process skills. To address the first research question, the Scoring Rubric for Scientific Processes - Adapted Fowler Test (Adapted from Fowler, 1990) was used to assess teachers' understanding of the design of science experiments. Although the instmment was designed with students in mind, the test is appropriate across a variety of age-levels due to the high ceilings (VanTassel-Baska, 2011), which would make it a suitable assessment for elementary teachers. The Adapted Fowler Test is an open-ended assessment of a student's ability to design a simulated-confrolled experiment in response to a scientific question. For example, students are asked to design an experiment to answer the question, "Are bees attracted to diet cola?" Responses describing a teacher's proposed design were scored across five criteria: (a) generates a prediction, (b) lists materials needed, (c) lists experiment steps/arranges steps in sequential order, (d) plans data collections, and (e) states plan for interpreting data for making predictions. Ratings ranged from no evidence (0) to strong evidence (3) with two additional points possible on one criterion resulting in 17 points possible. Understanding research design and the control of variables are considered key features of science competency (Klahr & Li, 2005). According to Adams and Callahan (1995), the general approach of the Adapted Fowler Test for assessing science process skills is requiring students to mimic intellectual processes used in real-world science. The same researchers examined the convergent and discriminant validity and found weak pattems of correlation that limit the use of scores for making decisions for individuals; however, the results did support use for decisions about groups of students. Reports of altemate forms, inter-rater, and infra-rater reliabilities ranged from .76 to .95 (Adams & Callahan, 1995). For the current study, raters

20



excluded from analysis only if they had missing data on the variables needed for that analysis. For the first research question, researchers conducted a one-way analysis of covariance (ANCOVA) to compare the Fowler post-test scores of teachers in the experimental and control groups, using the pre-test scores as a covariate to control for initial differences between the two groups on the measure. Researchers examined the homogeneity-of-slopes assumption and determined that the assumption was reasonable because the interaction term between the group variable and covariate was statistically nonsignificant [F(\, 50) = 0.04, p = .844], as was Levene's test for the homogeneity of variance assumption [F(], 52) = 1.31, p = .257]. Additionally, researchers conducted a trend analysis (Onwuegbuzie, Levin, & Ferron, 2011) of post-test gains in the experimental and control groups. For the second research question, researchers also planned to conduct ANCOVAs comparing the post-test scores of teachers in the experimental and control groups on the Teacher and Student Process Scales of the PASTeL using pre-test scores as a covariate. For both scales, data failed to meet the required assumptions. Researchers determined that the homogeneity-of-slopes assumption was not tenable on either scale because the interaction term between the group variable and covariate was statistically nonsignificant [Teacher Process Scale: F(l, 49) = 4.83, p = .033; Student Process Scale: F(l, 51) = 0.98, p = .326]. Data also failed to meet the homogeneity of variance assumption as measured by Levene's test [Teacher Process Scale: F(\, 52) = 1.31, p = .257; Student Process Scale: F(l, 51) = 0.01, p = .912]. Because the data did not meet the assumptions for ANCOVA, the researchers chose to conduct two dependent samples / tests between the experimental teachers' pre- and post-test scores on the Teacher and Student Process Scales using a Bonferroni adjusted alpha of .025.

participated in 6 hours of fraining and achieved an inter-rater reliability of .90. Teacher perceptions about science process skills. To address the second research question, the Perceptual Assessment of Science Teaching and Learning (PASTeL) was administered. The PASTeL assesses how teachers perceive their ability to teach, as well as their students' ability to learn science. The PASTeL consists of 50 items arranged in two scales, the Teaching Scale and the Student Learning Scale, with each item assigned to an overarching category: Content, Process, and Concept (Bracken et al., 2008). For the current study, the researchers utilized scores from the Process subscale defined as, "The understanding of fundamental science process skills such as: make observations, ask questions, learn more, design and conduct experiments, create meaning from the experiment, and tell others what was found" (Bracken et al., 2008, p. 8). Items such as, "I am confident in my ability to help students determine a testable question for a scientific investigation," appear on the teacher scale. Items such as, "when called upon, students compare and contrast results from multiple trials," appear on the student learning scale. The 4-point Likert-type scales are ordinal in nature, with a higher score indicating a greater degree of confidence in science teaching and/or student learning. Bracken et al. (2008) reported internal consistency reliability at .95 for scores from the total scale; estimates for the current study ranged from .95 to .97 across the pre- and posttest administrations of the Teaching- and StudentProcess Subscales. Data Collection Data from the two instruments (Fowler and PASTeL) were collected from the experimental teachers in June 2009, prior to the week-long summer institute, and in December 2010, the end of the second year of implementation. Data from the Fowler and PASTeL were collected from the confrol teachers at the beginning of the school year in September 2009, and in December 2010. The pretest was administered to the groups at different times because the experimental group attended a summer institute. STEM Starter project personnel were responsible for all data collection.

STEM Starters Impact on Teachers Research Question 1. To what degree are elementary teachers' understandings of science process skills impacted by 2 years of sustained professional development? Researchers conducted an ANCOVA test comparing the Fowler post-test scores (ME = 11.40, SDE = 4.24, Me = 5.90, SDc = 3.34) between the two groups using the pre-test scores (M^ = 7.68, SDE 4.10, «E = 25, Me = 4.62 SDc = 3.09, «e = 29) as covariates. The results revealed a statistically significant difference [F(l, 51) = 16.22, p < .001,77^ = .24] between the adjusted post-test scores for the two groups (ME = 10.68, Me = 6.52), with the experimental group scoring higher than did the

Results STEM Starters Impact on Teachers Data analysis. All data were entered into the Statistical Package for the Social Sciences (SPSS) version 20 for analysis. Researchers used casewise deletion of missing data; specifically, all cases remained in the sample and participants were Fall 2011

21

RESEARCH fN THE SCHOOLS

THE EFFECTS OF A STEM PROFESSIONAL DEVELOPMENT INTERVENTION ON ELEMENTARY TEACHERS' SCIENCE PROCESS SKILLS control group, indicating a large gain (rj^ = .24) in teachers' science process skills. Table 2 presents the means and standard deviations for each item on the Fowler for both administrations for each group. The consistency with which the post-test gains pertaining to the experimental group were larger than the post-test

gains pertaining to the confrol group across the six Fowler items was statistically significant (p = .016). This conclusion is valid even if the intermeasure correlations were as high as .15 (cf Table 1 in Onwuegbuzie et al., 2011).

Table 2 Means and Standard Deviations for Fowler Jtems and Total Score by Group

Item Generates a prediction Lists materials needed Lists experiment steps Arranges steps in sequential order Plans data collection States interpretation plan Total

Experimental Pre

Experimental Post

1.32(1.44) 1.04(1.17) 2.08(0.91) 1.50(0.60) 1.40(0.76) 0.52 (0.87) 7.68(4.10)

2.16(1.38) 2.28(1.02) 2.52 (0.82) 1.72(0.61) 1.92(0.94) 0.80(1.00)

Control Pre 0.38 (0.94) 0.38 (0.86) 1.62(0.94) 1.10(6.70) 1.00(0.80) 0.14(0.35)

Control Post 0.41 (1.05) 0.76(1.19) 1.93(0.75) 1.52(0.69) 0.93 (0.75) 0.34 (0.67)

11.40(4.24)

4.62(3.09)

5.90 (3.34)

Note. Nß=25. Ne = 29. The values listed in parentheses represent standard deviations. Results from a one-way ANCOVA revealed a statistically significant difference [F{\, 51) = 16.22, p < .001,77^ = .24] between the adjusted post-test scores for the two groups (A/i- = 10.68, Me = 6.52).

Research Question 2. To what degree are elementary teachers' perceptions about science process skills impacted by 2 years of sustained professional development? Results from the dependent samples t test revealed a statistically significant increase between participants' pre- (M= 0.54, SD = 0.18) and post-test (M = 2.86, SD = 0.84; scores on the Teacher Process Scale [i(25) = 13.60, p < .001, Cohen's d=2.67, 99% CI (1.97, 2.67)], with a large effect size refiecting a gain in teacher views of more than 2 points on a 4point scale. On the Student Process Scale, the results also revealed a statistically significant increase between participants' pre- (M = 2.13, SD = 0.69) and post-test (M = 2.80, SD = 0.65^ scores by teachers [i(25) = 3.74, p = .001, Cohen's d = 0.74, 99% CI (1.05, 3.74)], with a moderate effect size. The experimental groups agreed with teacher scale items associated with process (A/ = 2.86, SD = 0.54) and student scale items associated with process (M = 2.86, SD = 0.57). These results indicated that teachers agreed with statements such as, "I provide in class time for active scientific explorations," "My Fall 2011

students are eager to leam new science ideas," and "Students in my class draw reasonable inferences from data collected through scientific investigations." Results from the analysis of the PASTeL indicated that experimental teachers were becoming increasingly confident in their ability to lead science instruction and in their students' abilities to leam science, demonstrating that they were more confident in the area of science process. STEM Starters placed high emphasis on students' ability to design and to conduct experiments and in the teachers' ability to lead students in scientific investigations. Discussion After 2 years of intervention, results revealed a statistically significant gain on the Fowler in science process skills by elementary teachers in the experimental group when compared with teachers in the control group. These results indicated that experimental teachers were better able to design science experiments when presented with a realworld problem. The average scores for teachers in 22



summer inservice for teachers, could have infroduced a history threat to intemal validity. As Onwuegbuzie (2003) cautioned with most educational studies, population and ecological validity threats were concems. Specifically, the use of only two school disfricts and the small sample size of experimental (n = 30) and confrol teachers (n = 29) limit the extemal generalizability of the findings. Nevertheless, in light of the concems about elementary teachers and their failure to understand science processes deeply, the effects due to the STEM Starter intervention were noteworthy. As policy makers work to build a pipeline for science and mathematics talent, STEM Starters and similar programs that provide contentspecific professional development and in-classroom insfructional support are essential for needed improvements in elementary science classrooms.

the experimental group increased from 7.86 (pre) to 11.40 (post) out of 17 possible points on the Adapted Fowler Test. According to Supovitz, Mayer, and Kahle (2000), teacher facilitation of inquiry-type skills, such as the ability to design a science experiment, can be increased with content-focused sustained professional development. As noted previously by Beyer and Davis (2009), teachers' modeling of scientific behaviors, such as using science process skills to design an experiment, leads to a greater understanding of science content. After 2 years of intervention, experimental teacher perceptions in the areas of science process have changed. Examination of the subscale means for teachers in the experimental group revealed that teachers made progress in Process on both the Teacher and Student scales with gains ranging from 0.67 to 2.32 points per subscale on a 4-point scale. In other words, the experimental teachers have demonsfrated increased confidence in their ability to lead students in developing science process skills. Teachers with increased confidence in science teaching are more prone to encourage inquiryleaming in their classroom (Sinclair, Naizer, & Ledbetter, 2011). Furthermore, Liu, Lee, and Lim (2010) reported that positive teacher perceptions about inquiry-based instmction were associated with improved student achievement. Teacher beliefs and attitudes in science teaching, particularly the use of inquiry-based leaming, have a considerable impact on classroom instruction and student leaming (Choi & Ramsey, 2009). These teacher performance data support the implementation of intensive professional development in science process skills. In light of the concems about elementary teachers and their failure to understand science processes deeply, and their inability to profit from sporadic, low-intensity professional development, the effects documented in this study are noteworthy. Considering the importance placed on the STEM disciplines and the calls from policy makers to build a pipeline for science and mathematics talent, STEM Starters is a timely catalyst for developing such opportunities for elementary teachers and ultimately for their young students.

The lead editors for this article were Anthony J. Onwuegbuzie and John R. Slate.

References Adams, C , & Callahan, C. M. (1995). The reliability and validity of a performance task for evaluating science process skills. Gifted Child Quarterly, 39, 14-20. doi:10.] 177/001698629503900103 Appleton, K. (2008). Developing science pedagogical content knowledge through mentoring elementary teachers. Journal of Science Teacher Education, 7P, 523-545. doi:10.1007/sl0972-008-9109-4 Banilower, E. R., Smith, P. S., Weiss, I. R., & Pasley, J. D. (2006). The stattis of K-12 science teaching in the United States: Results from a national observation survey. In D. Sunal & E. Wright (Eds.), The impact of state and national standards on K-12 teaching (pp. 83-122). Greenwich, CT: Information Age Publishing. Beyer, C , & Davis, E. A. (2009). Supporting preservice elementary teachers' critique and adaptation of science lesson plans using educative curriculum materials. Journal of Science Teacher Education, 20, 517-536. doi:10.1007/sl0972-009-9148-5 Bitan-Friedlander, N., Dreyfus, A., & Milgrom, Z. (2004). Types of "teachers in fraining": The reactions of primary school science teachers when confronted with the . task of implementing an innovation. Teaching and Teacher Education 20, 607-619. doi:10.1016/j.tate.2004.06.007

Limitations of the Study The collection of baseline data from the experimental and confrol group teachers 3 months apart infroduced a potential insfrumentation threat to intemal validity. Ideally, baseline measures for both groups would have been collected at the same time; however, data collection from the control group teachers was not possible until the school year began. Additionally, intervening events, such as other Fall 2011

23


THE EFFECTS OF A STEM PROFESSIONAL DEVELOPMENT INTERVENTION ON ELEMENTARY TEACHERS' SCIENCE PROCESS SKILLS Bracken, B. A., Holt, K. A., Lee, M. L., McCormick, C. J., Reintjes, C. L., Robbins, J. I., & Stambaugh, T. L. (2008). Perceptual assessment of science teaching and learning preliminary examiner's manual. Williamsburg, VA: The College of William and Mary. Buczynski, S., & Hansen, C. B. (2010). Impact of professional development on teacher practice: Uncovering connections. Teacher and Teacher Education, 26, 599-607. doi:10.1016/j.tate.2009.09.006 Chase, A., & Wolfe, P. (1989). Off to a good start in peer coaching. Educational Leadership, 46(3), 37. Choi, S., & Ramsey, J. (2009). Constructing elementary teachers' beliefs, attitudes, and practical knowledge through an inquirybased elementary science course. School Science and Mathematics, 109, 313-324. doi:10.]lll/j.]949-8594.2009.tbl8101.x Committee on Science, Engineering, and Public Policy. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: National Academy Press. Cotabish, A., Dailey, D., Robinson, A., & Hughes, G. (in press). The effects of a STEM intervention on elementary students' science knowledge and skills. School Science and Mathematics. Cotabish, A., Robinson, A., MacFarlane, B., & Wood, B. (2010, April). Implementation of a gifted education STEM project: Teachers' initial perceptions, instructional behaviors, and knowledge of science content. Paper presented at the annual meeting of the American Education Research Association, Denver, CO. Council of Chief State School Officers. (2008). Does teacher professional development have effects on teaching and learning. (Grant No. REC 0438359). Refrieved from http://www.ccsso.org/projects/improving_ev aluation_of_professional development Dailey, D., Cotabish, A., Robinson, A., & Hughes, G. (2011, April). Interim effects of implementing a STEM initiative on elementary teacher perceptions and concerns about science teaching and learning. Poster presented at the annual meeting of the American Education Research Association, New Orleans, LA.

Fall 2011

Dailey, D., Cotabish, A., Robinson, A., & Hughes, G. (2012, April). Effects of implementing a STEM initiative on elementary teacher perceptions and concerns about science teaching and learning. Paper presented at the annual meeting of the American Education Research Association, Vancouver, BC. Duschl, R., Schweingruber, H. A., & Shouse, A. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: The National Academies Press. Eshach, H. (2003). Inquiry-events as a tool for changing science teaching efficacy belief of kindergarten and elementary school teachers. Journal of Science Education and Technology, 72,495-501. doi: 10.1023/B: JOST.0000006309.16842.c8 Fowler, M. (1990) The diet cola test. Science Scope, 7i(4), 32-34. Fulp, S. L. (2002). Status of elementary school science teaching. Chapel Hill, NC: Horizon Research. Retrieved from http://2000survey.horizonresearch.com/repo rts/elem_science/elem_science.pdf Gerard, L. F., Varma, K., Corliss, S. B., & Linn, M. C. (2011). Professional development for technology-enhanced inquiry science. Review of Educational Research, 81, 408448. doi:]0.3102/0034654311415121 Goodnough, K., & Nolan, B. (2008). Engaging elementary teachers' pedagogical content knowledge: Adopting problem-based leaming in the context of science teaching and leaming. Canadian Journal of Science, Mathematics, and Technology Education, 8, 197-216. doi: 10.1080?14926150802315130 Gustafson, B., Guilbert, S., & MacDonald, D. (2002). Beginning elementary science teachers: Developing professional knowledge during a limited mentoring experience. Research in Science Education, 32, 281-302. doi: 10.1023/A: 1020809916037 Kennedy, M. (1998). Form and substance of inservice teacher education (Research Monograph No. 13). Madison, WI: National Institute for Science Education, University of Wisconsin-Madison. Klahr, D., & Li, J. (2005). Cognitive research and elementary science instruction: From the laboratory, to the classroom, and back. Journal of Science Education and Technology, /4, 217-238. doi: 10.1007/sl 0956-005-4423-5 24



Koch, J., & Appleton, K. (2007). The effect of a mentoring model for elementary science professional development. Journal of Science Teacher Education, 18, 209-231. doi: 10.1007/s 10972-006-9036-1 Little, P. F. B. (2005). Peer coaching as a support to collaborative teaching. Mentoring and Tutoring, 7i(l), 83-94. Liu, O. L., Lee, H. S., & Linn, M. C. (2010). An Investigation of teacher impact on student inquiry science performance using a hierarchical linear model. Journal of Research in Science Teaching, 47, 807-819. doi:10.1002/tea.20372 Michaels, S., Shouse, A. W., & Schweingmber, H. A. (2008). Ready, Set, Science! Putting research to work in K-8 science classrooms. Washington, DC: The National Academies Press. Milner, A. R., Sondergeld, T. A., Demir, A., Johnson, C. C , & Czemiak, C. M. (2011). Elementary teachers' beliefs about teaching science and classroom practice: An examination of pre/post NCLB testing in science. Journal of Science Teacher Education. Advance online publication, doi: 10.1007/S10972-0] 1-9230-7 Murphy, C , Neil, P., & Beggs, J. (2007). Primary science teacher confidence revisited: Ten years on. Educational Research, 49, 415430. doi:10.1080/00131880701717289 National Science Board. (2010). Preparing the next generation of STEM innovators: Identifying and developing our nation's human capital (NSB-10-33). Retrieved from http://www.nsf.gov/nsb/publications/2010/n sblO33 Neuman, S. B., & Cunningham, L. (2009). The impact of professional development and coaching on early language and literacy insfructional practices. American Educational Research Journal, 46, 532-566. doi: 10.3102/0002831208328088 Onwuegbuzie, A. J. (2003). Expanding the framework of intemal and extemal validity in quantitative research. Research in the

Rice, D. C. (2005). I didn't know oxygen could boil! What preservice and inservice elementary teachers' answers to 'simple' science questions reveals about their subject matter knowledge. International Journal of Science Education, 27, 1059-1082. doi: 10.1080/09500690500069426 Showers, B. (1984). Peer coaching: A strategy for facilitating transfer of training (Report No. ED 271 849). Eugene, OR: Center for Educational Policy and Management, College of Education, University of Oregon. Showers, B., & Joyce, B. (1996). The evolution of peer coaching. Educational Leadership, 53(6), 12-16. Sinclair, B. B., Naizer, G., & Ledbetter, C. (2011). Observed implementation of a science professional development program for K-8 classrooms. Journal of Science Teacher Education, 22, 579-594. doi: 10.1007/s 10972-010-9206-z Supovitz, J. A., Mayer, D. P., & Kahle, J. B. (2000). Promoting inquiry-based insfructional practice: The longitudinal impact of professional development in the context of systemic reform. Educational Policy, 14, 331-356. doi: 10.1177/0895904800014003001 Supovitz, J. A., & Tumer, H. M. (2000). The effects of professional development on science teaching practices and classroom culture. Journal of Research in Science Teaching, 37, 963-980. doi: 10.1002/10982736(200011)37:93.0.CO;2-0 VanTassel-Baska, J. (1998). Planning science programs for high ability leamers. ERIC Clearinghouse on Disabilities and Gifted Education. Refrieved from http://www.ericdigests.org/] 9993/science.htm VanTassel-Baska, J. (2011). Implementing innovative curriculum and insfructional practices in classrooms and schools: Using research-based models of effectiveness. In J. VanTassel-Baska & C. A. Little (Eds.), Content-based curriculum for high-ability learners (2nd ed., pp. 437-465). Waco, TX: Pmfrock Press. Yoon, K. S., Duncan, T., Lee, S., & Shapley, K. (2008, March). The effects of teachers' professional development on student achievement: Findings from a systemic review of evidence. Paper presented at the annual meeting of the American Educational Research Association, New York, NY.

Schools, 100), 71-89. Onwuegbuzie, A. J., Levin, J. R., & Ferron, J. (2011). A binomial test of group differences with correlated outcome measures. Journal of Experimental Education, 19, 127-142. doi: 1080/00220970903352761

Fall 2011

25


Copyright of Research in the Schools is the property of Research in the Schools / MSERA and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.