Mathematical Thinking and Learning

Mathematical Thinking and Learning Investigating Why Teachers Continued Use and Sharing of an Educational Innovation After the Research has Ended --Manuscript Draft-Manuscript Number:

MTL-D-13-00005R2

Full Title:

Investigating Why Teachers Continued Use and Sharing of an Educational Innovation After the Research has Ended

Article Type:

Original Article

Corresponding Author:

Stephen Hegedus University of Massachusetts UNITED STATES

Corresponding Author Secondary Information: Corresponding Author's Institution:

University of Massachusetts

Corresponding Author's Secondary Institution: First Author:

Stephen Hegedus

First Author Secondary Information: Order of Authors:

Stephen Hegedus Sara Dalton Jeremy Roschelle William Penuel Margaret Dickey-Kurdziolek Deborah Tatar

Order of Authors Secondary Information: Abstract:

We investigated prospects for sustainable adoption and sharing of an educational innovation through survey research including online questionnaires and telephone interviews. This investigation is part of the Scaling-Up SimCalc experimental program, which combines dynamic representational algebra software (SimCalc MathWorlds®), with integrated curriculum, and small professional development workshops focused on how to use the software and how to integrate this intervention into the larger year-long curriculum. Teachers who (1) perceived the usefulness of the SimCalc professional development being consistent with personal aims and (2) perceived specific affordances of the software/curriculum to be valuable were more likely to both continue using the innovation after the research had ended and report sharing it with colleagues over time.

Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation

Cover Letter

COVER LETTER To the Editors of Mathematical Thinking and Learning: We faithfully submit the manuscript entitled: Investigating Why Teachers Continued Use and Sharing of an Educational Innovation After the Research has Ended

This note is to confirm that it has not been previously published or submitted elsewhere and the original research procedures were consistent with the principles of research ethics published by the APA. The authors are: Stephen J. Hegedus and Sara Dalton University of Massachusetts Dartmouth Jeremy Roschelle SRI International William Penuel University of Colorado Boulder Margaret Dickey-Kurdziolek and Deborah Tatar Virginia Tech

Regards Stephen Hegedus

Manuscript

SUSTAINED ADOPTION AND SPREAD OF AN EDUCATIONAL INNOVATION 1

Investigating Why Teachers Continued Use and Sharing of an Educational Innovation After the Research has Ended

Abstract We investigated prospects for sustainable adoption and sharing of an educational innovation through survey research including online questionnaires and telephone interviews. This investigation is part of the Scaling-Up SimCalc experimental program, which combines dynamic representational algebra software (SimCalc MathWorlds®), with integrated curriculum, and small professional development workshops focused on how to use the software and how to integrate this intervention into the larger year-long curriculum. Teachers who (1) perceived the usefulness of the SimCalc professional development being consistent with personal aims and (2) perceived specific affordances of the software/curriculum to be valuable were more likely to both continue using the innovation after the research had ended and report sharing it with colleagues over time.

IMPORTANCE OF TEACHERS IN INNOVATION

2

Introduction Researchers have made steady progress in developing innovative educational interventions that combine curriculum materials with integrated technological tools that improve student learning of standards-based content in mathematics and science and as well as in-depth subject knowledge that will remain important throughout students’ lives. These interventions have been called “coherent curricula” (Roseman, Linn & Koppal, 2008) and they represent carefully designed products of collaborations between researchers and practitioners. In mathematics, such materials—in the form of replacement units—have been tested in rigorous experimental studies and have been shown to be effective in supporting student learning of complex mathematical concepts (Clements, Sarama, Wolfe, & Spitler, 2013; Author, XXX; Sarama, Clements, Starkey, Klein, & Wakeley, 2008). An ongoing challenge to the success of such interventions, however, is to create changes in classroom practice that are both sustainable—the teacher continues to employ the intervention in the manner intended by its designers—and scalable—the use of the intervention is shared beyond its initial users in ways congruent with its designed intent (Clements, Sarama, Wolfe, & Spitler, 2013; Coburn, 2003). Part of the challenge lies in the transition from “hothouse” research environments, where support and funding is plentiful, to everyday practice, where teachers and schools are subject to multiple competing demands (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004). A related problem is that as interventions move beyond the immediate involvement of their developers, shifts in practice or “lethal mutations” can occur such that the meaning of the original materials is lost (Brown & Campione, 1996). Koellner, Schneider, Roberts, Jacobs, and Borko (2008) report on the scaling up of preparing instructional leaders to facilitate the Problem-Solving Cycle model of professional


3

development for middle school mathematics teachers. In this scale-up work, the authors report the instructional leaders started the Problem-Solving Cycle professional development without an agenda, lacking knowledge about the Problem-Solving Cycle materials, and lacking their own previous experiences with the materials. This, at the start, proved detrimental to the scale-up efforts and Koellner et al. reported this as a “lesson learned” in their implementation of a scaleup model for teacher professional development. These implementation challenges are a critical hurdle in the progress of education reform (Rogan, 2007) and one we noted in our implementation. One obstacle to successful implementation is the breadth of the state standards to which teachers must attend. There is evidence indicating that teachers are not well equipped to make decisions about material selection with respect to standards (Schmidt & Prawat, 2006) and as a result may opt for coverage of topics that are superficially aligned with standards but that are a “mile wide and an inch deep” (Schmidt, Wang & McKnight, 2005). Even if developers assure teachers that interventions are aligned with standards, teachers may still make choices that lead to shallow implementation with respect to the designers’ original intent (Lin & Fishman, 2006). In response, there has been widespread call for Professional Development (PD) to increase teacher capabilities with complex interventions (Borko, Elliot, & Uchiyama, 2002; Cohen & Hill, 2001). We have found that approaches emphasizing alignment can be overly “top-down” (Author, XXX), and have sought to understand how teacher perspectives on alignment may shape their enactment choices and responses to new interventions and associated PD. We do not question the importance of ensuring that interventions are aligned with standards, but we believe that the teachers’ understanding(s) also provide a crucial lens for the interpretation of standards and response(s) to them (Author, XXX; Spillane, 2000; Spillane, Reiser & Reimer, 2002).


4

Research has identified some of the conditions for successful continued implementation and sustainability of interventions after the research ends. Given the importance of teachers’ perceptions of coherence, it is not surprising that one of the conditions is providing ongoing opportunities for teachers to make sense of the intervention’s goals and to reflect on how they fit within the larger goals of their school and district (Coburn, 2005; Horn, 2011). Studies point to the importance of leaders supporting this sensemaking activity as well at the school level (e.g., Coburn, 2005). This sensemaking takes place within collegial groups, and to sustain interventions beyond an initial funding period requires that groups engage in conversations at some level of depth and in ways that foster ongoing reflection on practice (Coburn, Russell, Kaufman, & Stein, 2012; Franke, Carpenter, Levi, & Fennema, 2001). Notably, these past studies all examine sustainability in the context of school- or district-wide reform; the current study involved volunteer teachers, some of whom were the only teachers implementing the intervention in their school. Clements et al. (2013) examine sustainability across Pre-K to kindergarten to first grade with the aim of scaling up support for fidelity to their intervention through the phases of introduction, initial adoption, implementation, and institutionalization. Clements and colleagues (Clements, 2012; Clements, Sarama, Wolfe, & Spitler, 2013) have examined sustainability and diffusion of a research-based model for scale-up called TRIAD (Technology-enhanced, Research-based, Instruction, Assessment, and professional Development). This model includes professional development and the Building Blocks mathematics intervention in the early grades. In previous work, the authors used multilevel models across the three years to compare experimental conditions in order to measure the sustainability of outcomes. They found that continuing TRIAD helped children maintain the benefits of the pre-K intervention. The statistical


5

analysis showed TRIAD was a significant predictor of student learning in each year, and when sustained over the three years. Our measure of sustainability, what we refer to as “stick”, does not include an account of fidelity of implementation, however. Our account of “stick” is based on teachers’ reporting if they are still implementing the SimCalc resources in their classroom, several years after the intervention. Clements (2012) argued the TRIAD model may support diffusion of innovations and reports instances of districts hiring project staff to provide professional development, continued summer work on TRIAD by the teachers, adoption of materials at different preschools, and diffusion in the media such as local news paper articles, and National Public Radio. Our work aligns more closely with Rogers (1995) definition of diffusion: when people inside organizations become advocates for a particular innovation and share it with others. SimCalc research was funded under the “Interagency Educational Research Initiative” (IERI), a program in which the National Science Foundation, U.S. Department of Education, and National Institute of Child Health and Human Development jointly participated. The purpose of IERI was to scale up projects which would conduct “rigorous, well-controlled, large-scale empirical studies” and demonstrate “which proposed education approaches are in fact most effective in practice” (see http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5486). IERI emphasized randomized controlled trials (see Schneider and McDonald, 2007) but also included research on sustainability (e.g. Songer, 2007). Within a randomized controlled trial (RCT), teachers’ knowledge, attitudes and perceptions can be modeled as moderating and mediating variables, which may influence outcomes. Although the Scaling RCTs included such variables, stable findings about how teacher perceptions may have influenced outcomes were not identified (Author, XXX). The additional data collections and analyses reported here were deliberately


6

intended to complement the focus on measuring student impacts (“relative advantage”) in the RCT with information to better understand how the innovation was used and spread after the experiment ended, and thus to enable a more well-rounded understanding of scaling up. In the Scaling-Up SimCalc research program, teachers were recruited to participate in a wide-scale evaluation of SimCalc’s integration of professional development, representational technology, and paper curriculum. The experimental study results found that students learned more advanced mathematics when their teachers implemented the SimCalc approach compared to their peers in classrooms where their teachers did not (Author, XXX). One year after the formal research studies concluded, we wanted to conduct a diffusion study to investigate if teachers were still using and/or sharing the SimCalc resources and, if so, why teachers were doing so. While certain researchers may have an intuitive sense of what factors are important, few rigorous studies have investigated specific factors that contribute to the diffusion of new innovations. Even fewer studies have investigated factors that contribute to the diffusion of new innovations following a randomized control study such as the Scaling-Up SimCalc program. We focused our diffusion study on investigating why participating teachers would still be using the resources (i.e., sustainability or what we refer to more dynamically as “stick”) and why they would share them with colleagues (what we refer to as “reported sharing”). The diffusion of innovations is facilitated when people inside organizations become advocates for a particular innovation and share it with others, particularly those that are thought leaders within the organization (Rogers, 1995). In education, as in other fields, diffusion happens through networks, particularly networks of teachers within schools (Frank, Zhao, & Borman, 2004). We investigate whether or not teachers reported sharing resources related to SimCalc. The efficacy of sharing, however, is contingent upon the credibility of the person sharing, and educators value


7

the direct experience of peers over more distal sources of knowledge (Kennedy, 2005; Smylie, 1989). Therefore, our “stick” measure—in addition to being a measure of sustainability—is a good indicator of the likelihood that the teacher’s collegial sharing will result in effective diffusion. Given the SimCalc program was more than just a piece of software but an integrated suite of representationally-rich technology, curriculum materials, and professional development (Author, XXX) we wanted to investigate whether particular features of such an approach could explain why teachers reported the continued use of, and/or reported sharing of, the resources. Hence, we operationalized this investigation through the following research question: Do teachers continue to use and/or share the SimCalc resources with other teachers after the research program ends and if so, are there specific features of the SimCalc resources that are significant factors for sustained use and diffusion?

Literature Review Rogers (1962) created a framework for understanding how innovations spread throughout a society, based on his studies of how agricultural techniques spread. His work distinctively highlighted the role of social networks in spreading innovations and brought precision to the everyday notion of spreading ideas by “word of mouth.” He identified five key factors in diffusion of innovation: (1) relative advantage (effectiveness); (2) compatibility with other work practices; (3) complexity (or conversely, ease of use); (4) how easy it is to try the innovation before fully committing, and (5) observability (how easy it is for a member of a social network to show the innovation to other member). Rogers’ framework has since been usefully applied to the area of educational technology


8

(Dooley 1999; Surry and Farquhar, 1997): teachers are interested in relative advantage, but also are concerned with an innovations’ compatibility with requirements of their jobs, with complexity (given their limited time available to learn to use innovations), as well as opportunities to try and observe innovations with low risk. For example, Ely (1990) used diffusion of innovation as a framework to study educational technology adoption in Chile, Peru and Indonesia and found that such adoption was more of a “movement” than an organized process and that while many favorable elements were in place, participation of teachers and leaders was important to the spread of innovation and depending on their knowledge and perceptions. Ely (1990) focused on dissatisfaction with the status quo, knowledge and skills, availability of resources, availability of time, rewards or incentives, participation, commitment, and leadership as the key conditions for spread of educational technology. A related framework, the Concerns-Based Adoption Model (Anderson, 2002), also emphasizes teachers’ concerns as they influence adoption and suggests that adoption proceeds in stages. A theoretical synthesis (Straub, 2009) reviews these and related frameworks and argues for understanding adoption of educational innovations as a complex, social process based on how individuals form malleable perceptions which influence their adoption decisions. Consequently, our research explores how teachers’ perceptions shaped their decisions regarding use and spread of SimCalc after the formal experiment ended. The National Center for Improving Student Learning and Achievement (NCISLA) in Mathematics and Science (2004) published a report on the scaling-up of innovative practices in which they provide insight into what travels and what is necessary for travel (spread). The report acknowledges the importance of engaging teachers in the design process, that spread occurs through people and networks, and that institutional support is of critical importance to successful


9

spread of an innovation. SimCalc research has deep roots in “design-based research” traditions (Author, XXX) – a tradition that connected with diffusion of innovation research (Larson & Dearing, 2008) but also includes specific emphasis on development of learning theory (The Design-Based Research Collective, 2003). In design-based research (Kelly, Lesh & Baek, 2008), investigators develop and field designs of educational innovations (often involving technologies) in classrooms in order to simultaneously advance the relevant learning theory and the instructional design (see a review in Wang & Hannefin, 2005). As with diffusion of innovation research, teachers are seen as key actors in design-based research; in fact, in SimCalc research teachers were seen as key stakeholders in the “curricular activity system” which was being designed (Author, XXX) and included elements of technology, curriculum materials, and teacher professional development. Design-based research with technology in mathematics education has particularly emphasized the link between dynamic representations and conceptual understanding (Heid & Blume, 2008). Thus one could expect that teacher resonance with the conceptual understanding goals of the curriculum and the specific role of the technology (Becker, 2000) would be an important factor in adoption and spread. Prior Research Program that Ended The Scaling-Up SimCalc research program investigated the scale-up of an innovative integration of technology, curriculum, and teacher professional development aimed at improving mathematics instruction in grades 7 and 8. Three experiments comprised the program: 1. Seventh-Grade Experiment with seventh-grade content, students, and teachers. A random-assignment experiment with 105 seventh-grade teachers (1621 students) in year 1 of the study.


10

2. An embedded Seventh-Grade quasi-experiment (1048 students) following a delayedtreatment design in year 2 with a sub-set of the teachers from year 1 in which control teachers (also called the delayed-treatment teachers) began to use the SimCalc replacement unit, and treatment teachers (also called the immediate-treatment teachers) continued to use it. 3. The Eighth-Grade Experiment in year 2 (825 students), which was designed to extend the findings of the Seventh-Grade Experiment to eighth-grade content, students, and teachers and investigate a train-the-trainers approach to teacher professional development. The results have been broadly disseminated in published articles (see Author, XXX). Detailed information about the project is also available through a technical report series (see http://math.sri/com). The central finding of this research demonstrated that SimCalc enables a wide variety of teachers in a diversity of settings to extend students’ learning to more advanced mathematics (Author, XXX). For the seventh-grade study—the seventh-grade quasi-experiment (delayed treatment teachers across Years 1 and 2)—and the eighth-grade study, the main effects were statistically significant and showed that students in the treatment group (or Year 2) learned more than students in the control group (or Year 1). HLM revealed main effects with student-level effect sizes of .63, .50, and .56, respectively (see Figure 1). {Insert Figure 1 here} Across all three studies, the gain differences between the two groups occurred mostly on the portion of the tests that indicate the SimCalc interventions create an opportunity to learn advanced mathematics beyond the typical curriculum. The student-level effect sizes of the


11

portion of the test consisting of items that tested mathematics beyond what is covered at grade level (M2) were .89, .69, and .81, respectively. In contrast, the student-level effect sizes of the portion of the tests consisting of items that tested mathematics typically covered at grade level (M1) were .10, .13, and .19, respectively. This effect was significant only at the .05 level in the seventh-grade quasi-experiment and not at all in the other studies. Such success offered the rationale to conduct a diffusion study post-experimental phase to understand the reasons why teachers continue to use or share the SimCalc resources. We expected that in focusing on perceptions of the SimCalc program any findings would be germane to a wider variety of settings. Research Design Our diffusion study focused on addressing our research question through survey research methods. We first developed and implemented an online questionnaire instrument focused on teachers’ perceptions and use of SimCalc. We then used telephone interviews with participating teachers to support the online questionnaire findings to offer richer descriptions of exactly what this teacher population perceive is useful from the SimCalc PD and what features of the SimCalc resources are perceived valuable with respect to teaching and learning mathematics. We will refer to teachers using or sharing the “resources” throughout the paper but we wish to stress that the software and curriculum materials were designed as an integrated product. In addition, the professional development focused on important aspects of how to implement this integrated product in an effective and mathematically driven way and how to sustain classroom discourse related to the activities that previous studies had informed. Online Questionnaire The project leadership and an external advisory board developed an initial set of


12

questions based on the use and adoption of SimCalc and interviewed 17 teachers from prior projects. This preliminary work outlined particular areas to focus our main online questionnaire on including teacher perceptions of professional development, support for implementation, barriers to implementation, continuing use of the intervention resources, and communication with peers relating to the intervention resources, which guided our selection of items validated in prior studies of teacher Professional Development (PD) (Garet, Porter, Desimone, Birman & Yoon, 2001), implementation, and the scaling up of innovations (Author, XXX). These areas are also consonant with work done on the diffusion of innovations reviewed earlier but offer more specificity with respect to implementation of educational technology in schools. In addition, we used items from the TexTeams Survey (a PD initiative in Texas) and the “Post-Intervention” logs that we had asked teachers in the SimCalc study to complete. New questionnaire items created by the research team pertained to social interactions, continued use of current SimCalc activities, reasons for no longer continuing to use SimCalc activities, components of SimCalc which are deemed valuable, and teachers’ perceived importance of SimCalc. From this work, we created a 15-item online questionnaire with 158 sub-items (i.e. 1a,1b, …), which became our main survey research instrument for our diffusion study. A small number of pre-service student teachers took the online questionnaire to obtain a measure of approximately how long it would take (approximately 20 minutes) and assess clarity. The full questionnaire, its logic map, and complete source information for each item can be found in a technical report (see Author, XXX). Due to its length we cannot include it here but include a subset of the questions in the appendix, which are the focus of our main analysis. The questionnaire was administered using the on-line SurveyMonkey tool (see www.surveymonkey.com) with a front page outlining the purpose of the project.


13

We contacted all 189 teachers who participated in the original SimCalc studies to request their participation in the questionnaire. Teachers were contacted several times until there was a negligible change (2%) in the numbers of new responders. Seventy-nine teachers responded and completed the online questionnaire. Three responders had started the questionnaire twice. Their responses were not consistent and so both responses for these 3 responders were removed leaving 76 questionnaire responders, on whom we focus our analyses. We conducted a non-response analysis in order to determine whether the teachers who responded to our questionnaire differed in any meaningful way from teachers who did not respond. Using independent-samples t-tests, we compared initial student scores (t(145.729) = 1.647, p > .05), gain scores from pre-post testing (t(146) = -.772, p > .05), the geographic distribution of teachers (data from the original experiment) (t(178) = 1.516, p > .05), and Campus-level Socio-Economic Status (SES) (t(146) = -.371, p > .05). None of these comparisons indicated a significant difference between response and non-response groups, giving us confidence that the results of this study are not biased as a result of response patterns. Telephone Interviews All participating teachers were interviewed within 10 days of their administration of the post-test to their students. We used a semi-structured interview protocol that focused on various dimensions regarding the intervention from general questions on what went well/not well to specific issues regarding the use of the resources and the PD workshops. If teachers did not spontaneously bring up their students, class, school, region, technology, curricula, testing, prior training, beliefs about teaching, and career goals and choices, they were prompted, usually drawing upon something the teacher had already said. Very few prompts were required. Short interviews were caused by teacher schedule issues rather than reluctance.


14

During the first year of the study, 95 interviews were conducted, which amounted to 5163 minutes. There were 48 interviews with delayed treatment (control) teachers for a total of 2399 minutes; the remaining 47 interviews were with treatment teachers for a total of 2764 minutes. The average control interview lasted 50 minutes, with the shortest being 21 minutes and the longest lasting 1 hour and 55 minutes. The average treatment interview lasted 59 minutes, with the shortest being 29 minutes and the longest lasting 2 hours and 7 minutes. After the first year of the study, 27 of the year-one participants decided to discontinue their participation and did not complete the second year of the Scaling-Up SimCalc study. We gathered a total of 68 interviews with the remaining participants, which amounted to 3958 minutes of audio data. There were 31 interviews with delayed treatment teachers for a total of 1837 minutes, who were reporting on their first use of SimCalc. There were 37 interviews with immediate treatment teachers for a total of 2121 minutes, who were reporting on their second use of SimCalc. The average year-two delayed treatment interview lasted 59 minutes, with the shortest being 28 minutes and the longest being 2 hours and 4 minutes. The average year-two immediate treatment interview lasted 57 minutes, with the shortest being 25 minutes and the longest being 2 hours and 20 minutes.

Results We attend to our main research question by creating a simple index from the online questionnaire items focused on essential features of the SimCalc resources (software, curriculum and PD) that we expect effective SimCalc teachers to perceive as important. We examined if this simple index correlates with sustained use or reported sharing with other teachers. We outline the analytical process that led to this index and its usefulness in determining significant groups of


15

teachers in our study. We then turn to the interview data to look at a sub-sample of these teachers and present descriptive insights into what teachers perceive as useful or valuable in the SimCalc resources to investigate if teachers on the low or high scale of the index differ in qualitatively distinct ways. Questionnaire Analysis Questionnaire items #11 and #14 are dichotomous response items to measure sustained use of SimCalc resources and sharing of resources respectively. Item #11 asks, “Are you still using all or part of the SimCalc curriculum?” and involves logic that allows the responder to highlight which parts of the SimCalc resources they are still using (i.e., are still sticking with). Item #14 asks, “Have you discussed or shared what you learned with other teachers in your school or department who did not attend SimCalc professional development?” and also involves logic to allow the responder to highlight what they were discussing or sharing. These are our two main dependent variables. From the overall population of respondents, 48% (which we refer to as “Stickers”) reported on Item #11 that they are still using the SimCalc resources (indicating sustainability), and 67% (which we refer to as “Sharers”) had shared information with a colleague about the resources (as an indicator of spread—all stickers were sharers, and 12 responders were just sharers). Overall, given that no incentives were provided to teachers for these behaviours and the resources were not formally adopted or required in their schools, we found this rate of continued use to be encouraging. The questionnaire covered many aspects of use and adoption of SimCalc resources which included institutional pressures that prevent adoption, coherence with existing curriculum and practices, and perceived utility overall. We wished to investigate whether specific features of the


16

intervention resources were predictors of sustained use and indicators of diffusion. The research and development team in consultation with external expert users of SimCalc resources reviewed the questionnaire and selected specific sub-items that we regarded as important factors in understanding how to use and implement SimCalc resources as well as its specific benefits. Since the intervention is an integrated product of software, curriculum, and PD we specified what is valuable, important, and useful with regards to classroom practice, PD, and students’ learning and engagement. In generating such a set of sub-items from the 158 potential questions we would create a measure that could be used to evaluate how close a SimCalc teacher’s perception of the resources were to the designers’ intentions. We explain how this measure was established and used to investigate our research question of whether there are specific features of the SimCalc program that are significant factors for sustained use and indicator of diffusion. From the 15 questions of the questionnaire, the 158 sub-questions (i.e., item 1a, b, c …, etc.), became 158 variables. We selected 17 such variables to define a “model SimCalc teacher” that defines such expectations in response to our questionnaire eliminating questions that were deemed irrelevant to someone valuing SimCalc resources at large, or desirable/undesirable for any teacher and not specific to values and principles behind the SimCalc approach. Most of the sub-items were Likert-response questions on a scale of 0 (low) to 4 (high). For example, question 8 asked: “How useful were each of the following activities of the SimCalc PD workshop to you in preparing to teach the unit?”

{Insert Table 1 here}

For each of these activities included (items 1, 2, 5, 6, and 8), the anticipated response of a teacher more in line with the model SimCalc teacher would yield high agreement that these items


17

are important and useful in a SimCalc PD workshop that prepares teachers to teach the SimCalc unit. A high agreement response would indicate the responder finds that group dynamics are beneficial and necessary in the SimCalc classroom. These types of activities included are fundamental to the classroom structure of the SimCalc activities as intended by the material designers, PD leaders, and researchers. While we do not discount the benefits that can arise from reviewing student work, developing lesson plans, and making presentations to the group in the PD workshops, these activities were not specifically linked to someone valuing SimCalc resources and the PD workshop in preparation for teaching the unit. Table 2 presents a list of all 17 sub-items (variables) that were selected, and the rationale for selection. Question and variable columns are listed for the items selected. The response column is what we believe to be a desirable response, and did not determine selection of responses in the survey. {Insert table 2 here or in appendix} Three of these 17 variables were further removed for statistical reasons. Responses to each of these 14 items were standardized as z-scores and accumulated to create a SimCalc Teacher Index variable per teacher, hereon called the “SCT Index”. The higher the score on the index, the more aligned a teacher’s mindset is with the researchers and designers of the SimCalc innovation in terms of what they perceive is valuable, important, and useful with regards to classroom practice, PD, and students’ learning with respect to the specific features of the SimCalc learning environment. The internal consistency of the index using 14 variables is high (α = 0.909). The index was used to compare those responders who are still using SimCalc resources with those who are not and those responders who are sharing SimCalc resources with those who


18

are not. The SCT Index of those teachers who continued to use the SimCalc resources (M = 1.74, SE = 1.287) is significantly different, t(60) = -1.740, p = .044, from the SCT Index of those teachers who did not continue to use the SimCalc resources, (M = -2.13, SE = 1.889). There is a medium sized effect, d = .44, r = .21. The SCT Index of those teachers who shared the SimCalc resources (M = 1.77, SE = 1.132), is significantly different, t(60) = -2.974, p = .002 from the SCT Index of those teachers who did not share the SimCalc resources, (M = -5.59, SE = 2.581). There is a large sized effect, d = .82, r = .38. A Principal Components Analysis (PCA) was conducted on the 14 variables of the SCT Index to investigate whether there were more degrees of variance to the index that would be useful in our analyses as we differentiated between a “sticker” and a “non-sticker” or a “spreader” and a “non-spreader.” To motivate the reader, this process was profitable as it produced two main components or sub-scales to measure differences within our sample and reduced the necessary questionnaire dataset further. Following this iterative procedure, two components accounted for 63% of the total variance and these were distributed across six of the original 14 variables for Component 1 and four of the original 14 variables for Component 2. Following the removal of four more variables, we ran tests to examine the suitability for running a final PCA on the remaining variables. The Kaiser-Meyer-Olkin measure of sampling adequacy for the sample (.784) was large (Kaiser, 1974). Bartlett’s test of sphericity was significant, χ2 (45) = 338.05, p < .0001. Diagonals for the anti-image correlation matrix were all above .5. Following a Varimax rotation and forcing two components, all variables were significantly correlated (p < .01) with at least one other variable. These statistics indicate this structure represents our dataset well.


19

We define Component 1 as “Perception of the usefulness of SimCalc PD with respect to personal teaching needs,” which accounts for 45.72% of the variance and Component 2 as “Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics,” which accounts for 17.62% of the variance. The internal consistency for each of the two components is high indicating a sound set of sub-scales to measure teacher’s perception of usefulness and perception of valuable features of the SimCalc program (Component 1: α = 0.86; Component 2: α = 0.82). The first component describes how teachers perceived the usefulness of the professional development piece of the SimCalc program, with specific active ingredients that cohere with their expectations for integrating technology into classrooms. The second component is focused on how teachers perceive what specific features of the SimCalc software and curriculum are valuable, such as linking tables and graphs with simulations, student mathematical justification with their work, and activities structured to investigate slope as rate, as examples. These components describe important characteristics in understanding the importance of the SimCalc program, valuing critical components of the program including the software, curriculum, and professional development, and how it relates to classroom practice. These characteristics of innovation can be used to help explain reported sharing (Koellner, Schneider, Roberts, Jacobs, & Borko, 2008). Independent samples t-tests (assumptions for normality and homogeneity of variances were met) were conducted on each component to see if there are statistically significant differences between stickers & non-stickers and sharers & non-sharers. SCT Index scores for stickers were significantly higher than non-stickers for the first but not the second component: Component 1: t(60)= -1.742, p = .043; Component 2: t(60) = -.813, ns. For component 1 this


20

represents a medium sized effect, d = .45, r = .20. For component 2 this represents a small sized effect, d = .22, r = .11. SCT Index scores were significantly higher for sharers than non-sharers for both components: Component 1: t(60) = -2.186, p = .016, Component 2: t(60) = -2.021, p = .024. For component 1 this represents a medium sized effect, d = .66, r = .27. For component 2 this represents a medium sized effect, d = .50, r = .21. All tests were one-tailed. Table 2 illustrates the 10 items that make up the 2 components and the remaining 4 items eliminated following the PCA. In summary, participating teachers who are most likely to share SimCalc resources have a high SCT index which provides an important diffusion factor with regards to particular teachers’ mindsets. These teachers perceive the overall usefulness of the SimCalc PD and its alignment with present practices as well as specific features and functionalities of the software and curriculum as valuable with respect to teaching and learning mathematics.

Interview Data The interviews with teachers were not focused on continued use or sharing of the resources per se but are a useful dataset in offering richer insights into teachers’ perceptions of the SimCalc intervention and resources with respect to valuable features of the software, curriculum, and professional development including its alignment with their present practices. In order to reduce the number of interviews to analyse we wished to examine teachers with high and low SCT index scores to investigate qualitative differences in their specific utterances. We ranked the normalized SCT index scores from our datasets and examined all teachers whose scores were 1.5 standard deviations from the mean as extremes. There were six in each group (to which we refer to as high SCT index and low SCT index) totalling 12 teachers whose


21

complete interviews were analysed. In terms of which studies the teachers were part of across three years, for the Low SCT Index group, 3 out of 6 were delayed treatment (dT), i.e., they did not use SimCalc in Year 1 and the other 3 were immediate treatment (iT). One of the individuals from the iT group dropped in Year 1 and did not continue with the study. Of the remaining individuals, only one remained in the study through Year 3. In the High SCT Index group 4 out of 6 were immediate treatment (iT), while the other 2 were delayed treatment (dT). We believe this is a balanced sub-sample of the teachers in the extremes of the SCT index. Based on the results of the PCA of the online questionnaire, two researchers coded these 12 interviews for teachers’ perceptions of the usefulness of, and their perceived values of the SimCalc intervention (curriculum, software, and PD) with respect to teaching and learning mathematics. Two members of the research team coded the transcripts with a high inter-rater reliability ( = .9). Differences were examined by the project lead and re-coded as necessary. We examined the teacher utterances closely to investigate potential qualitative distinctions between the low and high SCT index teachers in terms of how they articulated the perceived usefulness of the SimCalc PD and the perceived value of SimCalc resources with respect to teaching and learning mathematics. There were more statements in the high SCT Index group that articulated specific elements that were purposefully designed and situated within a theoretical tradition, e.g. linking simulations with graphs and tables, enabling students to model real-world situations in mathematically important ways (Table 4). Statements from the low SCT index (Table 3) group were generic and vague, relating to the overall experience vs. specific learning episodes either in the classroom or in their professional development.


22

{Insert Table 3 and Table 4 here} For component 1 many of the utterances from high SCT Index teachers on the perceived usefulness of the SimCalc PD were directly related to his or her teaching practice. The high SCT Index teachers discussed specific pieces from the PD workshop that they integrated into their teaching practice, examples include: asking open ended questions, facilitating discussions as opposed to lecturing, asking students to clarify what they know and how, or why they did something, and utilization of the specific SimCalc teacher questioning prompts provided in the resources. This can be contrasted with the utterances on the perceived usefulness on the SimCalc PD from the low SCT Index teachers. The low SCT Index teachers perceived usefulness in terms of coverage, the SimCalc resources were useful for their coverage of content, and filling a PD workshop requirement. Two of the low SCT Index teachers did not perceive the SimCalc strategies presented in the PD workshop as particularly useful, one teacher cited time constraints as a problem in the classroom. An additional teacher responded to the lack of attention to assessment of learning in the PD workshop. We realize, for a teacher, time, coverage, and assessment are practical concerns and are not to be dismissed. However, the utterances from the low SCT Index teachers regarding the SimCalc PD workshop lacked reference to classroom strategies, probing questions, or facilitating whole class discussions which are key components both within the SimCalc resources, and a focus during the PD workshop. For component 2, there were several utterances focused on the perceptions of what teachers thought the software and curriculum does, and how it relates to what their students should be learning. There were several comments made (particularly in the low SCT index group) that illustrated a misunderstanding of how the resources should be used, and what was being represented mathematically. Many of the utterances from the high SCT Index teachers on


23

the perceived features of SimCalc resources as valuable with respect to teaching and learning relate to specific elements that were purposefully designed and situated within a theoretical tradition, e.g. linking simulations with graphs, tables, formulas, and/or narratives, enabling students to model real-world situations in mathematically important ways, and enabling students to start and participate in mathematical discussions (Table 3 for low SCT Index utterances and Table 4 for high SCT Index utterances). Three of the high SCT Index teachers also responded that they perceived the SimCalc resources enabled their students to perform at a higher level of thinking. With respect to perceived features of SimCalc resources as valuable for teaching and learning, utterances from the low SCT Index teachers related to student knowledge needed prior to the start of a lesson, establishing classroom rules for use of the resources, classroom management issues, and length of time to cover the material. There was a response from a low SCT Index teacher who perceived the SimCalc resources enabled their students to perform at a higher level of thinking and cited graphing of the distance formula as a major benefit for student learning. However in the response, this teacher did not mention a connection between or across these representations, rather that these separate representations went further into an understanding of slope because there was more than one representation given in the resources. The SimCalc approach, however, focuses on links or connections across representations, a connection that was mentioned in the utterances of several of the high SCT Index teachers. Discussion Overall, these results strengthen the claim that teachers who have been trained/supported to use a particular set of mathematically rich educational resources (including technology and curriculum) and can perceive the affordances of the innovation, are more likely to continue to use and share the resources (The National Center for Improving Student Learning and


24

Achievement in Mathematics and Science, 2004). The perceptions held by SimCalc teachers are both about the nature and value of the intervention. From the principal component analysis, we found two areas that were important: (1) teachers perceive the usefulness of the SimCalc professional development as valuable with respect to their teaching needs as a whole, and (2) teachers perceive specific features of SimCalc software and curriculum as valuable with respect to teaching and learning mathematics. Given the design of the index this demonstrates an alignment of the teachers’ perceptions of what is valuable in the SimCalc resources with the original intentions of the designers. SimCalc resources were more likely to be continued in use and be shared with teachers who reported a higher agreement with these factors, a finding that aligns with Becker’s (2000) important factors for adoption and spread discussed earlier. This has implications for professional development and scale-up. We propose that in teacher education explicit attention should be made to: 

The expectations of how a technology resource should be used,



What the designers value and believe are important functionalities in the technology resource as well as their theory of change with respect to how students can interact and learn, and



How it can build on existing State frameworks.

These are important factors in establishing a healthy mindset for teachers involved in replacing or reforming their curriculum in a sustainable way. Our SCT Index—which models a set of positive perceptions towards the effectiveness and use of the SimCalc program—correlates well with “stick” and “share”. We have illustrated that the components of the SCT Index differentiate those who stick with the SimCalc program or not, and those who choose to share it or not.


25

We have attempted to investigate and record what particular key ingredients are necessary in the mindset of a teacher that might ensure the continued use of the SimCalc resources. The SCT Index does factor reliably into two components. Each of these components is a significant factor in explaining why someone will continue to use the SimCalc resources and/or share it. It is important to reflect on what this actually means in practice. It appears that the subpopulation of SimCalc teachers who continue to use a resource after research participation ends can be defined by how they value the resource. Thus, schools and program developers should aim not only to seek teachers’ program compliance and fidelity of implementation, but also their positive valuation of the resource provided. Further, we believe schools and program developers should plan for additional PD activities aimed at sustaining use; this PD might include a focus on what is unique and important about the specific research and the innovative design elements, so as to enable teachers to feel ownership over its unique active features which extends the necessary diffusion elements highlighted in the literature examined earlier with more specificity. For some teachers, a sense of alignment between their values and what the resources offer may come naturally; for others, seeing the relationship from their values to what a new resource offers may take more effort. We highlight that the resource was effective for students whether or not their teacher readily perceived this fit; hence to serve the maximum number of students who could benefit from a resource, developers may have to put energy not only into creating a resource that works with a wide variety of teachers, but also into expanding the range of teachers who perceive a fit between their teaching practice and the usefulness of the professional development, functionality of the software, alignment of the curriculum, fit of the pedagogical expectations and how all these elements combine into a coherent, easy-to-use new resource.


26

Acknowledgements XXX References Anderson, S. E. (2002). Understanding teacher change: Revisiting the concerns-based adoption model, Curriculum Inquiry, 27(3), 331-367. Becker, H. J. (2000). How exemplary computer-using teachers differ from other teachers: Implications for realizing the potential of computers in schools. Contemporary Issues in Technology and Teacher Education [Online serial], 1(2), 274-293. Borko, H., Elliot, R., & Uchiyama, K. (2002). Professional development: A key to Kentucky’s educational reform effort. Teaching and Teacher Education, 18(8), 969-987. Brown, A. L., & Campione, J. C. (1996). Psychological theory and the design of innovative learning environments: On procedures, principles and systems. In L. Schauble & R. Glaser (Eds.), Innovations in learning: New environments for education (pp. 289-325). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Clements, D. H., Sarama, J., Wolfe, C. B., & Spitler, M. E. (2013). Longitudinal evaluation of a scale-up model for teaching mathematics with trajectories and technologies persistence of effects in the third year. American Educational Research Journal, 50(4), 812–850. Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.


27

Coburn, C. E. (2005). Shaping teacher sensemaking: School leaders and the enactment of reading policy. Educational Policy, 19(3), 476-509. Coburn, C. E., Russell, J. L., Kaufman, J. H., & Stein, M. K. (2012). Supporting sustainability: Teachers' advice networks and ambitious instructional reform. American Journal of Education, 119(1), 137-182. Cohen, D. K., & Hill, H. C. (2001). Learning policy: When state education reform works. New Haven, CT: Yale University Press. Dooley, K. E. (1999). Towards a holistic model for the diffusion of educational technologies: An integrative review of educational innovation studies. Educational Technology & Society, 2(4), 35-45. Ely, D. P. (1990). The Diffusion and Implementation of Educational Technology in Developing Nations: Cross-Cultural Comparisons of Indonesia, Chile and Peru. ERIC report ED331469. Fishman, B., Marx, R., Blumenfeld, P., Krajcik, J., & Soloway, E. (2004). Creating a framework for research on systemic technology innovations. Journal of Learning Sciences, 13(1), 43-67. Frade, C., & Machado, M. C. (2008). Culture and affect: Influences of the teachers’ values on students’ affect. In S. Alatorre, T. Rojano & A. Sepúlveda (Eds.), Proceedings of the Joint Meeting of PME 32 and PME-NA 30 (Vol. 3, pp. 33-40). Morelia, Mexico: Program Committee. Frank, K. A., Zhao, Y., & Borman, K. (2004). Social capital and the diffusion of innovations within organizations: Application to the implementation of computer technology in schools. Sociology of Education, 77(2), 148-171.


28

Frank, K. A., Zhao, Y., & Borman, K. M. (2004). Social capital and the diffusion of innovations within organization: Application to the implementation of computer technology in schools. Sociology of Education, 77(2), 148-171. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945. Heid, M. K., & Blume, G. W. (2008). Algebra and function development. In Heid & Blume (Eds.) Research on technology and the teaching and learning of mathematics, (55-108). Charlotte, NC: Information Age. Horn, I. S. (2011). (Ir)rational claims about learning: Teacher discourse and the construction of school mathematics. Paper presented at the ISCAR Congress, Rome, Italy. Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31-36. Kelly, A. E., Lesh, R. A., & Baek, J. Y. (Eds.) (2008). Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. London: Routledge. Kennedy, M. M. (2005). Inside teaching: How classroom life undermines reform. Cambridge, MA: Harvard University Press. Franke, M. L., Carpenter, T. P., Levi, L., & Fennema, E. (2001). Capturing teachers' generative change: A follow-up study of professional development in mathematics. American Educational Research Journal, 38(3), 653-689.


29

Larson, R. S., & Dearing, J. W. (2008). Design research and the diffusion of innovations. In AE Kelly, RA Lesh, & JY Baek (Eds.), Handbook of design research methods in education (pp. 511 – 534). New York: Routledge Lin, H-T., & Fishman, B. (2006). Exploring the relationship between teachers’ curriculum enactment experience and their understanding of underlying unit structures. Paper presented at the 7th International Conference for the Learning Sciences, Mahwah, NJ. The National Center for Improving Student Learning and Achievement (NCISLA) in Mathematics and Science. (2004). Scaling up innovative practices in mathematics and science. Madison, WI: Carpenter, T. P., Blanton, M. L., Cobb, P. Franke, M. L., Kaput, J., & McClain, K. Philipp, R. A. (2007). Mathematics teachers’ beliefs and affect. In F. K. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 257-315). Charlotte, NC: Information Age Publishing. Rogan, J. (2007). An uncertain harvest: A case study of implementation of innovation. Journal of Curriculum Studies, 39(1), 97-121. Rogers, E. M. (1995). The diffusion of innovations. New York: The Free Press. Rogers, E. M. (1962). Diffusion of innovations. New York: The Free Press of Glencoe. Roseman, J. E., Linn, M. C., & Koppal, M. (2008). Characterizing curriculum coherence. In Y. Kali, M. C. Linn & E. Roseman (Eds.), Designing coherent science education: Implications for curriculum, instruction, and policy (pp. 13-36). New York: Teachers College Press. Sarama, J., Clements, D. H., & Jacobs Henry, J. (1998). Network of influences in an implementation of a mathematics curriculum innovation. International Journal of


30

Computers for Mathematical Learning, 3, 113-148. Sarama, J., Clements, D. H., Starkey, P., Klein, A., & Wakeley, A. (2008). Scaling up the implementation of a pre-kindergarten mathematics curriculum: Teaching for understanding with trajectories and technologies. Journal of Research on Educational Effectiveness, 1(2), 89–119. Schmidt, W. H., & Prawat, R. S. (2006). Curriculum coherence and national control of education: Issue or non-issue? Journal of Curriculum Studies, 38(6), 641-658. Schmidt, W. H., Wang, H. C., & McKnight, C. C. (2005). Curriculum coherence: An examination of US mathematics and science content standards from an international perspective. Journal of Curriculum Studies, 37(5), 525-559. Schneider, B. L. and McDonald, S. K. (2007), Scale-up in practice: An introduction. In Schneider, B. L. and McDonald, S. K. (Eds). Scale-up in education: Issues in practice, Vol. 1., 1-14. Smylie, M. A. (1989). Teachers' views of the effectiveness of sources of learning to teach. Elementary School Journal, 89(5), 543-558. Songer, N. B. (2007). Rethinking sustainability of curricular innovations: Notes from urban Detroit. In Schneider, B. L. and McDonald, S. K. (Eds). Scale-up in education: Issues in practice, Vol. 2., 165-182. Spillane, J. P. (2000). Cognition and policy implementation: District policymakers and the reform of mathematics education. Cognition and Instruction, 18(2), 141-179. Spillane, J. P., Reiser, B. J., & Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72(3), 387-431.


31

Straub, E. T. (2013). Understanding technology adoption: Theory and future directions for informal learning. Review of Educational Research, 79(2), 625-649. Surry, D. W., & Farquhar, J. D. (1997). Diffusion theory and instructional technology. Journal of Instructional Science and Technology, 2(1), 24-36. The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23.

SUSTAINED ADOPTION AND SPREAD OF AN EDUCATIONAL INNOVATION 32

Figure 1. Student mean scores at the student level by experimental group and participation.


33

Table 1 Selection of Sub-items From One Question Activity included in the survey

Selected

1. Participated in a whole-group discussion or session

Included

2. Participated in a small-group discussion or session

Included

3. Made presentations to the group 4. Developed a lesson plan 5. Discussed instructional techniques 6. Practiced using software 7. Reviewed student work 8. Making connections between SimCalc resources and standards

Not included Not included Included Included Not included Included


34

Table 2 Selected items for SCT Index and final components from PCA Question

Sub-item (Variable)

Response

Rationale

Component 1: Perception of the usefulness of SimCalc PD with respect to personal teaching needs Question 8 - How useful were each of the following activities of the SimCalc PD workshop to you in preparing to teach the unit?

PDS1_AL1 Participated in a wholegroup discussion or session (0-5)

Desirable response: 3, 4 or 5 (high)

The responder finds that group dynamics are beneficial to the correct implementation of SimCalc.

Question 8 - How useful were each of the following activities of the SimCalc PD workshop to you in preparing to teach the unit?

PDS1_AL2 Participated in a smallgroup discussion or session (0-5)




PDS1_AL5 - Discussed instructional techniques (0-5)




PDS1_AL8 - Making connections between SimCalc resources and standards (0-5)



Question 6 - Reflecting on your SIMCALC professional development, to what extent was the professional development characterized by the following?

PDS2_CG1 Consistent with your goals for your professional development (1-4)

Desirable The exposure to SimCalc & PD is believed by the response: 2 or 3 responder to be in alignment with their (high) advancement as a teacher.

Question 6 - Reflecting on your SIMCALC professional development, to what extent was the professional development characterized

PDS2_CG7 - Designed to integrate technology

Desirable The exposure to SimCalc and PD is believed by the response: 2 or 3 responder to assist in bringing technology and

IMPORTANCE OF TEACHERS IN INNOVATION by the following?

into your teaching (1-4)

35 (high)

teaching together in their classroom.

Component 2: Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics Question 12 - Were the following parts of the SimCalc Intervention useful in terms of your teaching and your students’ learning and engagement?

STK_VAL1 - Use of simulations in software (0-4)

Desirable Considering this is one of the fundamental aspects response: 3 or 4 of SimCalc, it should be believed by the responder (high) that this is valuable.

Question 12 - Were the following parts of the SimCalc Intervention useful in terms of your teaching and your students’ learning and engagement?

STK_VAL2 - Use of interactive graphs in software (0-4)



STK_VAL6 - Students working in pairs or as part of a group (0-4)

Desirable Considering a favored, or intentional aspect of the response: 3 or 4 curriculum made for SimCalc has students creating (high) and exploring on their own, as opposed to a demonstration, the responder should find high value in this.


STK_VAL7 Curriculum in conjunction with the software (0-4)

Desirable Considering the curriculum was designed for use response: 3 or 4 with the software, and for their standards, the user (high) should find high value, and believe there is a connection between the software and curriculum.

Question 7 - To what extent did the SIMCALC professional development prepare you to help students do the following?

PDS2RAT4 - Solve problems that have more than one correct answer (1-4)

Desirable SimCalc PD changed the mindset of the responder response: 2 or 3 to be better prepared and more open to the (high) possibility of more than one correct answer. Often, the opposite is true with traditional methods, materials and textbooks

Question 7 - To what extent did the SIMCALC professional development prepare

PDS2RAT 7 - Explain Desirable What was learned during training and use of why a procedure response: 2 or 3 SimCalc effected the responders ability to students used worked to coherently explain varying procedures produced by

Removed following PCA but used in SCT Index


36

you to help students do the following?

solve a problem (1-4)

(high)

students back to all students in a way they would understand.


PDS1_AL6 - Practiced using software (1-4)




STK_VAL3 - The curriculum materials (1-4)

Desirable Considering the curriculum was aligned to response: 3 or 4 standards and designed for the software, a (high) responder should find high value in the curriculum/software.

Selected by Research team but removed from final SCT Index for statistical reasons Question 1.1 – To what extent have you made each of the following changes in your teaching practices as a result of your experience with SIMCALC and SIMCALC professional development?

IMP_PRACTICEC – The instructional methods I employ (0-3)

Desirable This question-variable-response was selected response: 2 or 3 because the exposure to SimCalc and correct (high) SimCalc methods and practice via PD had an impact, as perceived by the responder, to be a large change. This indicates a change in pedagogical methods because of SimCalc. Presumed positive.

Question 1.1 – To what extent have you made each of the following changes in your teaching practices as a result of your experience with SIMCALC and SIMCALC professional development?

IMP_PRACTICEE – The ways I use technology in instruction (calculator or computer) (0-3)

Desirable This indicates a change in pedagogical methods response: 2 or 3 with respect to technology in the classroom, due to (high) the exposure to SimCalc. Presumed positive.

Question 5 - How would you describe your SIMCALC class’s coverage of the student workbook?

IMP_COV1 - We completed the entire book from start to finish (0-1)

Desirable response: 1 (yes)

This indicated a belief that the curriculum was coherent and aligned with standards and frameworks required by TAKS. It also indicated that supplemental material is not needed and completion of the material is fundamental.


37

Table 3 Utterances for Low SCT Indexed Teachers Component 1: Perception of the usefulness of SimCalc PD with respect to personal teaching needs “I was glad to be in the SimCalc group because the TexTeams stuff is just like your regular old run-of-the-mill workshop where they give you activities for each TEK. But with the SimCalc, that one - just doing the workbook covers so much even in one day you cover so much stuff that I enjoyed it better. I didn't do any of the TexTeams stuff. I just did the SimCalc stuff. I’m not opposed to the TexTeams stuff but I just saw that SimCalc covered so much of the material that that’s all I did.” “I do what I’m instructed to do. We’re required by our district to do five days of outside workshops a year and then we get those back as waiver days throughout the school years. And I really like that because then you can find your own and you can find things that apply to you and your field rather than just going to an all day speaker who has nothing to do with math.” [On using any of the SimCalc strategies presented in the PD workshop] “I used a little bit, not a whole lot. Normally I do ask for more depth and get their, or why do you think this is the answer and well, as far as having actual time sometimes, I just felt rushed and didn’t have time to do that so I didn’t do it. Normally, I do.” “You know, I have really good experiences towards the workshop. … And he [superintendent] doesn’t like really like to go to workshops during the school year that’s and we do a lot in the summer for this [SimCalc PD], it really helps me out on my workshop hours.” [On using any of the SimCalc strategies presented in the PD workshop] “Umm. I don’t really remember any strategies; we just kind of went over the whole work, looked like the kids did. So I don’t remember any particular strategies.” [On the SimCalc PD workshop] “Well, and I think that probably the other teachers had the same problem – I don’t know what they’re grading – what they are mandated to do for grades, but it’s tough when so much of it is group work to put a grade on it.” Component 2: Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics


38

“The proportionality thing just – unless the teacher brought it up and sort of keyed the students in on ‘okay, do you see because “x” changes and because “y” changes and these are proportional’ and unless the teacher said that, it was not brought up in the lesson until the very end.” “SimCalc differed because it dealt with graphing because everything we did we graphed it and before, you know, I'll just - they'll just have an equation. Distance equals rate times time and they'll have a short story problem on it and they'll write down their formula, they'll plug in the numbers they know, and then they'll solve the equation and this just always had a graph to it. So, it was even more of, you know, got them to some of those higher level thinking skills because it did involve the slope aspect of things and how the vertical changed over the horizontal and that type of stuff. So, it differed because it went further than what I usually do.” “We started off kind of rough, because it was my students’ first time in the lab and so you know the computer problems and making sure they weren’t playing with the mouse or playing with the graphs when they were supposed to be listening, things like that. Once we finally established the rules and they understood what I wanted, then the other days went kind of smoother.” “I know one of the assignments – it said, ‘go back to some of the other graphs in this workbook and find the slope of the lines.’ We tried to do that at first and it just didn’t seem conducive to what we were trying to do to find the slope. It just never seemed to work out right. So, that’s why we went ahead and made up our own sheet.” “I think it helps them a lot. I really think it helps them. I think that 3½ weeks is too long. But I wanted to do the whole book, that was my goal for myself because I told them guys I really want to do this whole book; you know well ma’am we are tired of doing this and it got to the third week and its like we are almost finished.” “The hard part was moving on the graph where you had to move the lines and drag and make the lines look what they wanted to look like … I had to guide them because a lot of them weren’t familiar since they don’t have computers at home.” “Managing the class for me was difficult… I found I couldn't handle it, I went back to the room with a computer and an LCD projector and as a class we all worked with it.”


39

Table 4 Utterances for High SCT Indexed Teachers Component 1: Perception of the usefulness of SimCalc PD with respect to personal teaching needs “This has got to be very much facilitating. You have to be a facilitator, not a lecturer. I don’t think it would work if you stood up there and just lectured.” [In response to best teaching strategies for PD workshop] “I did use them [the strategies presented in the PD workshop] very much. You know I think I used those somewhat before, but making them more formal. I think the big thing was like, if the students said they didn’t understand or something, the deal of saying ‘what do you know’ or ‘what do you get’ not quite that way, but clarifying, having them clarify what they already understand.” [On using the SimCalc classroom questioning prompts] “Well, it went pretty good. But I would like to do more though, a lot more.” [On using any of the SimCalc strategies presented in the PD workshop] “Yes, yes. In fact, I am trying to improve my teaching skills by asking more and more open-ended type questions where they have to think and look at what they have done and ‘why did you do it this way?’ yes I am doing that more and more.” “I don't know about the other teachers but what helped me a lot was the training that we took before during the summer and then we went over all the notes and the booklets that we had. And I don't know if that helped because at that point we had a lot of questions and all the presenters they were very good at answering all our questions.” Component 2: Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics “Each of the unit lessons they are taught, they do probe for the higher level thinking skills, like explain or describe or come up with the story that matches this as scenario and those are things that are helpful because our lessons don’t always in the curriculum that we use.” “Now, I did the simulations with them but I didn’t say anything. They would come back and – it seemed like the simulations would start the


40

discussions … you’ve got to be the facilitator and not the lecturer. Make them figure things out. I think that’s what makes it so meaningful.” “I have a real positive feeling about the whole unit and I think some of the maths was difficult for my students but I really think that the kids were able to do some things that they would not have been able to had they not had the software and worksheets. They could perform at a higher level I think because of the software.” “Of course, we teach rate and proportionality as proportions and n over 6 is equal to 8 over 12 and ‘what is n?’ and something like that. In fact, as far as my class is concerned, I’ve never connected a graph to proportionality until this year with this lesson. … I showed them with the proportionalities and they seemed to catch onto that real quick.” “The thing that I think impressed me the most as an overall - was the fact that we started out with our constants and then we went into the constants of proportionality and they saw the rates. Then when we got into actually calling it slope, they actually made that connection. We're talking about the same thing, just different situations and that to me was the most phenomenal thing for 7th graders to feel comfortable enough with slope to actually make connections in other areas.” “It's just a much higher level. I feel like they have a greater understanding than I've ever had my students achieve.” “The students were able to graph better and make tables. They were able to find the missing numbers more easily and some things like that.” “Going from a simulation to creating a graph improved every time, or going from a table to a graph. Their graphing ability really improved.” “What I really liked about this year is the kids had a good grasp on the rate of change of two different slopes and they also understood going forward and going backward, I thought that was something they picked up on. I was very pleased about that because they understood that time still goes on whether you are standing still or moving, time is still continuing and that’s why it’s a flat line.” “They can graph, they got that down, but as far as looking at a graph, they have probably gone from a distance over time graph and solved for speed in science. But they have not connected that the speed would be the slope, that that translates to the slope of the line.” “I have a young man that's in one of my classes. And he has failed one of two 6 weeks in my class because he just didn't want to do the work. Well, on the post-test, he made an 80, which I was so impressed so I ran up to the cafeteria and I said, ‘look! You made an 80 on this test!' Well, I


41

went on back down to my room and here he comes and he looks at me and he says, ‘I understand how to figure out speed.' He said, 'but does speed and acceleration have anything to do with one another?' And that came out of the blue. I don't know where it came from. If we can catch the interest of these kids that aren't very motivated, then this is a wonderful unit.”

Response to Reviewers

Responses to Diffusion Paper 2013 Mathematical Thinking and Learning Editor: You have addressed some of the reviewers' concerns adequately but not others. In responding to some of their concerns, your paper has lost potentially important data. As one of the reviewers commented to me, "Additionally, and more seriously, the author chose to address several reviewers' concerns with certain aspects of the qualitative analysis by dropping that part of the study, without recognizing that the reviewers were saying that some of the more interesting data were in that section and that the use of both qualitative and quantitative methods greatly enhanced the paper. Now we are left with a "smaller" paper." We have decided to include part of the original interview analysis back into the paper but focused on “Perceptions” as this was the main dimension that the reviewers believed came out of the questionnaire. We believe this is the most important piece of the analysis that triangulates the results from the survey work with the telephone interview data well within space constraints. Other concerns include whether your article provides a substantial contribution to the field, and some difficulties in drawing specific conclusions partly because of a lack of clear definitions/descriptions about the qualities of teacher perceptions. Given a primary focus now on perceptions we believe that the contribution is more specific and substantial. See our responses and revisions below to defend this argument. We have substantiated our findings by situating them with respect to several literature focused on scale-up and lack of diffusion studies following the reviewer’s suggestions. Reviewer 2 provides a very comprehensive review with suggestions on how you can improve your paper including better framing and theoretical development, and developing your key constructs more effectively. The removal of some qualitative data also makes it difficult to interpret(sic) the tables, as both reviewers note. We agree and respond to the many issues raised by reviewer #2 in our specific responses below and within our revised paper. Finally, there are numerous grammatical errors that will need correcting in your revision. // //

Reviewer #1: The reported research investigates why teachers continue to use and share an educational innovation. This is an important topic as "few rigorous studies have investigated specific factors that contribute to the diffusion of new innovations" (p. 2). Indeed, we need more information on the specific factors and we need to understand the reasons for teachers' stick and spread. However, I felt that I did not learn much new from this paper. The main finding is that teachers continue to use and share the innovation perceive the professional development and the software useful (or value it). This kind of general finding, that is common sense, does not contribute substantially to investigating why teachers continue to use and share an educational innovation. Put like that we agree but we felt that various specific factors were not highlighted in a significant way, which we have attended to in the new revision: 1. This is part of an RCT 2. We have discovered specific factors related to perceptions of the specific features and design principles valued by the developers of the integrated software/curriculum/PD package that associated with whether a teacher continues to use or share the package. There is some discussion in page 19 on factors that were related to the teachers' stick and spread. The three such factors were: "(1) teachers perceive the professional development they received as aligning with the value of the SimCalc materials in the classroom, (2) teachers see the integration of software and curriculum as valuable, and (3) teachers see the professional development they received as valuable to their teaching as a whole." I would like to see much more detailed discussion of these factors. In addition, I had difficulties to understand how the three factors are concluded from the components presented in Table 2. For example, I did not see which of the questions presented in Table 2 were related to the first factor. Maybe the authors just meant that the teachers' perceived the PD activities as useful. This has been re-phrased into two components that are related to the two dimensions: 1. Perception of usefulness of SimCalc PD with respect to personal teaching needs 2. Perception of value of SimCalc resources with respect to specific design principles to enhance teaching and learning mathematics Table 2 now outlines which items relate to which components. So the reader can see the exact items that compose each of the components resulting from the PCA. Additionally, to clarify, we have used the term ‘component’ instead of ‘factor’ everywhere in the paper as we had completed a principal components analysis as opposed to a different type of factor analysis. We hope that this will clear up some

of the confusion between ‘component’ and ‘factor’ in the presentation and discussion of the analysis. This is different from the colloquial use of ‘factor’. Difficulties to draw more specific conclusions may be partly due to lack of definitions/descriptions of what do the authors mean by perceptions and values and how does this relate to existing research. For example, the component 2 in Table 2 is about values according to the authors. However, it could also be interpreted to be about teachers' perceptions of usefulness of the SimCalc intervention. We agree and have removed values as a component. We are primarily focused on teachers’ perceptions of the usefulness of the SimCalc PD and what they perceive is valuable (not their values) – a subtle distinction. This is what the survey was focused on and is what the interviews can help triangulate. I still think that it could be possible to write a new paper about the topic. Maybe the following suggestions give some ideas how to work on the paper: Make substantial conclusions more visible and add discussion on them if possible. Be more specific about the qualities of teacher perceptions that are related to stick and spread and connect these to the existing literature. If the statistical analysis does not offer making substantial conclusions, reconsider presenting results about interviews that could help to understand why teachers continue to use and share an educational innovation. We believe our changes (commented above) attend to this suggestion. Reviewer #2: As a reader of the original manuscript, I was pleased to see many improvements, especially the richer framing of the article in terms of a more general problem facing the field, rather than primarily in terms of the SimCalc Project. However, there were also some problems that were identified in the initial review that have not been addressed adequately, the most problematic being the authors' decision to drop the qualitative data analysis from the original review, rather than to address the problems that I and other reviewers identified. By choosing this route, the authors have weakened the results and significance of the paper. That said, the (quantitative) results are presented more clearly in the revised manuscript than in the original. Thank you. We agree that eliminating some of the qualitative results weakened the paper and were very hesitant in doing this. The main reason was that the PVE framework was too much to apply (as it did create problems for other reviewers). Based on feedback we believe that the results are most significantly situated in teachers’ perceptions and have re-framed the results focused on this singular dimension which has allowed us to bring back in (subject to word restrictions) the most important part of our interview statements regarding “perceptions” that are more closely aligned with our quantitative findings.

Below, I review the revised manuscript in terms of the three main issues identified in my original review. I also added a fourth section on some grammatical problems, which was not in my first review, since I was not reading at such a fine-grained level in the first round of review. 1. Framing and Theoretical Development A. Diffusion of Innovation Nice improvement in terms of the inclusion of the diffusion of innovation literature. The revised manuscript is also better motivated in terms of more general issue that affect multiple projects instead of being framed only in terms of SimCalc. However, I can still see the tension that the authors may have experienced in making this transition and recommend the following changes to further strengthen the framing of the article: * Omit the first paragraph * Omit the heading "Framing of the Study" * Start with "Researchers have made steady progress in developing?" Done * The flow of the argument starts to get in trouble on page 4. I recommend that after the first full paragraph (the identification of obstacles to successful implementation), the authors include a paragraph on successful implementations (to identify what has been done but what is left to be known), followed by a paragraph stating what the field still needs to know (the general problem). Then it makes sense to fold in some of what is in the first paragraph with parts of the bottom of p. 4 and top of p. 5 with a restatement of the problem in terms of the SimCalc Project. Done Additional note: As I read the other reviews (and the authors' responses), I was struck by the authors' comment to Reviewer 1 regarding citations in the discussion section: "We are not sure why this [referring to the use of citations] is necessary in a discussion section." To my way of thinking, citations in a Discussion are one indicator that the authors have come back to the needs/issues presented in the literature, which are presumably addressed by the particular study. There are a few references to the larger issues in the discussion, but this section seemed to fall back on concerns of the Simcalc Project and could be further strengthened by connecting it more strongly to the framing at the beginning of the paper. B. Context

I appreciate the inclusion of my suggestions regarding the scaling up initiatives by IES and NSF (now on pp. 8-9), but I don't think it fits well in that section (which would be strengthened by simply calling it Diffusion of Innovations and DesignBased Research, instead of Literature Review). Instead, the funding context can be used to motivate interest in the issues discussed in this article, and as such, would fit better in the new introductory section. Done p. 8, second paragraph. It doesn't make sense to start with "A third methodology?" Diffusions of innovations research is not a methodology. In general, I thought this entire paragraph needed to be refined and moved into the introductory section. Done C. Relevant Research in Mathematics Education Much better! I would, however, move the paragraph that starts at the bottom of page 7 up into the introductory section. It doesn't fit well where it is, which is in a subsection that is supposed to be focused on design-based research. Most of the first paragraph on page 8 does not fit where it is. This argument needs a bit more refining here. Done

D. Development of Constructs. My comments on "spread" were not addressed satisfactorily. The authors appear to define the term "spread" on p. 4 without building the case for it (as I recommended in the first review). They write "..and why they would share them with colleagues (what we refer to as "spread"). But "spread" is used in the diffusion of innovations literature to refer to the actual adoption of an innovation by groups of individuals. It's possible to build a case for using the term "spread," but I would expect acknowledgement of how it is used in other literature and a strong case for using the term differently here. A simpler fix would be to use the more accurate term of "reported sharing." "Spreader" is used later in the paper, with is not as potentially misleading as "spread" (although it's really "self-reported spreader"), because "spreader" suggests a focus on the person verbally sharing and doesn't imply a claim about subsequent adoption by others. Though "sharers" could also be used, and sharing is in the title; just make it clear that the sharing is what teachers report, not what was observed. Then accordingly, remove "spread" and add "reported" to "sharing": Agreed. We have updated the language to reference “sharers” and not “spreaders” as we did not follow up on the actual adoption in cases where the materials were shared. We mark this as done.

o Title should be more accurate, e.g., "Investigating Why Teachers Report Continued Use and Sharing of an Educational Innovation after the Research has Ended" o Abstract needs to acknowledge that this is survey research. o Last line of abstract should acknowledge that this is "reported" use and sharing rather than documentation of either. o Omit "spread" in first line in abstract All done o Use more careful language each time "spread" or "share" is used. Examples: * P. 5. Replace "We therefore investigate diffusion or spread through a measure of whether people share resources related to Simcalc" with a more accurate sentence such as "We investigate whether or not teachers report sharing resources related to Simcalc." * P. 5. Replace "...we wanted to investigate whether particular features of such an approach could explain why teachers continued to use and/or share the resources" to "...we wanted to investigate whether particular features of such an approach could explain why teachers reported the continued use and/or sharing of the resources." All done * Do a search on "share" and "spread" to find the others. Done. We have referred to sharing and not spreading throughout now. We agree this is a better way of describing our dimensions.

* I also asked the authors to define and motivate the three constructs of perceptions, expectations and value. Instead, they removed the qualitative data analysis, which included some of the more interesting data in the paper, and consequently made the article less compelling. We have re-included Perceptions now. And we have aligned the qualitative data with the questionnaire results, i.e. the two components that explained the majority of the variance in the online questionnaire instrument. This has resulted in the narrowing of our focus in the teacher interview data to perceptions, which we have defined, hopefully to your satisfaction. We believe this helps to support the questionnaire findings qualitatively and supports the overall message of the paper which focuses on perceptions. To do this, we focused on the portions of the coded

interview data related specifically to perceptions of PD and perceptions of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics. This also excludes the second cycle of coding that differentiated between positive and negative statements with relation to perceptions. Rather than focusing on positive and negative statements with relation to perceptions, we focused on the utterances coded as perceptions of PD and similar instruction (related to component 1) and perceptions related to curriculum, students, and technology (related to component 2). This was not a new analysis of the interview data. 2.Methods section A.Online survey. OK. B. Telephone survey. The authors omitted the telephone interview. I think this weakens the interest and significance of the study. We have re-included “Perceptions” now. See reasons above and in the new version. We have re-added back in the interview survey design 3.

Analysis Section

A. Table 1. I found the new Table 2 more clear. B. Triangulating Teachers Responses with Interview Data. As I mentioned in the first review, the interview data were important to include, because they offer a way to connect the meaning of various aspects Table 1 (now Table 2) with verbal statements from the teachers. I also mentioned that the most interesting material, from my point of view, was the teachers' utterances, which were then buried in Table 3. I don't think omitting the interviews adequately addresses these comments. We have re-included Perceptions now. See reasons above and in the new version. 4. Grammatical Issues

I found a surprising number of grammatical problems, for a revised manuscript. I am including corrections up through p. 7, in the hopes that this will be sufficient to show the authors the general nature of the problems throughout the manuscript: * P. 3. Scaling and Scale-up should not be capitalized * P. 3. Introduce the acronym PD the first time the term "professional development" is used. * P. 4. Professional Development should not be capitalized. PD is introduced as an acronym here but the authors have already used it several times. * P. 4. The sentence "We have found that approaches emphasizing ?" would be strengthened by splitting into two sentences and rewriting the first without "we have" since the citations are not the authors' work. * P. 4. The following sentence (last sentence in that paragraph) should also be rewritten without the "we." * P. 6. Follow APA style for citations. Should be (Dooley, 1999; Surry & Farquhar, 1997). * P. 6. Split extremely long first sentence (which carries over from 4 lines on bottom of p. 5) * P. 6. Split first paragraph (probably at "A related framework?" ), which is very long and seems to address several main points. * P. 6. There is 1-sentence paragraph. * P. 6-7. Use parallel construction in sentence "The report acknowledges ?" * P. 7. There is another 1-sentence paragraph. Done throughout. With regard to excluding “we” on page 4, “we” is left in as this is the work of one of the authors.

Table1

Table 1 Selection of Sub-items From One Question Activity included in the survey

Selected

1. Participated in a whole-group discussion or session

Included

2. Participated in a small-group discussion or session 3. Made presentations to the group 4. Developed a lesson plan 5. Discussed instructional techniques

Included Not included Not included Included

6. Practiced using software 7. Reviewed student work

Included Not included

8. Making connections between SimCalc resources and standards

Included

Table2

Table 2 Selected items for SCT Index and final components from PCA Question

Sub-item (Variable)

Response

Rationale

Component 1: Perception of the usefulness of SimCalc PD with respect to personal teaching needs Question 8 - How useful were each of the following activities of the SimCalc PD workshop to you in preparing to teach the unit?

PDS1_AL1 Participated in a wholegroup discussion or session (0-5)




PDS1_AL2 Participated in a smallgroup discussion or session (0-5)




PDS1_AL5 - Discussed instructional techniques (0-5)




PDS1_AL8 - Making connections between SimCalc resources and standards (0-5)



Question 6 - Reflecting on your SIMCALC professional development, to what extent was the professional development characterized by the following?

PDS2_CG1 Consistent with your goals for your professional development (1-4)

Desirable The exposure to SimCalc & PD is believed by the response: 2 or 3 responder to be in alignment with their (high) advancement as a teacher.

Question 6 - Reflecting on your SIMCALC professional development, to what extent was the professional development characterized

PDS2_CG7 - Designed to integrate technology

Desirable The exposure to SimCalc and PD is believed by the response: 2 or 3 responder to assist in bringing technology and

by the following?

into your teaching (1-4)

(high)

teaching together in their classroom.

Component 2: Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics Question 12 - Were the following parts of the SimCalc Intervention useful in terms of your teaching and your students’ learning and engagement?

STK_VAL1 - Use of simulations in software (0-4)



STK_VAL2 - Use of interactive graphs in software (0-4)



STK_VAL6 - Students working in pairs or as part of a group (0-4)

Desirable Considering a favored, or intentional aspect of the response: 3 or 4 curriculum made for SimCalc has students creating (high) and exploring on their own, as opposed to a demonstration, the responder should find high value in this.


STK_VAL7 Curriculum in conjunction with the software (0-4)

Desirable Considering the curriculum was designed for use response: 3 or 4 with the software, and for their standards, the user (high) should find high value, and believe there is a connection between the software and curriculum.

Question 7 - To what extent did the SIMCALC professional development prepare you to help students do the following?

PDS2RAT4 - Solve problems that have more than one correct answer (1-4)

Desirable SimCalc PD changed the mindset of the responder response: 2 or 3 to be better prepared and more open to the (high) possibility of more than one correct answer. Often, the opposite is true with traditional methods, materials and textbooks

Question 7 - To what extent did the SIMCALC professional development prepare

PDS2RAT 7 - Explain Desirable What was learned during training and use of why a procedure response: 2 or 3 SimCalc effected the responders ability to students used worked to coherently explain varying procedures produced by

Removed following PCA but used in SCT Index

you to help students do the following?

solve a problem (1-4)

(high)

students back to all students in a way they would understand.


PDS1_AL6 - Practiced using software (1-4)




STK_VAL3 - The curriculum materials (1-4)

Desirable Considering the curriculum was aligned to response: 3 or 4 standards and designed for the software, a (high) responder should find high value in the curriculum/software.

Selected by Research team but removed from final SCT Index for statistical reasons Question 1.1 – To what extent have you made each of the following changes in your teaching practices as a result of your experience with SIMCALC and SIMCALC professional development?

IMP_PRACTICEC – The instructional methods I employ (0-3)

Desirable This question-variable-response was selected response: 2 or 3 because the exposure to SimCalc and correct (high) SimCalc methods and practice via PD had an impact, as perceived by the responder, to be a large change. This indicates a change in pedagogical methods because of SimCalc. Presumed positive.

Question 1.1 – To what extent have you made each of the following changes in your teaching practices as a result of your experience with SIMCALC and SIMCALC professional development?

IMP_PRACTICEE – The ways I use technology in instruction (calculator or computer) (0-3)

Desirable This indicates a change in pedagogical methods response: 2 or 3 with respect to technology in the classroom, due to (high) the exposure to SimCalc. Presumed positive.

Question 5 - How would you describe your SIMCALC class’s coverage of the student workbook?

IMP_COV1 - We completed the entire book from start to finish (0-1)

Desirable response: 1 (yes)

This indicated a belief that the curriculum was coherent and aligned with standards and frameworks required by TAKS. It also indicated that supplemental material is not needed and completion of the material is fundamental.

Table3

Table 3 Utterances for Low SCT Indexed Teachers Component 1: Perception of the usefulness of SimCalc PD with respect to personal teaching needs “I was glad to be in the SimCalc group because the TexTeams stuff is just like your regular old run-of-the-mill workshop where they give you activities for each TEK. But with the SimCalc, that one - just doing the workbook covers so much even in one day you cover so much stuff that I enjoyed it better. I didn't do any of the TexTeams stuff. I just did the SimCalc stuff. I’m not opposed to the TexTeams stuff but I just saw that SimCalc covered so much of the material that that’s all I did.” “I do what I’m instructed to do. We’re required by our district to do five days of outside workshops a year and then we get those back as waiver days throughout the school years. And I really like that because then you can find your own and you can find things that apply to you and your field rather than just going to an all day speaker who has nothing to do with math.” [On using any of the SimCalc strategies presented in the PD workshop] “I used a little bit, not a whole lot. Normally I do ask for more depth and get their, or why do you think this is the answer and well, as far as having actual time sometimes, I just felt rushed and didn’t have time to do that so I didn’t do it. Normally, I do.” “You know, I have really good experiences towards the workshop. … And he [superintendent] doesn’t like really like to go to workshops during the school year that’s and we do a lot in the summer for this [SimCalc PD], it really helps me out on my workshop hours.” [On using any of the SimCalc strategies presented in the PD workshop] “Umm. I don’t really remember any strategies; we just kind of went over the whole work, looked like the kids did. So I don’t remember any particular strategies.” [On the SimCalc PD workshop] “Well, and I think that probably the other teachers had the same problem – I don’t know what they’re grading – what they are mandated to do for grades, but it’s tough when so much of it is group work to put a grade on it.” Component 2: Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics

“The proportionality thing just – unless the teacher brought it up and sort of keyed the students in on ‘okay, do you see because “x” changes and because “y” changes and these are proportional’ and unless the teacher said that, it was not brought up in the lesson until the very end.” “SimCalc differed because it dealt with graphing because everything we did we graphed it and before, you know, I'll just - they'll just have an equation. Distance equals rate times time and they'll have a short story problem on it and they'll write down their formula, they'll plug in the numbers they know, and then they'll solve the equation and this just always had a graph to it. So, it was even more of, you know, got them to some of those higher level thinking skills because it did involve the slope aspect of things and how the vertical changed over the horizontal and that type of stuff. So, it differed because it went further than what I usually do.” “We started off kind of rough, because it was my students’ first time in the lab and so you know the computer problems and making sure they weren’t playing with the mouse or playing with the graphs when they were supposed to be listening, things like that. Once we finally established the rules and they understood what I wanted, then the other days went kind of smoother.” “I know one of the assignments – it said, ‘go back to some of the other graphs in this workbook and find the slope of the lines.’ We tried to do that at first and it just didn’t seem conducive to what we were trying to do to find the slope. It just never seemed to work out right. So, that’s why we went ahead and made up our own sheet.” “I think it helps them a lot. I really think it helps them. I think that 3½ weeks is too long. But I wanted to do the whole book, that was my goal for myself because I told them guys I really want to do this whole book; you know well ma’am we are tired of doing this and it got to the third week and its like we are almost finished.” “The hard part was moving on the graph where you had to move the lines and drag and make the lines look what they wanted to look like … I had to guide them because a lot of them weren’t familiar since they don’t have computers at home.” “Managing the class for me was difficult… I found I couldn't handle it, I went back to the room with a computer and an LCD projector and as a class we all worked with it.”

Table4

Table 4 Utterances for High SCT Indexed Teachers Component 1: Perception of the usefulness of SimCalc PD with respect to personal teaching needs “This has got to be very much facilitating. You have to be a facilitator, not a lecturer. I don’t think it would work if you stood up there and just lectured.” [In response to best teaching strategies for PD workshop] “I did use them [the strategies presented in the PD workshop] very much. You know I think I used those somewhat before, but making them more formal. I think the big thing was like, if the students said they didn’t understand or something, the deal of saying ‘what do you know’ or ‘what do you get’ not quite that way, but clarifying, having them clarify what they already understand.” [On using the SimCalc classroom questioning prompts] “Well, it went pretty good. But I would like to do more though, a lot more.” [On using any of the SimCalc strategies presented in the PD workshop] “Yes, yes. In fact, I am trying to improve my teaching skills by asking more and more open-ended type questions where they have to think and look at what they have done and ‘why did you do it this way?’ yes I am doing that more and more.” “I don't know about the other teachers but what helped me a lot was the training that we took before during the summer and then we went over all the notes and the booklets that we had. And I don't know if that helped because at that point we had a lot of questions and all the presenters they were very good at answering all our questions.” Component 2: Perception of what specific features of SimCalc software and curriculum are valuable with respect to teaching and learning mathematics “Each of the unit lessons they are taught, they do probe for the higher level thinking skills, like explain or describe or come up with the story that matches this as scenario and those are things that are helpful because our lessons don’t always in the curriculum that we use.” “Now, I did the simulations with them but I didn’t say anything. They would come back and – it seemed like the simulations would start the

discussions … you’ve got to be the facilitator and not the lecturer. Make them figure things out. I think that’s what makes it so meaningful.” “I have a real positive feeling about the whole unit and I think some of the maths was difficult for my students but I really think that the kids were able to do some things that they would not have been able to had they not had the software and worksheets. They could perform at a higher level I think because of the software.” “Of course, we teach rate and proportionality as proportions and n over 6 is equal to 8 over 12 and ‘what is n?’ and something like that. In fact, as far as my class is concerned, I’ve never connected a graph to proportionality until this year with this lesson. … I showed them with the proportionalities and they seemed to catch onto that real quick.” “The thing that I think impressed me the most as an overall - was the fact that we started out with our constants and then we went into the constants of proportionality and they saw the rates. Then when we got into actually calling it slope, they actually made that connection. We're talking about the same thing, just different situations and that to me was the most phenomenal thing for 7th graders to feel comfortable enough with slope to actually make connections in other areas.” “It's just a much higher level. I feel like they have a greater understanding than I've ever had my students achieve.” “The students were able to graph better and make tables. They were able to find the missing numbers more easily and some things like that.” “Going from a simulation to creating a graph improved every time, or going from a table to a graph. Their graphing ability really improved.” “What I really liked about this year is the kids had a good grasp on the rate of change of two different slopes and they also understood going forward and going backward, I thought that was something they picked up on. I was very pleased about that because they understood that time still goes on whether you are standing still or moving, time is still continuing and that’s why it’s a flat line.” “They can graph, they got that down, but as far as looking at a graph, they have probably gone from a distance over time graph and solved for speed in science. But they have not connected that the speed would be the slope, that that translates to the slope of the line.” “I have a young man that's in one of my classes. And he has failed one of two 6 weeks in my class because he just didn't want to do the work. Well, on the post-test, he made an 80, which I was so impressed so I ran up to the cafeteria and I said, ‘look! You made an 80 on this test!' Well, I

went on back down to my room and here he comes and he looks at me and he says, ‘I understand how to figure out speed.' He said, 'but does speed and acceleration have anything to do with one another?' And that came out of the blue. I don't know where it came from. If we can catch the interest of these kids that aren't very motivated, then this is a wonderful unit.”

Figure1 Click here to download high resolution image

Mathematical Thinking and Learning

Mathematical Thinking and Learning

Suggest Documents

Mathematical Thinking and Learning

Mathematical Thinking and Learning Developing

Developing Contextual Mathematical Thinking Learning Model to

MATHEMATICAL THINKING STYLES AMONG

Teachers' Mathematical Values for Developing Mathematical Thinking ...

Teachers' Mathematical Values for Developing Mathematical Thinking ...

Learning to Investigate Students' Mathematical Thinking: The Role of ...

Fostering mathematical thinking in the learning of ...

Pertinent Elements of Critical Thinking and Mathematical Thinking

relationship between mathematical thinking and ...

Seeking Evidence of Thinking and Mathematical ... - CiteSeerX

Students' Critical Mathematical Thinking Skills and Character ...

Improving Students' Mathematical Thinking And Disposition Through ...

Thinking styles of Mathematics and Mathematical ... - Pythagoras

Promoting Mathematical Thinking and Communication among Pre ...

Critical and Creative Mathematical Thinking with

Mathematical Thinking & the Brain - CiteSeerX

ORIGINS OF MATHEMATICAL THINKING - CiteSeerX

Making mathematical thinking visible. TLRI

WORKING GROUP 2 Affect and mathematical thinking

CRITICAL THINKING TEACHING AND LEARNING

System Thinking, Evaluations and Learning

CRITICAL THINKING TEACHING AND LEARNING

mathematical teaching and learning environment