ZDM Mathematics Education (2015) 47:79–92 DOI 10.1007/s11858-014-0635-6
ORIGINAL ARTICLE
Scaling a technology-based innovation: windows on the evolution of mathematics teachers’ practices A. Clark-Wilson · C. Hoyles · R. Noss · P. Vahey · J. Roschelle
Accepted: 25 September 2014 / Published online: 8 October 2014 © The Author(s) 2014. This article is published with open access at Springerlink.com
Abstract This paper reports research on effects on teachers’ classroom practices resulting from their engagement in sustained professional development and classroom teaching of a resource that embeds carefully designed dynamic technology within middle school mathematics (11–14 years). The research investigated the self-reported evolution of teachers’ classroom and departmental practices as they sought to integrate the materials and the technology into their teaching. The sample comprised 203 English middle school mathematics teachers who participated in the professional development and taught the materials during 2013–2014. The methodology used questionnaires, administered to match individual teachers’ teaching schedules, and the resulting data were analyzed quantitatively to give summative statistics, and qualitatively to elicit more nuanced contextualised information. The questionnaire data were thus supplemented by two case studies to illustrate how the trajectory of development of teacher practices was shaped, first by their initial motivations to participate in the innovation, but later more strongly by the ways they chose to align their practices to the institutional goals of their schools in order for the innovation to be sustained.
A. Clark-Wilson (*) · C. Hoyles · R. Noss London Knowledge Lab, Institute of Education, University of London, London, UK e-mail:
[email protected] P. Vahey · J. Roschelle SRI International, Menlo Park, CA, USA
1 Introduction Research into the introduction of dynamic technologies to bring about transformative change to mathematics teaching is recognised as a complex process for individual teachers, which takes several years and for which professional development is a necessary element (see Hoyles and Lagrange 2009; Clark-Wilson, Aldon, Cusi, Goos, Haspekian, Robutti, and Thomas 2014). Whilst knowledge of the design and impact of large-scale PD programmes for mathematics teachers is developing (Zehetmeier 2015; RöskenWinter, Schüler, Stahnke, and Blömeke 2015) far less is known about how the specific design and impact of such programmes that embed digital technologies. Thus, our main research question concerns the factors that indicate success of this type of innovation at scale. Such a question can only be researched in the context of a specific design and implementation along with some explicit criteria on what constitutes success. Here we report research undertaken in the context of a specific initiative, Cornerstone Maths (CM), as it scales from design experiments (in USA and in England) to implementation in hundreds of classrooms across England. From a methodological perspective, our primary source of data has enabled us to map how, from the initial face-toface professional development, teachers’ practices evolve as they begin to integrate dynamic technologies into their teaching. CM is a USA/English collaboration designed as an integrated package of web-based software, teacher and student materials and PD support that uses transformative dynamic mathematical technology to address selected challenging curriculum topics in middle school mathematics (See Hoyles, Noss, Vahey, and Roschelle 2013). By transformative technology, we mean computational tools through which
13
80
A. Clark-Wilson et al.
students and teachers can (re-)express their mathematical understandings, understandings which are simultaneously externalised and shaped by the interactions with the tools (see Hoyles and Noss, 2003). Here the technology refers to a bespoke web-based software environment within each CM curriculum unit that supports dynamic exploration and student conjecture through a range of representations and real-life contexts. In designing the software, we tried to achieve a compromise between the range and level of mathematical expressivity and the provision of resources that teachers can implement successfully within diverse classroom contexts. The investigation of the teachers’ evolutions starts with their reflections on their practices following engagement in professional development (CPD is mandatory for teachers who want to use CM materials) and follows with their thoughts on subsequent classroom teaching. The sample comprises teachers in 100 CM schools teaching the first CM unit on linear functions, an overview of which is shown in Fig. 1. At each stage of the design process, the developed materials were rigorously evaluated to assess their efficacy. Moreover, the materials were shown to have stood the test of time and been ‘implementable’, that is they could be integrated into the school curriculum. For a fuller account of this earlier stage of the CM project, see Hoyles et al. 2013, which reported that successful outcomes were related Fig. 1 The software design principles, mathematical context and key mathematical ideas within the CM curriculum unit on linear functions
13
to the extent to which teachers evolve their thinking from mere adoption—in which they might be thought of as ‘implementing’ the approach—to one of ‘adaptation’, in which teachers come to see the innovation as their own and contribute to its fit to their personal and institutional goals. Here of critical interest is the role of technology through which transformative change is catalyzed and becomes visible, and, crucially at scale, by which communities can be nurtured at a distance. The paper is structured in the following way: It begins with an outline of the framing theoretical ideas and a summary of the USA work that concludes with the set of characteristics hypothesised to be key to both successful implementation at scale and sustainability. After a brief methodological interlude, the empirical data resulting from the English study are reported and compared with the USA indicators. We conclude by identifying a set of factors that impact on the trajectory of teachers’ practices as they implement the innovation.
2 Conceptual framework The earlier phases of the Cornerstone Maths programme were conceptualised within a design-based implementation research (DBIR) paradigm (Kelly 2004; Cobb, Confrey, diSessa, Lehrer, and Schauble 2003; Penuel, Fishman,
Cornerstone Maths Curriculum Unit on Linear functions
Mathematical content for students
Software design principles:
Context: Develop games for mobile phones, using mathematics to analyze and create the simulated motion.
•
Dynamic simulation and linking between representations.
•
Simulation controlled by the graph or the mathematical function.
•
Show/hide representations, as appropriate.
Key mathematical ideas •
coordinating algebraic, graphical, and tabular representations
•
y= mx+c as a model of constant velocity motion – the meaning of m and c in the motion context
•
velocity as speed with direction
81
Scaling a technology-based innovation
Cheng, and Sabelli 2011; Hoyles et al. 2013), where teachers are positioned as ‘designers of lessons’, and in the case of CM units of work, (re)conceptualise them according to the constraints of their unique classroom settings (Laurillard 2012). These adaptations provide an insight into practice, as teachers foreground the mathematical content and pedagogical approaches they perceive to be important. Within the context of a new innovation, it is inevitable that some elements of a teacher’s established practices are in tension with those of the innovation and by developing research methodologies that open a window into teachers’ reflective accounts we are able to build knowledge of their professional trajectories. 2.1 Interpreting the ‘fidelity’ of teachers’ classroom practices The term ‘implementation fidelity’ is used widely within educational research to imply the degree to which an innovation is ‘implemented as intended’ so that the outcomes can be defined and assessed (F. Jackson 2012; Munter, Garrison, Cobb, and Cordray 2010; Mowbray, Holter, Teague, and Bybee 2003; Berman and McLaughlin 1976). However, this term tends to underestimate the complexities of the mathematics classroom, particularly where dynamic technology and teaching approaches that exploit their use are being promoted. In studies of classroom effectiveness, implementation fidelity frequently manifests itself as a checklist of key features to be observed or reported, with quantitative measures established to arrive at an overall conclusion (e.g. Slavin, Sheard, Hanley, Elliott, Chambers, and Cheung 2014). Thompson and Wiliam (2008) discuss how interventions should be tight but loose, with the adherence to the central design principles needing to be tight, whereas the flexibility of the intervention so that it can be implemented in a range of contexts should be loose. The authors warn of an intervention being too loose such that the teachers involved in the innovation are unable to make sensible decisions concerning its implementation. Although this is a useful distinction, there are two concerns about how its operational effectiveness. First, it is surely difficult to know a priori what is ‘too loose’ or ‘too tight’. We explore this notion of fidelity with respect to the particular design principles for CM unit 1 and its effects on the CM CPD in the next section.
and Huang 2010), leading to ‘a shift in ownership such that a reform can become self-generative’ (Coburn, 2003, p. 3). The products are the tangible outcomes that can be evidenced as a result of the project, i.e. the number of schools/departments/teachers; the geographical reach; and improved student outcomes. However, the processes are the main actions taken to achieve the products. We adopted and elaborated Hung et al.’s framework, which conceptualises innovation scaling as a set of processes that need to be re-created/re-instantiated/re-enacted in the milieu of the products of the innovation, namely [the] artefacts and boundary objects (ibid, p. 90). According to Hung et al., it is at the point of intersection with boundary objects (as entities that allow different groups to collaborate on a common task, see Wenger, 1998) that both ‘legitimate’ and ‘lethal’ mutations of an innovation occur (Hung et al. 2010). Hung defines legitimate somewhat tautologically as in line with sound learning principles. We define legitimate mutations as those that are consistent with the learning principles and teaching practices that have informed the design of the Cornerstone Maths innovation and lethal mutations as those we consider inconsistent or detrimental to it (for CM, see Table 1).1 A limitation of Hung et al.’s framework is that it does not consider the local conditions in which the scaling is taking place, possibly due, at least in part, to a ‘top-down’ approach to scaling within the Singaporean educational system, an aspect the authors acknowledge. This contrasts with the current English school context with its diversification of types of state schools2 and a relaxation of its mandatory national curriculum for newly established school ‘types’ (Department for Education 2013). 2.3 Designing PD for scaling within Cornerstone Maths There are a number of important assumptions concerning how PD is conceived and designed within CM, the justification for which is firmly grounded in the existing research: –– Secondary mathematics teachers are not a homogenous group and a ‘one size fits all’ approach to PD design is not appropriate, particularly where dynamic mathematical technologies are involved (Noss, Sutherland, and Hoyles 1991; Pierce and Ball 2009; Clark-Wilson,
2.2 Defining large-scale change 1
The longer-term outcomes of sustainable interventions at large can only be considered successful if successful outcomes can be articulated. We define scaling beyond the conventional interpretation of merely ‘scaling-up’ (i.e. more schools, more classrooms etc.) to consider both the products and processes of scaling over time (Hung, Lim,
We recognise that legitimate and lethal mutations are not on a bipolar scale. 2 The state funded English school system comprised schools that have different degrees of autonomy from local education authorities and religious organizations: Community schools, Faith schools, Free schools, Academies. (See https://www.gov.uk/types-of-school/overview). Some school types were selective and non-selective in their admission of students.
13
82
A. Clark-Wilson et al.
Table 1 Examples of legitimate and lethal mutations within CM Boundary object
Legitimate mutation
Lethal mutation
Teachers devise additional tasks for students to learn Use of the to use and exploit the use of the mathematical dynamic features of the software software Use of the student Teachers identify underlying aims and purposes of workbook key tasks and re-present these to students during episodes of whole-class discourse
Teachers tell the students the expected behaviour of the dynamic representations
The teachers’ guide
Teachers do not adopt CM Unit 1 pedagogic approaches for the use of the software (e.g. predict, check, explain) without adequate rationale
Teachers engage with and use the pedagogic framework in the teacher’s guide (e.g. predict, check, explain)
Robutti, and Sinclair 2014; Fuglestad, Healy, Kynigos, and Monaghan 2009). –– Teachers should adopt a range of roles within PD activities to include that of the learner, the teacher in the classroom and the teacher within a professional community (Even and Loewenberg Ball 2009; Healy and Lagrange 2009). –– Prevailing PD practices in England take a range of forms that include face-to-face events, asynchronous online events and ongoing online communication and collaboration (Gouseti, Potter, and Selwyn 2011; de Geest, Back, Hirst, and Joubert 2009). The initial CM PD took place over one whole day at the University (with some follow-up meetings in schools), followed by webinars and ongoing support through an online project community.3 The face-to-face day was designed to enable teachers to: see the ‘big picture’ of the mathematical progression embedded in the curriculum unit; experience a reflective hands-on experience in a range of roles (learner and teacher); envision new classroom practices involving student uses of dynamic technologies; plan and share their lesson designs (to include looking at students’ work); learn of the expectations of their involvement in the project community; and learn of their role in the research process (which was primarily to teach the unit and provide feedback). As the project aimed to create a new practitioner community to continue the PD opportunities, emphasis was placed on the expectation that teachers would adopt an ongoing and active role by participating in the online community forum, sharing experiences, and engaging in professional reflection through the project questionnaires and webinars. We now turn to the antecedents of CM in its research and development in the USA. 3 The online project community was facilitated by the government funded National Centre for Excellence in the Teaching of Mathematics portal http://www.ncetm.org.uk.
13
Teachers do not read (or ignore) the teachers’ guide so do not encourage students to engage with the sequence of activities within the student workbook (in any format), without a clear reason
3 Antecedents of the project and scaling Cornerstone Maths in England 3.1 The USA studies and findings Our initial USA studies were based on a design experiment paradigm, in which researchers investigated how technology-based dynamic representations could be used to reconsider both the mathematics content to be taught in key stage 34 as well as the pedagogical approaches to teaching mathematics that would allow students with diverse backgrounds to succeed in learning advanced mathematics (Kaput and Roschelle 1998). A detailed account of the USA project and how it was redesigned for CM in England is given in the earlier ZDM article (Hoyles et al. 2013). We summarise the key characteristics for scaling the CM innovation that were identified from the prior work in the USA: • A shift from the design of materials to the design of a curricular activity system (Roschelle, Shechtman, Tatar, Hegedus, Hopkins, Empson, Knudsen, and Gallagher 2010; Vahey, Knudsen, Rafanan, and Lara-Meloy 2013) in which the interactions between technology, curriculum materials, teachers and students must be taken into account alongside the overarching institutional context, including accountability systems and the overall affordances and constraints of the educational system (e.g. how much and what type of PD is available to teachers) (Vahey, Roy, and Fueyo 2013). • The development of materials and the PD is carried out in close partnership with local PD providers (Knudsen 2010). • PD is designed not only so that teachers understand how to teach the curriculum unit effectively and gain the necessary mathematical knowledge for teaching (MKT; Hill, Schilling, and Ball, 2008), but to understand how 4
In England, key stage 3 defines a 3-year phase: 11–14 years, which is sub-divided into Year 7 (11–12 years), Year 8 (12–13 years) and Year 9 (13–14 years).
83
Scaling a technology-based innovation
the materials could be used in a way that is aligned with their local context as well as with their own teaching goals, beliefs and values (that is, has high perceived coherence). • PD support is provided through an organizational structure that is familiar to the teachers and builds on existing PD infrastructures. • The importance of nurturing a small number of teachers who could become advocates for the programme. After the design research phase in England, the CM project began to scale to over 100 schools in November 2012. A more detailed overview of the project and research activities is provided in Appendix 1. This paper concerns those activities related to the first CM module of work on linear functions (Unit 1) and draws its conclusions from the data collected by the end of January 2014. We describe the CM project schools as either ‘design’ schools (most closely involved with the project designers, researchers and PD team) or ‘focus’ schools (more distantly involved due to the wider scaling of the project). The longitudinal nature of CM meant that school-level choices about classroom access to technology were continually under review as schools reconsidered modes of access (e.g. to include laptops with touch-screen functionality, iPads and other tablets). In addition, the English national curriculum was in the midst of a major review, which required schools to review their mathematics curricula in preparation for the first implementation from autumn 2014.
(note that the CM project is not mandated and schools and teachers choose to participate or not). To investigate our research question, we identified success as sustainability, i.e. teachers choosing to teach further CM units, and planning to continue to use the resources beyond the timeline of the current funded project. At the school level, success was judged as when the units were ‘written into’ the departmental schemes of work5 and PD was scaled ‘within school’ to include all of the teachers in the mathematics department (See Kaur 2015 for more on the theme of school-wide scaling of an intervention). A fundamental question was whether use at teacher and or school level aligned with the vision and aims of the original innovation. We expanded Hung et al.’s products and processes of scaling technologically based educational innovations to form an organizing frame within the context of the English CM project, as shown in Table 2. We introduced the idea of a professional network or ‘hub’, a group of schools that have a wider school affiliation to share practice for the project. The hubs are diverse in terms of size, institutional network, leadership and management and geographical spread. Central to the development of the hubs was the expansion of the team of ‘hub PD leaders’ who would assume ownership of, adapt and deliver the localised PD support. It is acknowledged that, within the process of scaling, the PD requirements and opportunities for the community of PD hub leaders or multipliers (Rösken-Winter et al. 2015) are as important as those of the participant teachers (Even and Loewenberg Ball 2009; Fuglestad et al. 2009; K. Jackson, Cobb, Wilson, Webster, Dunlap, and Appelgate 2015).
3.2 Indicators of success We first questioned the factors that influenced success at scale from the perspective of the individual teacher through evidence of: –– expression of satisfaction with the PD and teaching materials; –– alignment between the PD and teaching materials and their goals as a teacher; –– use of materials and the extent to which they create legitimate adaptations; –– positive outcomes in their classroom; –– activity and engagement within the professional community and with the project team; and ultimately: –– the extent to which teachers redefine powerful learning of their students in the light of the innovation. We next moved to study broader factors and explored motivations to become (and remain) involved in the innovation at both the level of the individual school and the teacher, which were not necessarily aligned with each other
4 Methodology The broad research question concerns how teachers’ practices evolve within their classroom/school. However, the scale of the project required us to consider how to report the outcomes of teachers’ implementations both at scale and in sufficient detail to be able to offer evidence of transformative change. Our resulting methodology considered the process of data analysis according to two grain sizes. The largegrain size analysis used data collected from the complete cohort of teachers (n = 203), who were preparing to teach (or had taught) the CM unit between June 2012 and January 2014. We drew our findings from their feedback to three online questionnaires, devised specifically for the project, piloted by the Design school teachers and 5
In England, each school devises its own approach to the teaching and assessment of the national curriculum, which is known as the scheme of work.
13
84
A. Clark-Wilson et al.
Table 2 Scaling Cornerstone Maths: themes, products and processes Theme
Products
Processes
1. Geographical reach
a) Number of schools involved
a) Development of web-based curriculum activity system b) Development of teacher community c) Development and maintenance of regional hub-based offer of professional support d) Development of school clusters, supported by project team leading to development of local hubs with local CM project lead e) School-devised methods to evaluate students’ outcomes f) Development of school-based PD g) Support to embed CM within local of schemes of work h) Teacher use of the materials beyond their original project commitment. (e.g. with older classes or revision classes)
b) Number of local hubs involved
2. School buy-in
c) Improved student attainment d) Number of whole departments involved e) Wider use of the materials
3. Penetration in mathematics department
f) Number of participating teachers in each school
then, following revisions, implemented with the Focus school teachers. These questionnaires were administered as follows: • Questionnaire 1: Sent immediately after the face-to-face PD event, and prior to beginning teaching of the curriculum unit. (100 % completion rate, n = 203). • Questionnaire 2: Sent immediately after the teachers had commenced their teaching of the curriculum unit. (88 % completion rate, n = 195). • Questionnaire 3: Sent immediately after the teachers had completed their teaching of the unit. (19 %, completion rate, n = 38).6 The surveys aimed to probe teachers’ dispositions towards CM, its PD and their evaluations of their resulting classroom implementations: • Was the CM curriculum unit implemented in the classroom? • How was the CM curriculum unit implemented? (choice of class? choice of technology? pathway through the unit?) • Were teachers positive about the materials? Would they use them again? • How did teachers engage with the project community? The questionnaires design collected: demographic and contextual data; Likert-style statements of agreement; and open text responses to ascertain teachers’ perceptions and actions. Where responses were open text, a constant comparison method was used to develop a set of recurring responses 6 As each teacher decided when they would teach the CM unit, some teachers did not complete their teaching by the end of January 2014.
13
i) Development of a lead practitioner (who may be the subject leader) j) Development of peer-support for participating teachers
or themes, which resulted in a useful and informative summary of the complete cohort (Glaser and Strauss 1967). For the fine-grain size of analysis, which aimed to elicit more detailed evolutions of practice, a case study approach led to the creation of portraits of selected CM teachers. The production of the case studies included artefacts and their pattern of use that had been produced at the boundaries of the classroom and professional collaborations, such as lesson adaptations and student productions. Additional data about localised schemes of work and school-developed assessments was sought from the case study teachers that extended beyond their individual classrooms and provided a deeper understanding of their school and wider institutional context, seeking evidence of within-school scaling. We include two contrasting case studies that give a sense of the teachers’ trajectories of practice and acknowledge the importance of the institutional dimension.
5 Results We report the emerging outcomes of teachers’ engagements with the PD both ‘at scale’ and ‘over time’ as we sought evidence of the evolution of classroom practices and the determinants of the sustained change that might support the scaling process. 5.1 Large-grain analysis: evolving practices at scale We begin by reporting: the teachers’ evaluations of the PD; their aspirations for their future practices with dynamic technologies through the teaching of the first CM unit; and additional contextual data about their classroom implementations. Analysis of this set of data provides an insight into how the teachers at scale perceive the forms of alignment (or
85
Scaling a technology-based innovation
coherence) of CM to their classroom and institutional settings. We then report how teachers’ reports of their teaching evidenced the mathematics that had been foregrounded and provided some insight into evolving classroom practices. 5.1.1 How CM aligned to teachers’ existing practices Teachers’ initial engagement with CM was during the faceto-face PD event, following which they completed evaluations of its appropriateness in supporting them to develop their classroom practice using dynamic technologies. The responses (in the form of a 4-point Likert scale from excellent to unsatisfactory) were overwhelmingly positive with 88 % of the cohort of 203 teachers (n = 195) judging the PD to be ‘excellent’ or ‘good’. Further analyses of their qualifying statements provided an insight into how they perceived their practice might develop through teaching CM unit 1 (Table 3). The diversity of responses reflected the mixed community of teachers and their school roles: 84 classroom teachers; 17 coordinators of the school’s KS3 curriculum; 20 deputy heads of mathematics; 17 heads of mathematics; 3 lead mathematics teachers and 10 ‘other’ school roles. 5.1.2 How CM aligned to teachers’ individual motivations We investigated how teachers’ individual motivations to participate in the project were related to the ways they had learned about the project. These routes included: self-initiated (11); invited/nominated by Headteacher
Table 3 Teachers’ views of how their practice would be improved Aspect of classroom practice (that would be improved)
% of teachers (n = 195,96% response rate)
Develop use of technology for mathematical learning Improve learning outcomes for students Develop more exploratory teaching approaches Improve student engagement Highlight links between mathematical representations Promote student independence Improve differentiation and questioning techniques Improve classroom discussion and collaboration Provide more challenge for students Highlight the use of context Relinquish some classroom control Offer more sustained teaching of linear functions Use more innovative practices
46
Develop colleagues’ practices
1
19 15 11 10 9 8 8 7 7 6 2 2
(14); invited/nominated by their head of mathematics (or another senior colleague) (113); and invited/nominated by a regional mathematics adviser (15). The selfinitiated teachers stated motivations that related to the development of innovative practice and involvement in a prestigious research project. On the other hand, the overwhelming majority of the group who had been asked or nominated by a senior colleague (127) offered reasons that were highly consistent with the project’s overarching aims (i.e. to develop use of technology for mathematics learning), implying that they had, at least, understood the project aim’s and, at most, indicated an initial sense of coherence. Only a handful of nominated teachers implied a reluctance to be involved. 5.1.3 How CM PD impacted on teachers’ confidence to teach with dynamic technology A second more pragmatic baseline was established at the end of the PD event with respect to the individual teachers’ levels of confidence to teach the unit in their classrooms (Fig. 2). Given that CM aims to put dynamic technologies into thousands of students’ hands, a goal that has been historically challenging to achieve at scale, this result is reassuring in that the vast majority of teachers felt that the initial PD event had prepared them to use the software and materials in their classrooms. The first questionnaire also asked teachers to select unlimited responses from a series of statements (and add their own) to reveal data on the further professional support that they would welcome to support them to implement the curriculum unit in their classroom (Table 4). This would inform the further development of the curricular activity system (or Process a in Table 2). The prevalence of responses that related to aspects of assessing the students’ outcomes either informally not at all conident, 4.6%
reasonably conident, 60.3%
totally conident, 5.3%
very conident, 29.8%
Fig. 2 Personal level of confidence to teach Cornerstone Maths curriculum unit 1 (n = 151,73 % response rate)
13
86
A. Clark-Wilson et al.
Table 4 Teachers’ indications of topics for further support (n = 144) Ideas to support the formative assessment of students’ learning 60 % as they work through the unit 56 % Features of the best CM lessons Information about the formal pre- and post-tests Video clips of CM lessons in other classrooms Ideas (and resources) to adapt the unit for students who had English as an Additional Language (EAL) or with Special Educational Needs (SEN) Technical support to access and use the software with students Wider reading and research about the ideas that underpin the design of the unit
54 % 46 % 33 %
Webinars to discuss the unit with the project team and other teachers
8 %
19 % 16 %
(classroom-based formative assessment) or formally (using questions from the previously developed pre- and posttests) resonates with the current English context where school inspection practices require teachers to demonstrate tangible evidence of students’ learning both within individual lessons and over time. This necessitates the differentiation of teaching to focus on the needs of all students, and also explains the high response (33 %) with reference to students with English as an Additional Language (EAL) and Special Educational Needs (SEN). As CM units are designed with an abundance of rich questions that probe deeper understanding, the teachers’ responses highlight a need to develop classroom practices in making best use of these questions (i.e. informing Process j in Table 2). By the end of January 2014, 82 of the Focus school teachers had begun to implement the curriculum unit with a diversity of classes at the KS3 level (16 % in year 7, 46 % in year 8, 27 % in year 9 and 11 % in year groups beyond KS3). At the same point in time, 38 (of the 82) teachers had completed their teaching of CM unit 1 and reported their final outcomes through the third questionnaire, which provided an insight into aspects of the teachers’ classroom practices. 5.1.4 How CM supported teachers’ evolving practices The teachers reported very different pathways through the curriculum unit and all teachers completed the curriculum activities that combined all of the mathematical representations (simulation, graph, table and function). 68 % of the teachers reported that they had reached a key investigation in which students encountered negative gradient for their first time and 27 % of teachers reported that their class had completed the unit in full. The remaining 73 % of teachers all indicated an intention to revisit the unit later in the academic year and/or key stage. These teachers had given
13
diverse reasons for delaying their teaching of the complete unit: a lack of curriculum time, a lack of continued access to suitable digital technology and the need to return to the formal schemes of work for that class due to internal (termly) assessment arrangements. It is not possible to assume in general that any of these data are themselves indicative of changed practices. However, analysis of the teachers’ qualitative questionnaire responses highlights the nature of the mathematics that had been foregrounded during the CM lessons, providing some insight into how their personal reflections aligned (or not) with the design principles for CM. Fourteen teachers commented within the final questionnaire (n = 33) on how their students’ interactions with the software promoted students to make use of dynamic mathematical connections (a fundamental design principal of CM), for example, The triple representations really forced the students to make the connections for themselves—if they needed to alter the animation; they were forced to engage with the graph or the table of values, which connected the ideas much better than could be done on paper. Other responses that aligned with the CM design principles referred to aspects of the students operating (physically or cognitively) on the different representations by: creating; editing; manipulating; comparing; and linking the mathematical objects. Conversely, in response to a question that asked teachers to describe any examples where the CM software had hindered their students from learning the mathematical ideas, 17 of the 33 respondents said that it had not. The majority of the 6 teachers who gave examples said the hindrances were technical, but there were a few teachers who cited issues relating to the particular way students were required to edit the graph. One teacher’s comment in this respect was, The manipulation of the graphs—they did not need to be anchored where they were—it did not facilitate or increase learning. This comment contrasted with another teacher’s view of the same ‘hindrance’, Students found moving the lines on the graph difficult at first. They wanted to touch the endpoint and move it to the place required. They became more adept once they realised the axes moved independently. These two examples demonstrate how a subtle design decision may or may not be noticed by teachers—and how teachers’ reflections on particular features of the CM materials might provide an insight into their personal alignment to the design principles of the innovation. One teacher commented on how the new powerful ideas that students had accessed supported them to unlock mathematical concepts that had not previously been available to the students:
87
Scaling a technology-based innovation
The students had the animation, graph, and table to help them describe the movement of something in relation to position and time, but his was the first time they also had the equation. One student was so excited about making the connection between speed and the coefficient of x that he went back through the earlier investigations to work out the equation of the lines. The above range of quotations indicate how teachers may have individually ‘bought into’ the principles of CM, and how some may be well placed to become advocates of CM, one of the key characteristics for sustainability emerging from the prior USA studies. In addition, we would hypothesise that individual teacher ‘buy-in’ is a necessary pre-cursor to school buy-in and a pre-requisite to Products b, d and f and Processes b, d, f, i and j within Table 2. Finally, with respect to the development of teacher community (Process b in Table 2), 23 of the 38 respondents to the third questionnaire indicated that they had used the CM online community in the following ways: keep up to date about the project (n = 15); read questions/comments (n = 21); post questions/comments (n = 6); access the e-version of the Teacher’s Guide (n = 11); access the e-version of the Student’s Workbook (n = 9); and one teacher uploaded her own adapted lesson resources to the community’s shared resource area. 5.2 Fine-grain analysis: towards sustainability and scaling The two case studies selected here highlight how teachers’ participation in CM supported their classroom practice to evolve within their individual school and policy contexts, a central element of school buy-in (Theme 2 from Table 2) and penetration in mathematics department (Theme 3). Hence the large-grain analysis from the questionnaires, which provided some insights into these two themes, led us to consider the underlying scaling processes more closely by means of the case studies. Both of the teachers involved were in middle management roles but with differences in school context and experience of CM. The first teacher, Emma, was selected for the case study because she had taught the complete curriculum unit to her students and also responded in the third teacher questionnaire that the unit would become part of her school’s curriculum in the future. By contrast the second teacher, Robert, also part of the middle management in his school, had been head of mathematics for nearly 7 years and had been granted by the head teacher considerable autonomy to take forward curriculum innovation. Robert had been involved in the project since the first pilot in autumn 2011. His case offers a more longitudinal opportunity to understand how middle management alignment, as one of the key characteristics for scaling, is apparent in a successful school scenario.
5.2.1 Case study 1: Emma Emma was the Head of Maths at a newly formed ‘Free school’7 that had a strong philosophical vision to promote creative, exploratory approaches to learning, student independence and leadership. Emma attended the one-day PD event in June 2013 where she engaged enthusiastically in the PD tasks and activities. She had no previous experience of the CM teaching units. Emma’s responses to PD tasks and surveys revealed her desire to ‘give students control over variables’ and encourage them to predict-checkexplain, implying an allegiance to the CM pedagogic approach (see Table 4). Emma chose a class of twenty-three, 12–13-year-olds, a homogenous group that she described as having ‘higher ability’ within the year group, whom she felt would be challenged by the curriculum unit. They accessed the software using shared laptops (2–3 students) in their normal mathematics lessons. Emma commented that she would co-plan the lessons with a colleague and that they would be ‘building in questions’ to additional resources they would create to support the teaching (Processes f, g, i and j). She also planned to develop some ‘tasks designed to assess what they had done’. Emma dedicated the first half term’s mathematics lessons (approx. 25 h) to her teaching of the CM curriculum unit. She had also considered how the school would evaluate the project, which involved: feeding back to colleagues in department meetings; a systematic assessment of the students’ outcomes; inviting senior school to observe lessons; and reporting the final outcomes to school leaders (Process e). Emma’s legitimate lesson adaptations, which were most evident through the presentation slides she displayed in the classroom, concerned: enhancements to the unit’s context that strengthened to students’ personal connection to the project; the sharing of explicit learning objectives; the reiteration of the ‘predict-check-explain’ pedagogic approach; ‘review’ questions; and directions to students concerning where they should start and stop and what would be discussed during the whole class plenaries. Emma completed Unit 1 (14 investigations) and in her response to the third questionnaire reported that it had been very easy for her to integrate the unit within her current scheme of work and that the school planned to teach the unit in the future (Process j). She made the following comment about how she would extend the students’ work,
7
As a Free school the teachers were neither mandated to teach the English national curriculum nor to follow traditional school timetabling structures. However, the students would be assessed via the existing national assessment framework and the school was subject to scrutiny using national accountability measures.
13
88
We are now going to develop our own stories and put together a cross curricula project where we create a story of a journey and then we have to represent the story with a drawing, graph, equations and table of values. Emma’s overall response to the project evidenced her enjoyment of CM, but it also suggests how the design of the course resonated with her ideas about the mathematics, as she felt compelled to build upon the idea of the narrative with her students. The students spent a further 8 h within ‘project based learning’ curriculum time developing their written stories and creating travel graphs that represented their journeys. In total, Emma dedicated 33 h of her teaching time to the topic—considerably longer than the 12 h we had expected. In their final project work students demonstrated high levels of transfer of the mathematical ideas they had encountered within the CM unit and the more able students extended the mathematical content, for example identifying the equations of piece-wise linear functions that did not intersect with the y-axis (Process h). 5.2.2 Case study 2: Robert Robert became involved in the CM project in summer 2011 as one of two teachers from his school that were to pilot the first adaptation of the Unit 1 software and materials that had been originally developed for the Texas randomised control trial. His school was also in East London and, although in an area with a history of educational underachievement, it was a school that was reversing this trend, particularly with respect to mathematics. The school had higher than average numbers of students with EAL and the organization of students within mathematics was unusual in that students were taught in heterogeneous classes in KS3 (11–14 years). In Robert’s summary of his experience during the English pilot project, he commented, Initially I was very sceptical—I was really not sure—but now we’re really excited about the unit and the project, I’m very hooked on it because I can see the impact it has had on my students. We’re definitely going to use this unit again—it’s going to be something that we embed. For us it’s not a trial and we’re very excited about it. (Product c; Process g). For scaling, Robert’s change of heart, and later commitment to expand the project in his school are crucial to Themes 2 and 3 within our elaborated framework. Whilst the pilot was still in process, Robert expanded the teaching of CM throughout his school by organizing other teachers to observe CM lessons and by leading a 2 h PD session for the department in their faculty meeting time, during which they worked through the initial investigations alongside the student workbook (Products d, f; Processes f,
13
A. Clark-Wilson et al.
g, i, j). Once the department had agreed where the curriculum unit would be located in their KS3 schemes of work, Robert organised the teachers in pairs to plan and evaluate their lessons, justifying this by saying ‘this leads to greater collaboration and discussion’ (Processes f, g, i, j). Their resulting schemes of work included Unit 1 (split into two parts) and this included ‘assessment and review’ lessons at the end of each unit, which had been hand-written by the member of the department with responsibility for the KS3 (Process e). Finally, Robert was aware of the need to maintain an ongoing dialogue with his Headteacher, particularly as Robert saw it as his role to ensure that students and teachers had access to the technology (to include classroom support from a dedicated technician) as designated by the scheme of work.
6 Discussion Our research question sought to find out in a large scale project how teachers’ classroom and departmental practices evolved as, following CPD, they integrated the CM materials and technology into their teaching and the extent to which the practices were aligned with the original goals of the innovation. The large-grain data provided insight into the variety of pathways through the CM unit taken by the teachers, which were clearly influenced by their institutional constraints. However, it was the fine-grain data of the case studies that led us to clarify the role of shorter lesson sequences as a more effective component of the curriculum activity system, and facilitate greater alignment with institutional practices (Process a). The USA studies concluded that the key elements of the PD were: addressing teachers’ mathematical knowledge for teaching (MKT, see Even and Loewenberg Ball 2009) such that they were able to teach the mathematical content; developing the pedagogies inherent in CM and aligning the materials with existing classroom practices. Although we might have conjectured that MKT would not be a central issue for the majority of KS3 mathematics teachers in England (the teachers in England are secondary trained and mathematics specialists in contrast to the middle school teachers in USA, see Hoyles et al. 2013); we recognised too that that their mathematical understandings would be challenged as they were asked to rethink connections between representations in a dynamic way. The teachers’ feedback indicated that the majority seemed to have understood and embraced the underlying pedagogy of CM, as experienced during the initial PD event and tried to put it into practice. However, more longitudinal data are needed to draw
89
Scaling a technology-based innovation
meaningful conclusions on how these aspirations played out in practice. The early indications are that, where the underlying CM pedagogy resonated with existing or developing practices, teachers were more able to implement the project in their classrooms. However, at the implementation stage, over half of the teachers who had completed Unit 1 (n = 38) found it difficult to dedicate the time needed to teach the whole unit due to institutional constraints such as their existing schemes of work, timetable constraints or common year group assessments. As Emma’s and Robert’s cases illustrate, for scaling, we need to consider alignment beyond the individual teacher’s practice to take account of the wider institutional context (Product d; Process j). The USA studies positioned teacher support within the prevailing organizational structures built around existing infrastructures, however, the very different and evolving English context did not provide a consistent set of opportunities to organise the PD on a national scale. The scaleout design aimed to ensure that the different institutional settings, each with their own associated vision and goals, were taken as the starting point (Product b; Process c, d). We expect that where there was strong institutional alignment and support within the hubs, they would thrive and grow: time will tell. Earlier CM research (Hoyles et al. 2013) concluded the need for the project online community to be established from the outset and, to support scaling, its use would need to be set as a norm during the PD sessions, exploiting the support that a vibrant community could offer. Finally, with respect to the development of teacher community (Process b in Table 2), early data indicated that teachers were beginning to use the CM online community to: keep up to date with the project news; to read questions or comments by the Community; to post questions or comments to the Community; to access the electronic version of the Teacher’s Guide; to access the electronic version of the Student’s Workbook; and one teacher uploaded her own adapted lesson resources to the community’s shared resource area. Consequently, we continue to research the impact of the growing national project network that has provided a conduit for teachers to gain information, locate and share resources and communicate with each other (and the project team). (Processes b, c). As the project scaled, the analysis began to highlight how the design principles of individuals (and clusters of individuals) and their institutional settings influenced their adaptations to meet their personal and/or local priorities or needs. Prior to teaching the unit, the teachers’ aspirations for their teaching of the unit indicated that they perceived ways in which their teaching practices would develop beyond only improving their practices with technology. The fine-grain analysis provided by the case studies highlighted
this further. For example, the tight resonance between the CM pedagogic approach and the teaching philosophy and structures in Emma’s school made it easy for her to take up the project and see how her time investment in the creation of additional resources would pay off in the longer term (leading to Processes g, i, j). Indeed teachers’ willingness and the time allowed to make these adaptations, with and without the collaboration of others, may be a key characteristic of sustainability to be explored further. A secondary consideration is the individual teacher’s locus of control with respect to the mathematics curriculum in their school and beyond. Where they are empowered to make decisions about teaching approaches and resources in their department, they may be more motivated to invest their time and energy, as is the case with both Emma and Robert.
7 Conclusion We begin by revisiting the research question as we organise our findings in relation to trajectories by reflecting on themes, products and processes of scaling (as elaborated from Hung et al (2010) in Table 2). We used this framework to connect the following factors that were shown to have impacted on the trajectory of teachers’ practices. Although positioned at the level of the individual teacher, these factors are of course influenced by the teacher’s institutional setting, which includes other actors and systems that are outside of their direct control. 1. Teachers’ individual PD needs resonate with the overarching design principles of CM, particularly, but not exclusively concerning the role of technology within teaching and learning mathematics (Product a; Processes a and b leading to Process d). 2. Teachers have supportive relationships with colleagues and engage in collaborative professional activities (supported by local PD Leaders and mediated by technology) that focus on adapting the resources to suit the local school needs (Products b, d and f; Processes b, c and d). 3. Teachers are empowered (and are given time to) contribute to the revision and development of schemes of work to take account of the project’s outcomes (Products c and d; Processes e, f, g, leading to Product f; Processes i and j). 4. School interpretations of national policy align with the overarching aims of CM (Process g). 5. More senior colleagues (e.g. subject leads such as Emma and Robert) actively engage with CM teachers to support the implementation stage in order to evalu-
13
90
A. Clark-Wilson et al.
ate the outcomes and pave the way for ‘within school’ scaling (Product c and Process e). The data analyses have highlighted important indicators of success in scaling and sustainability were multiple forms of alignment, beginning with the aspirations and classroom practices of the individual teachers but extending to their institutional settings at departmental, school local and national levels. The fine grain analysis led us to identify one factor that did not immediately fit within our elaborated framework: school and departmental support (technological in particular) is aligned to the aims of the project and is organised on an ongoing basis. In Robert’s school the senior management supported the purchasing of additional technologies to facilitate within-school scaling and provided technician time to support its implementation. Hence it is our role to provide guidance to schools on the nature of such alignment, as it is a pre-cursor for teachers’ (and students’)
wider and sustained access to CM units over time. These factors will inform our data analyses as teachers’ trajectories within the CM project are charted over coming years and months. Acknowledgments We gratefully acknowledge funding by the Li Ka Shing Foundation. The research has been an intensive collaboration between teams at the London Knowledge Lab, Institute of Education, University of London and at the Center for Technology in Learning, SRI International, Menlo Park, USA. Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Appendix See Table 5.
Table 5 Overview of project and research activities Design phase (Nov 2012–April 2013)
Scaling (cycle 1) (May 2013–Feb 2014)
Recruitment of 52 ‘Focus’ schools Project implementation Recruitment of 6 ‘Design’ Schools Development of software and materials (91 teachers) Revisions to software, materials and Development of PD materials PD Implementation of one PD event Implementation of CM unit in schools Implementation of 3 face-to-face PD events Mediation of online community Implementation of CM unit in schools Teacher evaluations of efficacy of PD Research activities Collection of contextual data on event schools, teachers, classes, technolTeacher first online questionnaire ogy etc Teacher evaluations of efficacy of PD Teacher 2nd online questionnaire Sample of lesson observations event Sample of teacher interviews Lesson observations of each teacher Teacher 3rd online questionnaire and Teacher interviews follow-up correspondence Teacher online questionnaire and Focus group meeting follow-up correspondence
13
Scaling (cycle 2) (Dec 2013–Jul 2014) Development of 5 CM hubs Recruitment of 59 ‘Focus’ schools (113 teachers) Implementation of 6 face-to-face PD events Mediation of online community Animation of asynchronous online PD Implementation of CM unit in schools Teacher evaluations of efficacy of PD event Teacher first online questionnaire Teacher 2nd online questionnaire Sample of lesson observations Sample of teacher interviews Teacher 3rd online questionnaire and follow-up correspondence Focus group meeting
Scaling a technology-based innovation
References Berman, P., & McLaughlin, M. W. (1976). Implementation of educational innovation. The Educational Forum, 40(3), 345–370. Clark-Wilson, A., Aldon, G., Cusi, A., Goos, M., Haspekian, M., Robutti, O., et al. (2014a). The challenges of teaching mathematics with digital technologies—the evolving role of the teacher. In P. Liljedahl, C. Nichol, S. Oesterle, & D. Allan (Eds.), Proceedings of the joint meeting of PME 38 and PME-NA 36 (Vol. 1, pp. 87–116). Vancouver: University of British Columbia. Clark-Wilson, A., Robutti, O., & Sinclair, N. (2014). The mathematics teacher in the digital era: An international perspective on technology focused professional development (Vol. 2, Mathematics Education in the Digital Era). Dordrecht: Springer. Coburn, C. (2003). Rethinking Scale: Moving Beyond Numbers to Deep and Lasting Change. Educational Researcher, 32(6), 3-12. Cobb, P., Confrey, J., di Sessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. de Geest, E., Back, J., Hirst, C., & Joubert, M. (2009). Final report: researching effective CPD in mathematics education. Sheffield: National Centre for Excellence in the Teaching of Mathematics. Department for Education. (2013). The academy/free school presumption. London: Department for Education. Even, R., & Loewenberg Ball, D. (Eds.). (2009). The professional education and development of teachers of mathematics: The 11th ICMI study (Vol. New ICMI Study Series, Vol. 11). Berlin: Springer. Fuglestad, A. B., Healy, L., Kynigos, C., & Monaghan, J. (2009). Working with teachers; Context and culture. In C. Hoyles, & J. B. Lagrange (Eds.), Mathematics education and technology— rethinking the terrain: the 17th ICMI study (pp. 293–310, Vol. New ICMI Study Series, Vol. 13). Berlin: Springer. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine. Gouseti, A., Potter, J., & Selwyn, N. (2011). Assessing the impact and sustainability of networks stimulated and supported by the NCETM. London: London Knowledge Lab, Institute of Education. Healy, L., & Lagrange, J. B. (2009). Section 3 teachers and technology. In C. Hoyles, & J. B. Lagrange (Eds.), Mathematics education and technology—rethinking the terrain: the 17th ICMI study (pp. 287–345, Vol. New ICMI Study Series, Vol. 13). Berlin: Springer. Hill, Heather, Ball, Deborah, Schilling, Stephen. (2008). Unpacking pedagogical content knowledge: Conceptualizong and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4), 372–400. Hoyles, C., & Lagrange, J. B. (Eds.). (2009). Mathematics education and technology—rethinking the terrain: the 17th ICMI Study (Vol. New ICMI Study Series, Vol. 13). Berlin: Springer. Hoyles, C., & Noss, R. (2003). What can digital technologies take from and bring to research in mathematics education? In A. Bishop, M. Clements, C. Keitel, J. Kilpatrick, & F. Leung (Eds.), Second international handbook of mathematics education. Dordrecht: Kluwer Academic. Hoyles, C., Noss, R., Vahey, P., & Roschelle, J. (2013). Cornerstone mathematics: designing digital technology for teacher adaptation and scaling. ZDM—The International Journal on Mathematics Education, 45(7), 1057–1070. Hung, D., Lim, K., & Huang, D. (2010). Extending and scaling technology-based innovations through research: The case of Singapore. In Organisation for Economic Co-operation and Development (Ed.), Inspired by technology, driven by pedagogy [electronic resource]: a systemic approach to technology-based school innovations (pp. 89–102): OECD Publishing.
91 Jackson, K., Cobb, P., Wilson, J., Webster, M., Dunlap, C., & Appelgate, M. (2015). Investigating the development of mathematics leaders’ capacity to support teachers’ learning on a large scale. ZDM—The International Journal on Mathematics Education, 47(1). Jackson, F. Measuring fidelity of mathematics intervention programme implementations in primary school settings. In C. Smith (Ed.), Proceedings of the British Society for Research into Learning Mathematics, 2012 (Vol. 32, pp. 46–51, Vol. 2): British Society for Research into Learning Mathematics. Kaput, J., & Roschelle, J. (1998). The mathematics of change and variation from a millenial perspective: Now content, new context. In C. Hoyles, C. Morgan, & G. Woodhouse (Eds.), Rethinking the mathematics curriculum (pp. 155–170). London: Springer. Kaur, B. (2015). What matters? From a small scale to a school-wide intervention. ZDM—The International Journal on Mathematics Education, 47(1). Kelly, A. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 115–128. Laurillard, D. (2012). Teaching as a design science: building pedagogical patterns for learning and technology. New York: Routledge. Mowbray, C. T., Holter, M. C., Teague, G. B. T., & Bybee, D. (2003). Fidelity criteria: development, measurement, and validation. American Journal of Evaluation, 24(3), 315–350. Munter, C., Garrison, A., Cobb, P., & Cordray, D. (2010). Evaluating math recovery: measuring fidelity of implementation. Paper presented at the SREE. Noss, R., Sutherland, R., & Hoyles, C. (1991). Final report of the Microworlds Project Vol. II: Teacher attitudes and interactions. London: Institute of Education. Penuel, W. R., Fishman, B. J., Cheng, B., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331–337. Pierce, R., & Ball, L. (2009). Perceptions that may affect teachers’ intention to use technology in secondary mathematics classes. Educational Studies in Mathematics, 71(3), 299–217. Roschelle, J., Shechtman, N., Tatar, D., Hegedus, S. J., Hopkins, B., Empson, S., et al. (2010). Integration of technology, curriculum and professional development for advancing middle school mathematics. American Educational Research Journal, 47(4), 833– 878. doi:10.3102/0002831210367426. Rösken-Winter, B., Schüler, S., Stahnke, R., & Blömeke, S. (2015). Effective CPD on a large scale: examining the development of multipliers. ZDM—The International Journal on Mathematics Education, 47(1). Thompson, M., & Wiliam, D. (2008). Tight but loose: A conceptual framework for scaling up school reforms. In E. C. Wylie (Ed.), Tight but loose: Scaling up teacher professional development in diverse contexts. Princeton: ETS. Vahey, P., Knudsen, J., Rafanan, K., & Lara-Meloy, T. (2013a). Curricular activity systems supporting the use of dynamic representations to foster students’ deep understanding of mathematics. In C. Mouza & N. Lavigne (Eds.), Emerging technologies for the classroom: A learning sciences perspective (pp. 15–30). New York: Springer. Vahey, P., Roy, G., & Fueyo, V. (2013b). Sustainable use of dynamic representational environments: Toward a district-wide adoption of SimCalc-based materials. In S. Hegedus & J. Roschelle (Eds.), The SimCalc visions and contributions: democratizing access to important mathematics (pp. 183–202). New York: Springer. Wenger, E. (1998). Communities of practice. Learning, meaning and identity. Cambridge: Cambridge University Press.
13
92 Knudsen, J. (2010). Scaling Up SimCalc Project. Design and development of curriculum units and professional development (Technical report). Menlo: SRI International. Slavin, R., Sheard, M., Hanley, P., Elliott, L., Chambers, B., & Cheung, A. (2014). Effects of co-operative learning and
13
A. Clark-Wilson et al. embedded multimedia on mathematics learning in key stage 2: final report. Institute for Effective Education: University of York. Zehetmeier, S. (2015). Sustaining and scaling up the impact of professional development programmes. ZDM—The International Journal on Mathematics Education, 47(1).
