Evaluation of the PDST ePortfolio initiative

Evaluation of the PDST ePortfolio initiative Formative Assessment Using ePortfolios

Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O’Hara Dr. Shivaun O’Brien, Paige Poole and Dr. Denise Burns EQI - Centre for Evaluation, Quality and Inspection

.

Dedication This work is dedicated to David McMahon, a role model and colleague to many, who, with selfless commitment and a tireless work ethic, spearheaded - not only the PDST ePortfolio initiative, but also many other aspects of education in Ireland. Ní bheidh a leithéid arís ann.

About (EQI) Centre for Evaluation, Quality and Inspection The Centre for Evaluation, Quality and Inspection (EQI) is a multidisciplinary research group based at DCU Institute of Education. It brings together evaluators, policy analysts, and economists and explores the thematic areas of School Evaluation and Inspection and Culturally Responsive Evaluation and Assessment (CREA). It also hosts the Irish Evaluation Network (IEN) – the national database for evaluators working on the island of Ireland.

Address (EQI) Centre for Evaluation, Quality & Inspection DCU Institute of Education DCU St. Patrick's Campus Drumcondra Co. Dublin Ireland Tel: Web: Twitter:

353-1 - 8842254 www.dcu.ie/ceqie/index.shtml @EQI_DCU

Please cite this publication as: Brown, M., McNamara, G., O’Hara, J., O’Brien, S., Poole, P., and Burns, D. 2017. Evaluation of the PDST ePortfolio initiative: Formative Assessment using ePortfolios. Dublin: (EQI) Centre for Evaluation, Quality and Inspection.

ISBN: 978-1-873769-77-5

© EQI 2017 You can copy, download or print content from this publication for your own use, provided that suitable acknowledgement of the authors is given. All requests for public or commercial use should be submitted to (EQI) Centre for Evaluation, Quality & Inspection, DCU Institute of Education, DCU St. Patrick's Campus, Drumcondra, Co. Dublin, Ireland. Requests for permission to photocopy portions of this material for public or commercial use should also be addressed to (EQI) Centre for Evaluation, Quality & Inspection, DCU Institute of Education, DCU St. Patrick's Campus, Drumcondra, Co. Dublin, Ireland.

Evaluation of the PDST ePortfolio Initiative Formative Assessment Using ePortfolios Dr. Martin Brown, Professor Gerry McNamara, Professor Joe O’Hara Dr. Shivaun O’Brien, Paige Poole and Dr. Denise Burns (EQI) Centre for Evaluation, Quality and Inspection

...

TABLE OF CONTENTS Abbreviations Acknowledgements

viii ix

Executive Summary

xi

Chapter 1 Introduction and Background 1.1 Introduction 1.2 Background 1.3 Deconstructing ePortfolios in Education 1.4 ePortfolio Barriers and Supports

2 3 4 7

Chapter 2 De Constructing Continuing Professional Development in Education 2.1 Introduction 2.2 Exploring Professional Learning 2.3 Core Features of Effective Professional Development 2.4 Key Factors that Affect the Impact of CPD 2.5 Conclusion

12 12 17 20 24

Chapter 3 Overview of PDST ePortfolio Initiative 3.1 Introduction 3.2 About the PDST 3.3 PDST Supports Available to Teachers 3.4 Why Transition Year? 3.5 Overview of the PDST ePortfolio Initiative 3.6 Phases of the PDST ePortfolio Initiative

26 26 27 27 28 29

Chapter 4 Research Methodology 4.1 Introduction 4.2 Research Design used in This Study 4.3 Exploring Guskey’s Framework for Evaluating Professional Development 4.4 Questionnaire Development, Distribution and Analysis 4.5 Interview Coding, Participant Selection and Analysis 4.6 Ethical Considerations 4.7 Limitations of the Study 4.8 Conclusion

33 33 38 39 42 45 46 46

Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | iv

Chapter 5 Presentation and Analysis 5.1 Introduction 5.2 Participant Profiles 5.3 Level 1: Participants’ Reactions 5.4 Level 2: Participants' Learning 5.5 Level 3: Organisation Support and Change 5.6 Level 4: Use of New Knowledge and Skills 5.7 Level 5: Student Learning Outcomes 5.8 Peripheral Barriers and Supports towards ePortfolio Integration in Education

48 48 51 57 62 65 66 70

Chapter 6 Discussion and Conclusion 6.1 Introduction 6.2 Discussion 6.3 Additional Discussion points 6.4 Conclusion

73 73 76 78

Bibliography

80

List of Tables Table 1 Training, award-bearing models adapted from Kennedy (2014) Table 2 Standards-based, and coaching/ mentoring, CPD models adapted from Kennedy (2014) Table 3 Community of practice, action research and transformative CPD models adapted from Kennedy (2014) Table 4 ePortfolios for assessment - online course (PDST Technology in Education) Table 5 Outline of two-day Seminar: Day 1 - Unlocking the doors to ePortfolios Table 6 Outline of two-day Seminar: Day 2 - Unlocking the doors to ePortfolios Table 7 Outline structure of PDST advisor school visits Table 8 Participants prior experience with ePortfolios Table 9 Participants attitudes towards PDST Online ePortfolio course

v | Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios

14 15

16 29 30 30 31 48 52

Table 10 Participants attitudes towards usefulness of PDST seminars on ePortfolios Table 11 Digital literacies obtained by participants (creating, storing and sharing digital files) Table 12 Participants understanding of copyright and the acceptable use of digital files Table 13 Digital literacies obtained by participants (Accessing and responding to peer and student feedback) Table 14 Confidence of the participants to support assessment of learning with ePortfolios in their school Table 15 Confidence of participants to support assessment for learning with ePortfolios in their School Table 16 Confidence of participants to set up and maintain an ePortfolio system for teachers involved in the PDST Transition year ePortfolio implementation project Table 17 Percentage of Schools who have an action plan for ePortfolio implementation Table 18 Likelihood of schools to use the storage functions of an ePortfolio as part of the PDST ePortfolio initiative Table 19 Likelihood of schools to use the showcase functions of an ePortfolio as part of the PDST ePortfolio initiative Table 20 Likelihood of schools to gather information from TY student work samples to improve teaching and learning (Assessment of Learning) as part of the PDST ePortfolio initiative Table 21 Likelihood of schools to provide feedback to TY students via a student/teacher digital workspace (Assessment for Learning) as part of the PDST ePortfolio initiative Table 22 Digital literacies obtained by participants (creating, storing and sharing digital files) Table 23 Students understanding of copyright and the acceptable use of digital files Table 24 Digital literacies obtained by students (responding to peer and teacher feedback)

54 58 59

59

60

60

61 63

63

63

64

64 67 68 68

Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | vi

List of Figures Figure 1 Research model of professional learning activity (Source: Kwakman, 2003: 158) Figure 2 Factors for successful CPD (Source: Bubb and Earley 2009: 25) Figure 3 Multi-phase research design used in the study Figure 4 Exploratory phase 1 Figure 5 Exploratory phase 2 Figure 6 Exploratory phase 3 Figure 7 Individual phases in the multi-phase design Figure 8 Five levels of professional development evaluation Figure 9 Data analysis in qualitative research Figure 10 Components of data analysis: Interactive model

vii | Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios

22 23 33 34 35 36 37 38 44 44

ABBREVIATIONS CPD

Continuing Professional Development

DES

Department of Education and Skills

ICT

Information and Communication Technology

JCT

Junior Cycle for Teachers

JMB

Joint Managerial Board

NCCA

National Council for Curriculum and Assessment

PDST

Professional Development Service for Teachers

TY

Transition Year

Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | viii

ACKNOWLEDGEMENTS A word of thanks to the Professional Development Service for Teachers for their commitment to this project; to the team of post-primary advisors for their work in designing and delivering the professional development throughout the initiative, to Rachel Farrell (Deputy Director. PDST) who initiated and with Brendan Waldron (PDST) led the project throughout its duration. To Sean Gallagher (Deputy Director, PDST), Mark Finlay (Team Leader, PDST) and Anthony Kilcoyne (Team Leader, PDST) for their knowledge and advice regarding the pedagogical use of Digital Technologies and to Ciara O’Donnell (National Director, PDST) for her insights and knowledge on continuing professional development and teacher education. Finally, a sincere depth of gratitude is owed to the participating teachers and students who made the vision for this project a reality in classrooms.

School Inspection in a Polycentric Context| ix

Executive Summary

School Inspection in a Polycentric Context| x

EXECUTIVE SUMMARY

The use of digital tools to enhance the quality of education has become increasingly important in Europe. In the case of Ireland, there is a real commitment to exploring ways in which digital tools such as ePortfolios can enhance teaching and learning across the continuum of education. At a policy level, the Digital Strategy for Schools 2015 -2018 (Department of Education and Skills (DES), 2015) outlines a range of engagements that can occur with ePortfolios including the provision of continuing professional development (CPD) through various sectoral support services such as the Professional Development Service for Teachers (PDST) and the Junior Cycle for Teachers (JCT). The PDST also provides online professional learning opportunities for all registered teachers in Ireland to enhance their understanding of ePortfolios through the PDST’s online learning platform, teachercpd.ie. In addition, as part of the DES ongoing commitment to exploring the use of ePortfolios, JCT and PDST have also participated in European projects on the integration of ePortfolios in education such as the EUfolio project. This report provides an evaluation of a more recent ePortfolio initiative initiated by the PDST, the purpose of which was to pilot a model of ePortfolio support at transition year (TY) level with 24 schools throughout Ireland. The overall outcomes of the evaluation were very positive. There was substantial agreement by participants that the professional development provided by the PDST, particularly the ongoing engagement of PDST management and advisors with participants enabled the initial adoption and understanding of ePortfolio use in the schools. A key theme that emerged was the gradual transfer of professional learning provision from the PDST as CPD providers to the teachers involved in the initiative. Respondents noted that in providing contextualised school based professional development with a strong focus on coaching models of CPD, the PDST empowered staff to take responsibility for their professional learning throughout the initiative. This was seen as being crucial for the building of professional capacity at the school level, which will ensure that practice is sustained beyond the life of the initiative. If this model of CPD is followed by all professional development support services, then the appropriate role of professional development support becomes one of facilitation and enablement as opposed to the generic supports that are quite frequently offered to schools in various jurisdictions. Regular contact with and follow-up by the PDST was also emphasised as a significant factor for the success of the initiative. This appears to be in direct contrast to traditional modes of CPD, where episodic interventions arguably reduce the embedding of new learning. This schoolspecific approach was perceived as growing the individual capacity of schools, regardless of infrastructural or professional capacity issues. In this way, a shared agenda and plan for improved teaching and learning with digital technologies emerged in schools. In parallel, it also appears that the initiative has acted as a springboard that has led to opportunities for teachers in creating what amounts to a self-sustaining professional forum for the exchange of ideas, materials and experience outside of their schools. This outcome is contrasted with traditional single-school whole-staff professional development opportunities, which, although very valuable in many respects, often tend to encourage schools to seek to solve complex, multifaceted problems at the school level. Although laudable, this approach often sees schools address issues that they are unable to address locally in an unsupported manner.

xi | Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios

EXECUTIVE SUMMARY

As was already suggested, an important question is whether it can be shown that learning activities such as online learning that in the long term may reduce exchequer funding can, in fact, replace face to face professional learning. In reality it is always difficult to establish such related effects in educational research however what can be said is that in this initiative, almost all respondents stated that while the online ePortfolio course developed by the PDST provided a high-quality surface level understanding of ePortfolio functions and assessment for learning, a deeper understanding of ePortfolios could have only been achieved via Face to Face group and individual school professional development opportunities. This finding would suggest that future iterations of system level priorities should initially be implemented with introductory online courses and that subsequently professional development providers should provide ‘challenging’ Face to Face professional development opportunities as was the case with this initiative. An important aspect to be considered is how to take this successful initiative's results regarding the implementation of ePortfolios and replicating these results in other schools . This is a difficult question to answer. As indicated in participant responses, many issues need to be resolved before ePortfolios become a standard tool for teaching and learning. These issues relate not so much to equity of access to robust wi-fi and broadband both in and outside of the school environment but instead, the need to, as was the case with the change orientation of teachers in this initiative, challenge assumptions relating to the benefits of ePortfolios as a powerful tool for learning, not only at TY level but across all levels of education. By way of final comment, if the overall goal of the PDST is ‘for it to be widely acknowledged as an innovative, responsive and trusted provider of continuing professional development and support for teachers and school leaders’ (PDST, 2017:13); PDST goals for innovation, responsiveness and trust were undoubtedly achieved in this initiative. Innovative in the sense longitudinal professional development initiatives of this depth and scale has rarely been carried out to the extent that it has been with this initiative. Responsive in the sense that, not only did the PDST efficiently respond to system level priorities such as the enhancement of assessment for learning and twenty-first-century learning skills but they were also responsive to the self-identified, contextualised needs of schools. Trusted in the sense that, in the face of a range of external systemic challenges the PDST initiated an external evaluation of an initiative that sought to enhance various aspects of teaching, learning and assessment across all school types. It is envisaged that the results of this project and subsequent evaluation will allow various stakeholders such as the DES, sectoral support services, Initial Teacher and Further Education providers and other interested organisations to enhance their understanding of the issues, mechanisms, training supports, and the value added attached to the systemic introduction of ePortfolios in Irish education. Lessons learned, it will also be useful to the PDST and DES as they embark on the next phase of supporting schools with the Digital Learning Framework (DES, 2017a;2017b).

Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | xii

Chapter 1 Introduction and Background

School Inspection in a Polycentric Context| iii

INTRODUCTION AND BACKGROUND 1.1

Introduction

The use of ePortfolios to enhance the quality of education has become increasingly important in Europe and other areas of the world. In the case of Ireland, there is a real commitment to exploring the ways in which ePortfolios can enhance teaching and learning across the continuum of education. At the policy level, the Digital Strategy for Schools 2015 2018 (DES, 2015) outlines a range of engagements with ePortfolios, including the provision of freely available online training on ePortfolios and assessment to all registered teachers in Ireland through teachercpd.ie.1 In addition, as part of the DES, ongoing commitment to exploring the use of ePortfolios in education, the PDST has tested a model of ePortfolio support at the TY level with 40 schools throughout Ireland. This report summarises research that set out to evaluate the impact of the PDST ePortfolio initiative that commenced in December 2015 and concluded in May 2017. The research draws on the experiences of participating schools and teachers to analyse the impact of ePortfolio supports that were provided to teachers and the change orientation of schools during this period. The evaluation uses Guskey’s (2000) five levels of evaluation as a framework for investigating the impact of the supports offered. Guskey (2000) suggests that effective CPD occurs at five levels: (1) participant reactions; (2) participant learning; (3) organisational support and change; (4) participant use of new knowledge and skills and (5) pupil learning outcomes. The report is divided into six chapters: Chapter 1 deconstructs the various functions of an ePortfolio and provides an analysis of ePortfolio barriers and supports in education. Next, chapter 2 describes good practice in CPD, and from a review of the literature, key factors that improve the impact of CPD are presented. This is followed by chapter 3, which provides an overview of the PDST and supports offered. The chapter also describes the various stages of the ePortfolio initiative. Chapter 4 describes the methodology used in the evaluation. This is followed by chapter 5, which provides a summary of the analysis of the impact of the ePortfolio initiative. Finally, chapter 6 provides an overall interpretation of the evaluation, as well as a discussion and concluding remarks relating to the overall impact of the initiative. It is envisaged that the results arising from this initiative will allow various stakeholders, such as the DES, sectoral support services, Initial Teacher and Further Education providers and other interested bodies, to enhance their understanding of the issues, mechanisms, training supports and the value added to the systemic introduction of ePortfolios in Irish education. As for the Lessons learned, it will also be useful to the PDST and DES as they embark on the next phase of supporting schools with the Digital Learning Framework (DES, 2017a;2017b). 1

TeacherCPD.ie is a course management system hosting online continuing professional development courses for teachers in Irish primary and post primary schools. The service is provided and managed by PDST Technology in Education. It contains courses from PDST Technology in Education specifically on ICT in teaching and learning, as well as some courses on other topics from PDST.

2 | Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios


Background

For more than two decades, it has been recognised that incorporating information and communications technology (ICT) into Irish schools poses ‘a major national challenge that must be met’ (DES, 1997: 2). The DES has sought to meet this ‘national challenge’ with the publication of various policy frameworks, such as Schools IT 2000 (DES, 1997) and, more recently, the Digital Strategy for Schools (DES, 2015). The Digital Strategy presents the clear intention to support the introduction of ePortfolios in Irish compulsory education. An objective of the first theme in the Digital Strategy is to ‘promote the use of digital portfolios (ePortfolios) for primary and post-primary students’ (DES, 2015: 27). This is no surprise given the espoused benefits of ePortfolios in an educational setting such as the enhancement of learning, assessment and professional development (see, Barrett 2010; Stefani, Mason and Pegler 2007). Butler et al. (2013: 2) notes that a ‘long-term vision for education’ is needed so that students can learn and receive all that is required to succeed in today’s environment. In addition to a vision, it has been argued that a series of practical steps are also needed. One such framework is the Digital Learning Framework (DES, 2017a;2017b). Indeed, Mulkeen (2003), among others, has critiqued the historical efforts to introduce ICT initiatives in Irish education, arguing that one of the main reasons for the unsatisfactory progress in the field was that policies often offered no guidance for teachers on how to integrate ICT successfully. Perhaps, taking this critique into account, and based on the experience of the EUfolio project (DES, 2015) and through collaborative efforts between the DES, the Teaching Council and other relevant bodies, the actions to be taken to implement ePortfolios in compulsory education as described in The Digital Strategy for Schools should focus on the promotion of ePortfolio pedagogy and purpose. As Marshal and MacNair (2005) observe, ‘the traditional debate on the use of portfolios has been on educational purpose (Meyer and Tusin, 1999), the embedding of reflection (Borko et al., 1997) and the demonstration of evidence (Klenowski, 2000), or any combination of these three interests’ (1). However, establishing a national, equitable ePortfolio system in compulsory-level Irish education is more than just a question of policy and promotion, not least because of the perceived financial burden associated with setting up and maintaining such a system. This is arguably because of what Buzzetto-More (2007), in reference to Tellefsen’s (1995) constituent orientation analysis framework, calls the often fragmented ‘key constituencies’ associated with ePortfolio implementation (educators, learners, system designers, and system managers). As outlined by Pelgrum (2001), the embedding of any component part of ICT in education involves many factors and includes many variables that need to be considered depending on the purpose and functions of an ePortfolio, of which there are many.

Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | 3


Deconstructing ePortfolios in Education

ePortfolio as a concept originates from its predecessor, the ‘traditional paper-based portfolio’ (Cyprus Pedagogical Institute, 2015: 10). Arguably, both traditional portfolios and ePortfolios share common educational purposes, such as storage, showcasing and assessment, but have differing characteristics. Perhaps the most apparent difference between the two types is the digital benefit of ePortfolios when it comes to the administration of data (Fitch et al., 2008). There are, however, some portfolio definitions that are specific to their context (Gibson, 2006: 136) and recent developments, such as the rise of cloud-based storage, for example, have led to, the emergence of a range of definitions in recent years (see, for example, Chang et al., 2014). It is to these that we will now turn. Paulson et al. (1991) define portfolios as predominantly having a storage and showcase function: ‘A portfolio is a purposeful collection of student work that exhibits the student’s efforts, progress, and achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for selection, the criteria for judging merit, and evidence of student self-reflection’ (60). Wolf and Siu-Runyan (1996: 31) highlight that portfolios can also be used to showcase an ‘advancement of student learning’. Turning to ePortfolios, Barrett (2010) describes them as ‘an electronic collection of evidence that shows your learning journey over time. ePortfolios can relate to specific academic fields or take on a more composite nature acting as a repository for information relating to, for example, lifelong learning. Evidence may include writing samples, photos, videos, research projects, observations by mentors and peers, and/or reflective thinking’ (6). It is possible to argue, therefore, based on this emerging understanding of the potential offered by emerging technologies, that ePortfolios ‘go beyond the limits of paper-based portfolios’ (Theodosiadou and Konstantinidis, 2015: 18) and can be seen as reforming portfolios ‘to enrich students’ learning experience’ (DES, 2008: 2). Similarly, while also recognising the increasing use of cloud storage and dynamic workspaces that allow users to share and simultaneously collaborate on digital content, EUfolio (2015a) offers a contemporary definition of an ePortfolio as a tool that allows students to store, reflect upon and showcase their work: ePortfolios are student-owned, dynamic digital workspaces wherein students can capture their learning and their ideas, access their collection of work, reflect on their learning, share it, set goals, seek feedback and showcase their learning and achievements. (8)


INTRODUCTION AND BACKGROUND INTRODUCTION AND BACKGROUND 1.2

Background

However, because ePortfolios serve different purposes, they inevitably fall into distinct categories, such as ‘developmental, assessment and showcase portfolios’ (Regis University, 2003), depending on the way in which they can further a student’s journey and on the curriculum specifications and assessment requirements of the education system. For Barrett (2010: 7), ePortfolios may be used for the purpose of ‘showcase’ or ‘accountability’ and ‘learning or reflection’. This issue of purpose is critically important. Thus, for a national ePortfolio system to be implemented, its purpose and how it links to curriculum specifications need to be explicitly stated. Barrett (2010: 8) highlights that ‘most ePortfolio systems tend to emphasise the showcase (portfolio as product) rather than the workspace (portfolio as process)’ function of ePortfolios. However, this focus on the product rather than the process appears to contradict the emerging aim of many education systems such as Ireland, which are seeking ultimately to ‘shift from course-centred learning to student-centred learning’ (McCloud, 2004: 6). In summary, therefore, it might be argued that ePortfolios have three overarching functions: (1) ePortfolios as storage; (2) ePortfolios as showcase and (3) ePortfolios to support assessment. 1.3.1

ePortfolios as storage

Chang et al. (2014: 188) note that ePortfolios can be accumulated and conserved through both ‘the Internet and cloud storage’, and Barrett (2010: 9) remarks that they can also be ‘stored on a server ... locally’. Thus, ePortfolios perform their function of storage in diverse ways, which naturally requires a robust ICT infrastructure. Ochola, Achrazoglou and Anthony (2015: 16) emphasise that this ‘invaluable infrastructure’ is a key criterion for the ‘successful implementation of an ePortfolio framework’. A robust system is obviously necessary for the storage of artefacts, but teachers and students will need to adapt to this new mode of evidence storage, and consideration must also be given to their capacity to collect, store, upload and download these artefacts (EUfolio, 2015a). It is also evident that teachers will develop new roles (McGhee and Kozma, 2001) and will be required to pass on their knowledge to guide students ‘on the types of artefacts to save’ (Barrett, 2010: 9).

S Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | 5 chool Inspection in a Polycentric Context| 5

INTRODUCTION AND BACKGROUND INTRODUCTION AND BACKGROUND 1.3.2

ePortfolios as showcase

Stefani, Mason and Pegler (2007: 18) suggest that ePortfolios display ‘a collection of the highest quality work that the student is capable of, and typically shows the range of work, perhaps with an idea of progression over time’. Students ‘showcase their reflections and achievements as well as contributions and feedback from peers and teachers’ (EUfolio, 2015a: 13). ePortfolios, therefore, have the potential to foster ‘reflective learning’ (Roberts et al., 2005: 7). However, as with any digital artefact, the standards and the acceptable use of ePortfolios need to be made explicit in a school’s or organisation’s acceptable usage policy (AUP). As with all forms of digital media, this, according to the Consortium for School Networking, will ‘protect students from harmful content on the Internet’ and ‘provide students with good access to digital media to support engaged learning’ (2013: 2). Indeed, as with the storage function of ePortfolios, through exemplars of ePortfolio integration within existing ICT AUPs, it is important that teachers and students are educated on the legal requirements of and ramifications relating to the storage and showcase of digital content. In the case of Ireland, for example, this can be done through various support structures such as the PDST webwise initiative. 2 1.3.3

ePortfolios to support assessment

Whereas summative assessment is often associated with end-of-topic testing, ePortfolios for summative assessment arguably can be linked to their first purpose, which is the storage of student artefacts (EUfolio, 2015b). On the other hand, ePortfolios for formative assessment are centred on a ‘collaborative, continuous discourse between teacher and student’ (EUfolio, 2015b: 5) and as with all forms of formative assessment can be used to revisit the teaching and learning processes to accommodate student needs (Black and William, 2010). Indeed, Hattie and Timperley (2007) argue that ePortfolios are important drivers of continuous feedback, suggesting that ‘It is the feedback information and interpretations from assessments, not the numbers or grades, that matter’ (2007: 104). However, as with all forms of assessment for learning where an individual provides feedback, the primary participant responsible for the effective use of ePortfolios for formative assessment is the teacher, with his or her ability to ‘promote learning’ through effective feedback (Rate, 2008: 22). This highlights the requirement for all educators working in the system to have an understanding of and capacity to provide effective feedback for improved learning. 2

Webwise is an internet safety initiative focused on raising awareness of online safety issues and good practice among students, their parents and teachers.



Background

Examples of effective feedback are provided in the NCCA Assessment toolkit (NCCA, 2014). In the Irish context, Hislop (2015: 14, in reference to Hattie, 2008) notes, ‘It is also selfevident that this sort of environment and teaching requires adequate teaching resources in each school, but also an investment in the quality and ongoing professional development of teachers’. Thus, ePortfolio deployment in education not only requires a robust ICT infrastructure and ePortfolio standards to showcase students’ work, but also demands a greater understanding of ‘pedagogy and technology’ (EUfolio, 2015a: 18). These enable ePortfolios to be implemented effectively in the classroom and to be supported by the appropriate technology. To truly harness the benefits of ePortfolios in education, their deployment also needs to promote capacity building in the context of providing effective feedback for improved learning. As Cramer (1993: 72) observes, ‘the portfolio itself is not a type of assessment, but an assessment tool’. As illustrated in the preceding example, there are a number of barriers and supports that need to be considered to harness the true potential of ePortfolios as a dynamic tool for teaching and learning. 1.4

ePortfolio Barriers and Supports

ePortfolio usage is inevitably challenging to initiate over a short period, and barriers to ePortfolio implementation need to be addressed. Strudler and Wetzel (2005: 418) identify a number of these barriers, highlighting issues ‘such as governance, leadership, and grants’. Lorenzo and Ittelson (2005) further specify the multiple issues that need to be explored, here focusing on ‘hardware and software, support and scalability, security and privacy, ownership and intellectual property, assessment, adoption and long-term maintenance’ (8). Young and Lipczynski (2007: 13), in reference to Love and Cooper (2004), identify common issues that need to be considered when designing ePortfolios for educational purposes: 1. 2. 3. 4.

The focus remains solely on the technical side rather than the administrative side. ePortfolio is used as a content management system, rather than an interactive learning tool. Stakeholders’ views or needs are not included in the development of an ePortfolio. ePortfolio is not fully integrated into the curriculum.

Although ePortfolios have been successfully implemented in the component parts of some countries, education systems such as England and Wales (see, Eifel, 2016), the literature on integration reveals significant challenges that need to be addressed. A study conducted in South Africa finds that the most significant constraint on ePortfolio implementation was an infrastructural one.

S Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios | 7


Background

This study notes that among participating schools, only ‘10% indicated ready internet access’ (Kok and Blignaut, 2009: 5). In addition to infrastructural problems, this research indicated that barriers to ePortfolio implementation also include issues of access, connectivity, socio-economic status and lack of integrated understanding of ICT, perceived skills deficits and lack of confidence. In the absence of a stable ICT infrastructure and what might be described as robust Wi-Fi, it is no wonder, therefore, that implementing an ePortfolio system that is entirely technology-based is likely to be a problem for any education system that attempts to integrate ePortfolios into core teaching and learning practices. The EUfolio project also reveals barriers unrelated to the technological infrastructure of an ePortfolio ecosystem. Citing the example of Cyprus, it reports on a situation where participants in an ePortfolio project voiced their concerns about the extra organisation required, the efficiency of their classes and the authenticity of student. Slovenian and Lithuanian participants also express worry that ePortfolios overemphasise the skills associated with their implementation, rather than the learning process itself work (EUfolio, 2015b: 5). Lithuanian and Cypriot teachers also stated that ‘there was a need to increase motivation for students’ work’ (EUfolio, 2015b: 10). This contrasts with Akçil and Arap’s assertion that the use of ePortfolios by students will ‘motivate them to study’ (2009, cited in Yastibas and Cepik, 2015: 517). It is quite often argued that because an ePortfolio exists in an online virtual environment, its use will lead to the development of the widely desired ‘twenty-first century skills’. However, it is suggested that the learning objectives and values be explored in detail with students before ePortfolios are implemented on a mass scale (Yastibas and Cepik, 2015: 515). In other words, in the case of Ireland, where there is a system dominated in the form of the ‘Points System’ by numerical conjectures of quality in the form of test scores, it is essential that not only teachers, but also, student values on formative assessment are challenged and explored. In the Irish education context, barriers to the integration of ePortfolios also mirrored those of their European counterparts. The EUfolio study finds that Irish teachers were of the view that a significant amount of time was needed to assess ePortfolios effectively (EUfolio 2015b), although one teacher points out that, as with any new initiative, this became easier with practice.



Background

Teacher CPD is thus vital if ePortfolios are to be integrated into Irish education. Indeed, Baylor and Ritchie (2002: 4) similarly observed that any form of ICT integration requires training, because ‘technology will not be used unless faculty members have the skills, knowledge, and attitudes necessary to infuse it into the curriculum’. Spanish teachers also highlight that there was a disconnect between ePortfolios and the curriculum, and the use of ePortfolios should be embedded into the curriculum to aid teachers with its establishment (EUfolio, 2015b: 16). In the past, this perspective has also resonated with the disconnect between ICT and the Irish compulsory education more generally. As stated by a member of the National Council for Curriculum and Assessment (NCCA) in Johnston (2014: 130) The discussion around curriculum and assessment has tended to be exclusive of or has tended to exclude to some extent or not be connected with discussion around ICT – we are simply part of a trend in that regard…we see that trend shifting and changing. There are, however, reasons to be optimistic. Despite the low priority accorded to investment in the knowledge economy during the period of sustained economic growth in the early 2000s (Austin and Hunter, 2013: 187), certain support networks and policy developments that will aid ePortfolio implementation are emerging in Ireland. For example, the Digital Strategy for Schools ‘sets out a clear vision for the role of ICT in teaching, learning and assessment for schools in Ireland’ (DES, 2015: 4) and acts as a blueprint to ‘promote the use of digital portfolios (ePortfolios) for primary and post-primary students’ (27). The complementary Digital Learning Framework for primary and post-primary schools (DES, 2017a;2017b) also provides ‘a common reference with descriptors of digital competence for teachers and school leaders promoting innovative pedagogical approaches which embed the use of digital technologies’ (1). As a result of Ireland’s participation in the EUfolio pilot project, documents and guides (such as: EUfolio, 2015a; EUfolio, 2013 and EUfolio, 2015d) have been produced to aid in the successful incorporation of ePortfolios into Irish schools. The EUfolio Process Specification document presents the work of the EUfolio team ‘for those who want to follow EUfolio’s model’ (Department of Interactive Media and Educational Technologies, 2015: 3). The EUfolio ePortfolio Implementation Guide for Policymakers and Practitioners (EUfolio, 2015a: 5)also ‘illustrates the actions needed to introduce ePortfolios into schools’.



Background

Other ePortfolio supports and specifications are also available to teachers in Ireland, and these supports include structured and proportionate school support provided by the PDST, as well as a 5-hour online ePortfolio training session for all registered primary and postprimary teachers provided by PDST – Technology in Education (PDST, 2015). Ultimately, however, the primary aim of any system - level initiative is to offer high-quality professional learning from those constituencies tasked with its implementation. The next chapter of the report describes the types of CPD approaches and provides an overview of good practice and key factors that improve the impact of CPD.


THE NORTHERN IRELAND EDUCATION SYSTEM

Chapter 2 Deconstructing Continuing Professional Development in Education

School Inspection in a Polycentric Context| 6

DE CONSTRUCTING CONTINUING PROFESSIONAL DEVELOPMENT IN EDUCATION

2.1

Introduction

Because the purpose of this study is to evaluate the impact of a CPD intervention, it is important to explore the current literature on professional development. This chapter begins by exploring professional learning. After this, the next two sections describe the various definitions and models of CPD. From this, the next two sections provide a description of research relating to the core features of effective professional development and the key factors that affect the impact of CPD. Finally, the chapter concludes with closing remarks of effective CPD and how it applies to education. 2.2

Exploring Professional Learning

Professional learning is recognised as an important and ongoing aspect of a teacher’s working life because it has the potential to impact positively on teachers’ performance, effectiveness, professional development, identity, sense of self-efficacy and job satisfaction. Because teacher quality directly influences student outcomes, professional learning is encouraged and supported across many jurisdictions through evidence-based government policy. The term ‘continuing professional development’ (CPD) refers to the professional development that takes place following initial teacher education. However, much of the current literature on the topic uses the more overarching term ‘professional development’, which may also encompass pre-service training. Given the changing nature of education and potentially long teaching careers, professional development is considered a lifelong pursuit for teachers, as acknowledged by the European Commission (2012): Teaching competences are [...] complex combinations of knowledge, skills, understanding, values and attitudes, leading to effective action in situation. [...] The range and complexity of competencies required for teaching in actual societies is so great that any one individual is unlikely to have them all, nor to have developed them all to the same high degree. [...] Teachers’ continuous professional development is, thus, highly relevant both for improving educational performance and effectiveness and for enhancing teachers’ commitment. (8-9) Guskey (2000:16) similarly suggests that CPD is designed ‘to enhance the professional knowledge, skills and attitudes of educators so that they might, in turn, improve the learning of students’. Overall, quality professional learning has been shown to benefit school performance and the realisation of national education system priorities (OECD, 2010). CPD is also associated with notions of accountability and fitness to practice on one end of the continuum and with concepts of teacher agency and personal development on the other, although these concerns are not mutually exclusive (Swennen, 2013).



2.1

Introduction

In many countries, participation in teacher CPD has become a prerequisite for promotion, salary upgrades and maintaining employment (European Commission/EACEA/ Eurydice, 2013). 2.2.1

Definitions of CPD

The literature on CPD for teachers suggests a range of definitions, including: ‘any activity or process intended to change any combination of the following: teachers’ beliefs and attitudes, teachers’ knowledge and teachers’ classroom practice’ (Clarke, 1991: 1), and the OECD (2009: 49) similarly defines professional development as ‘activities that develop an individual’s skills, knowledge, expertise and other characteristics as a teacher’. Sowder’s (2007) description of professional growth is ‘marked by change in teachers’ knowledge, beliefs, and instructional strategies’ (161). Day (1999) provides a more comprehensive definition of professional development that considers the context and process of teacher learning: Professional development consists of all natural learning experiences and those conscious and planned activities which are intended to be of direct or indirect benefit to the individual, group or school, which contribute through these, to the quality of education in the classroom. It is the process by which, alone and with others, teachers review, renew and extend their commitment as change agents to the moral purposes of teaching; and by which they acquire and develop critically the knowledge, skills and emotional intelligence essential to good professional thinking, planning and practice with children, young people and colleagues throughout each phase of their teaching lives. (4) This is very much in keeping with the emphasis in research on lifelong professional learning for teachers as an essential and expected component of professional activity and school improvement (Putnam and Borko, 2000; Sleegers, Bolhuis and Geijsel, 2005; Dam and Blom 2006). In this sense, teachers are expected to be ‘reflective practitioners’ and schools to be ‘learning organisations’. Claiming that the teacher’s role is not only causal but also transformative, Collinson et al. (2009) suggest that teachers ‘require advanced continuous learning as well as opportunities to engage in dialogue and inquiry to create new knowledge. They need opportunities to work collaboratively, disseminate their learning, and contribute to their own, their colleagues’ and the Organisation’s continuous improvement’ (4). Bubb and Earley (2009) argue that some teachers have a limited view of professional development, thinking of it mainly in terms of inputs or activities to be engaged in rather than as the actual development of their knowledge and expertise. Goldsmith et al. (2014: 20) claim that teacher learning ‘is often incremental, nonlinear, and iterative, proceeding through repeated cycles of inquiry outside the classroom and experimentation inside the classroom’.



2.2.2

Models of CPD

A great deal has been written on the criticism of ‘one-shot’, deficit-mastery models of professional development and approaches where professional development is something that is done to teachers (Clarke and Hollingsworth, 2002). Therefore, it is clear that not all CPD is effective or productive for the teacher, the school or the education system as a whole. Because time is often a barrier to teacher participation in programmes, it is important that such programmes be designed and delivered in a manner that would effectively enhance the quality of teaching and raise student achievement. CPD should, therefore, provide ‘value for money and some signs of impact’ (Powell, 2003: 390). Kennedy (2014) outlines nine key models of CPD: training; award-bearing; deficit; cascade; standards-based; coaching/mentoring; community of practice; action research; and transformative models of CPD. The dominant characteristics of each approach, as discussed by Kennedy, have been assembled as a list of points and set out in Tables 1-3.

Models

Key Characteristics ●

● ● ● Training ● ● ● ● Award Bearing

● ● ●

Supports a skills-based, technocratic view of teaching, whereby CPD provides teachers with the opportunity to update their skills, to be able to demonstrate their competence Generally delivered by an ‘expert’ Participant passive Usually delivered off-site and often lacks connection to the classroom context Relates to a standards-based view of teacher development and the achievement of national standards Central control and training needs identified and provided nationally Effective for introducing new knowledge but not for the application of training in practice Emphasises, the completion of award-bearing programmes of study – usually, but not exclusively, validated by universities Quality assurance provided by a validating body Standardises the experiences of those working toward chartered teacher status Perceived to be ‘academic’ as opposed to ‘practical’ although usually a combination of both

Table 1: Training and award-bearing models adapted from Kennedy (2014)



2.2.2

Models of CPD

Models

Key Characteristics ● ●

Deficit

● ●

● ●

Cascade

● ● ● ● ●

Standards based

● ● ● ● ● ● ● ●

Coaching/ Mentoring

● ●

●

Designed specifically to address a perceived deficit in teacher performance Attempts to remedy perceived weaknesses in individual teacher performance Related to concepts of ‘performance management’ Appears to attribute blame for poor performance to individual teachers as opposed to organisational and management practices or collective responsibility Assumes the need for a baseline measure of competence Involves individual teachers attending ‘training events’ and then cascading or disseminating the information to colleagues Commonly employed where resources are limited Involves sharing learning with colleagues but misses out on the experience of participation, collaboration and ownership Generally used for passing on skills or knowledge but rarely focuses on values or rationale Emphasis on the ‘professional actions’ as a way of demonstrating that the standard has been met Standards provide a common language, supporting teachers to engage in dialogue about their professional practice Can scaffold professional development Ignores the notion of teaching as a complex, context-specific political and moral endeavour Focuses on the competence of individual teachers at the expense of collaborative and collegiate learning Central control and uniformity May narrow the range of potential conceptions of teaching to focus on quality assurance and accountability Importance of the one-to-one relationship, generally between two teachers Purpose of the coaching/mentoring model varies, for example, mutually supportive and challenging or hierarchical and assessment driven Coaching more skills based and mentoring involves an element of counselling and professional friendship (novice/experienced teacher) Can take place within the school context and can be enhanced by sharing dialogue with colleagues Can support either a transmission view of professional development, where teachers are initiated into the status quo by their more experienced colleagues or a transformative view where the relationship provides a supportive, but challenging forum for both intellectual and affective interrogation of practice Focuses on confidentiality as opposed to accountability (assessment)

Table 2: Standards-based, and coaching/mentoring CPD models adapted from Kennedy (2014)



2.2.2

Models of CPD Models

Key Characteristics ● ●

Community of Practice

● ● ● ● ● ● ●

Action Research

● ● ● ● ● ●

Transformative ●

Generally involves more than two people, and would not necessarily rely on confidentiality Learning happens as a result of community and its interactions, not merely as a result of planned learning episodes such as courses Members of the community set and control the agenda Professional development is not a form of accountability or performance management Can potentially serve to perpetuate dominant discourses in an uncritical manner Can act as powerful sites of transformation, where the sum total of Individual knowledge and experience is enhanced significantly through collective endeavour Involves participants as researchers with a view to improve quality of action within a particular social situation Can be an approach used by communities of practice or by individuals Emphasis on practitioner development and active engagement Encourages teachers to see research as a process as opposed to a product Lessens the dependency on externally produced research and shifts power to teacher/s Develops the personal autonomy Central characteristic is the combination of practices and conditions that support a transformative agenda Promotes an awareness of issues of power, that is, whose agendas are being addressed through the process Relies on tensions: only through the realisation and consideration of conflicting agendas and philosophies, can real debate be engaged in among the various stakeholders in education, which might lead to transformative practice.

Table 3: Community of practice, action research and transformative CPD models adapted from Kennedy (2014) The models presented by Kennedy (2014) highlight a move away ‘from a technical-rationaltop down approach to CPD, toward a more cultural-individual interactive approach’ (Caena, 2011: 4). Therefore, CPD may include an element of training, more effective CPD may include a variety of approaches such as ongoing workshops, opportunities for reflection, follow-up, study, application of learning, observations and assessment.



2.2.2

Models of CPD

This reflects the long-term nature of learning and uses strategies that are more likely to lead teachers to improve their practice as professionals. The key difference between opposing models of CPD is articulated by Hargreaves (1994): The INSET [In-Service Education and Training] model during periods of reform treats teachers as needing occasional injections to pep them up, calm them down, or ease their pain. The professional development model requires a different metaphor: Unless teachers are offered through professional development a regular and balanced diet, they will not be effective practitioners. (430) Despite this widespread criticism, the traditional approach of ‘in-service-training’ continues to dominate in many countries, and it is argued by Day and Sachs (2004) to be the most costeffective way to reach large numbers of teachers, hence remaining a fundamental instrument of reform agendas.

2.3

Core Features of Effective Professional Development

For decades, policy makers have sought to identify the features of professional development that are most effective in raising student achievement. Various significant studies indicate that not all approaches to teacher CPD are equally effective in terms of their impact on student outcomes (Carpenter et al., 1989; Kennedy, 1998; Clewell et al., 2004; Yoon et al., 2007; Wayne et al., 2008). This suggests, therefore, that any investment in CPD should have a strong research base. Data from the 2013 TALIS study indicates that the higher the exposure of teachers to highquality professional development, the greater the chances are that they report using a wide variety of methods in the classroom (Barrera-Pedemonte, 2016). This research argues that certain features of professional development are positively associated with better educational outcomes and that the development of ‘quality teachers’ requires the provision of ‘quality professional development’. Desimone (2009: 183) sets out the characteristics of professional development that she identified from various studies (Hawley and Valli, 1999; Kennedy, 1998; Wilson and Berne, 1999; Jeanpierre, Oberhauser, and Freeman, 2005; Snow-Renner and Lauer, 2005; Johnson, Kahle, and Fargo, 2007; Penuel et al., 2007; Yoon, 2007; Timperley et al., 2007) and that which she claims, ‘are critical to increasing teacher knowledge and skills and improving their practice, and which hold promise for increasing student achievement’. These include (a) content focus; (b) active learning; (c) coherence; (d) duration; and (e) collective participation.



2.2

Models of CPD

Desimone explains each characteristic as follows: ●

●

● ●

●

Content focus: Activities that focus on subject matter content and how students learn that content with increases in teacher knowledge and skills and improvements in practice. Active learning: Active as opposed to passive learning opportunities. Can involve various approaches, including active workshops; observing expert teachers or being observed followed by interactive feedback and discussion; reviewing student work in the topic areas being covered; and leading discussions. These approaches allow teachers to practice new techniques and reflect on them. Coherence: The extent to which teacher learning is consistent with teachers’ knowledge and beliefs. Duration: Professional development activities should be of sufficient duration, including both span of time over which the activity is spread (e.g., one day or one semester) and the number of hours spent in the activity. In a study by Yoon et al. (2007), activities that had 14 hours or more of learning had a positive impact on student learning, although the reality was that professional learning activities rarely involved this much time. Collective participation: Collaborative and collegial learning activities that may involve participation of teachers from the same school, providing opportunities for staff interaction and discourse.

A number of these core characteristics are echoed in Schleicher’s (2016) OECD report: The most effective forms of professional development seem to be those that focus on clearly articulated policies, provide ongoing school-based support to classroom teachers, deal with subject-matter content as well as instruction strategies and classroom management techniques, and create opportunities for teachers to observe, experience and try new teaching methods. (44) In summary, ‘much of the research on effective teacher professional development has identified activities that are intensive, sustained, collaborative, and focused on materials and problems of practice as having more impact on teachers’ knowledge, classroom practices and student achievement’ (Opfer, 2016: 7). Teacher learning can involve a diversity of activities in a variety of settings. CPD can include informal, on-the-job learning, as Guskey (2000) suggests: ‘If we view professional development as an ongoing, job-embedded process, every day presents a variety of learning opportunities’ (19). Generally, it is seen as a more systematic approach to teacher learning one that frequently combines theory and practice.



2.2

Models of CPD

The TALIS study (OECD, 2014) shows that on average, across the study's participating countries, ‘about 88% of teachers report engaging in some professional development (defined as having taken part in at least one activity in the previous 12 months) over the survey period’ (97). This indicates that CPD is a common feature of teachers in most participating countries. The most frequent type of professional development undertaken by lower secondary teachers is courses and workshops (71%), followed by attending education conferences or seminars (44%) and participation in a teacher network (37%), although there is a wide variation between countries.

Bubb and Earley (2009: 30) categorise forms of CPD into four overlapping groups, as follows:

●

Individual/thinking; reading books, periodicals and the educational press; selfstudy; keeping a learning log or reflective diary.

●

Within school/talking to other staff (peers and those with expertise); coaching/ mentoring; training days; staff/team meetings; being observed; discussing a lesson; observing someone teach; collaborative planning; team teaching; listening to pupils’ views; observing some learners; seeing the world through the eyes of a pupil; action research groups; trying things out and doing things differently; taking on a new role; training others.

●

Cross-school networks/formal and informal networks; visiting other schools, similar to or different from yours; reading and talking to others; working with people from other schools; networks of local schools or ones set up for a specific project; developing people from other schools.

●

Other external expertise, for example one-day events, courses leading to a qualification or status; blended learning programmes that involve some external expertise and school-based activity; conferences; working with or seeking advice from consultants, local authorities, universities, government agencies or subject associations.

According to Bubb and Earley (2009), staff development ‘[involves] discussing, coaching, mentoring, observing and training others was said to be highly effective’ (31).



2.2

Models of CPD

McArdle and Coutts (2010) argue that combining aspects of the literature on communities of practice together with literature on reflection offers improved CPD experiences for teachers compared with either concept can provide alone. They suggest that such an approach is most powerful when used in conjunction with teachers taking action together ‘to make shared sense of action and practice’ (209). They propose principles for the design of a CPD programme, as follows: 1. Both action and reflection are required, and both have individual and social dimensions. 2. Both reflection and action are concerned with individual and social identity formation. 3. Reflection must involve a challenge and have a critical edge. 4. Sense-making through engagement is necessary to integrate the dimensions of reflection and action. 5. Sense-making and engagement are self-sustaining and lead to professional renewal. 2.4

Key Factors that Affect the Impact of CPD

Although there appears to be consensus in the literature on the key characteristics of effective professional development, research also indicates that such effects are mediated by a number of factors, key among them being: prior teacher knowledge and practice in the classroom, a supportive school context and teacher beliefs that are conducive to learning and improvement (Ingvarson et al., 2005; Fishman et al., 2003; Guskey and Sparks, 2004; Garet et al., 2001; Kennedy, 1998). Opfer (2016) also claims that individual beliefs and practices, as well as the school-level context, are the key factors that impact the effectiveness of professional development activities. In relation to the former, she concludes that: ...the literature on teacher beliefs about pedagogy, self-efficacy, preparedness, and satisfaction suggests that some teachers may have individual characteristics that lead them to be more amenable to professional learning and subsequent changes in their teaching practice than others. Teachers who hold constructivist pedagogical beliefs and who have typical levels of self-efficacy, feelings of preparedness for their teaching assignment and satisfaction with their teaching may participate in more learning activities and be more willing to try out new practices because they see teaching efficacy as incremental and changeable. Teachers who hold transmissive pedagogical beliefs, who suffer from low self-efficacy and feel unprepared or dissatisfied with their teaching, may be less interested in participating in professional development and also less willing to try new practices. Teachers with extremely high levels of belief in their teaching may also be less likely to participate in professional development because they hold static notions of teaching efficacy. (10)



2

Models of CPD

Discussing the importance of school context as a factor, Opfer (2016) claims that the norms of the school and its structures and practices, influence teachers’ professional learning. As such, she explains that school-level beliefs and normative control ‘will sanction an individual teacher’s practice when that practice violates group pedagogical beliefs’ (10) and is especially felt by new or inexperienced teachers. Other unsupportive conditions include a lack of coordination and leadership, little collegial activity and no obvious commitment to professional development. This is in contrast to instructional-focused leadership, which is associated with teacher participation in professional development. Kwakman (2003) explores these elements in more detail, identifying 13 different factors that impact teacher professional learning: five personal factors, five task factors, and three work environment factors. The personal factors identified by Kwakman are derived from the adult learning theory and social psychological theory of work stress. These refer to meaning that is attached to the teacher’s professional role and responsibilities; the extent to which CPD activities are perceived as feasible and meaningful and emotional exhaustion and loss of personal accomplishment. The task factors acknowledge that stress, as well as learning, result from job demands and worker’s discretion to meet such demands. The assumption here is that learning and growth will occur when job demand and personal autonomy are high. The task factors include the pressure of work; emotional demands of the job; job variety (diversity and learning opportunities); autonomy (freedom to determine one’s approach to work); and participation (influence a teacher has over the working environment and opportunities to take part in decision making). Kwakman’s final three factors that address the work environment include three types of support: management support; collegial support; and intentional learning support. These factors suggest that the school ‘has to provide an environment in which participation in professional learning activities is widely appreciated and therefore intentionally stimulated’ (157).



2

Models of CPD

Figure 1: Research model of professional learning activity (Source: Kwakman, 2003: 158) Of these three categories, Kwakman indicates that personal factors appear to have the most significant impact on teacher participation in professional development activities when compared to task factors and work environment factors. This view is supported by Geijsel et al. (2009), who find that ‘psychological’ factors (e.g., teachers’ sense of self-efficacy and internalisation of school goals into personal goals) appeared to have a greater impact than organisational factors (e.g., leadership, opportunities and support for learning) on teacher participation in CPD activities. McMillan et al. (2016) consider ‘motivating and inhibiting factors’ as being related to teacher engagement in CPD. The study claims that teachers are mainly motivated to engage in CPD when it is compulsory for them to do so. Conversely, teachers also engage for personal reasons, such as career advancement or personal growth and achievement, and therefore prefer to identify and pursue CPD that meets their own personal or professional needs.



Models of CPD

The study also reports on school-related factors that impact teacher motivation to engage in CPD, and these include interpersonal relations such as peers talking about courses they have attended, the support provided by the school, a school culture of CPD and school policies on promotion. The OECD (2014) TALIS report found that participation rates of teachers in professional development was higher when the scheduled time was provided for such activities during regular working hours at the school, whereas low levels of participation occurred when such activities conflicted with work schedules and where there was an absence of incentives for participation. From a study of 38 schools over a 20-month period, Bubb and Earley (2009) show 10 factors that support the impact of CPD, leading to improvement in teachers’ practice, as set out in Figure 2. 1. 2. 3. 4.

The leadership and management of staff development needs to be effective People need a clear and shared understanding of staff development The school needs to develop a learning-centred culture Individual's’ development and should be linked to a needs analysis through performance management and career development as well as self-evaluation and school improvement 5. The goal, and the reasons for it, must be clear and ultimately should make a difference to pupils 6. The quickest, most effective and best value for money forms of staff development should be chosen based on what will suit individuals 7. Staff development that involves discussing, coaching, mentoring, observing and developing others is highly effective 8. Time needs to be made for staff development 9. Staff development should be monitored, and its impact needs to be evaluated 10. Learning and development should be shared, acknowledged and celebrated for improvement to be sustained

Figure 2: Factors for successful CPD (Source: Bubb and Earley 2009: 25)

The factors set out by Bubb and Earley (2009) suggest that merely providing CPD and requiring teachers to attend CPD is insufficient in terms of improving the outcomes for pupils. The factors for success should also be in place to maximise any potential benefit. Schleicher (2016) also highlights the importance of skilled school leaders who create a culture of collegiality and improvement in schools, as well as systems of career development that reward teachers who ‘innovate, share their learning, and help achieve school goals’ (44).



Models of CPD

He claims that professional development is the most effective when it connects teacher’s own development, his or her teaching responsibilities and school goals. In addition to personal and school-related factors, research from the US indicates that policy can support increased engagement in professional development. States with high levels of teacher participation in CPD were those with standards for the professional development of teachers and those that had a body for teacher licensing with minimum levels of professional development linked to licence renewal (Jaquith et al., 2010). 2.5

Conclusion

Research supports a positive correlation between quality professional development and student outcomes. Teachers are generally expected to engage in professional development throughout their careers, and it is now widely considered a professional responsibility. However, in most jurisdictions, it remains an optional activity with no consequences for teachers who do not engage. Despite national professional development policies, significant investment, and ongoing provision of CPD to teachers in many jurisdictions, the challenge of motivating teachers to change and improve practice following CPD, remains a pervasive issue. Not all CPD is effective, and many different models may be applied. Scholars converge on the key characteristics of effective CPD, and such findings are well documented. However, an ongoing concern is highlighted by, the TALIS study (OECD, 2014), which indicated low levels of teacher participation in professional development that has shown to be effective. The study indicates that teachers are participating most often in non-school embedded professional development activities, such as workshops and qualification courses and less often in school-embedded professional development that involves teacher collaboration on activities within their school. Attempting to balance cost and the disruption of student learning on the one hand with the potential benefits of providing quality professional development on the other hand is a challenge that requires further research. Where resources and availability of time are limited, how can teachers be provided with a range of CPD interventions that will meet both the needs of the system and the needs of individual teachers, and which of these should take priority? It is clear that the link between the provision of quality CPD and its potential to impact student learning is complex and multi-layered. A range of factors can mediate teachers’ participation in professional learning, and a further set of factors can influence its application in the classroom and eventual impact. However, by supporting the conditions and activities most associated with effective professional development, we can increase the likelihood that students will be impacted positively by such interventions.


INSPECTION IN NORTHERN IRELAND

Chapter 3 Overview of PDST ePortfolio Initiative


OVERVIEW OF PDST ePORTFOLIO INITIATIVE

3.1

Introduction

This chapter begins by providing a description of the PDST and its aims, guiding principles and supports available to teachers. The next section provides background information on the initial conceptualisation for the ePortfolio initiative. Finally, an overview of the various stages involved in the PDST ePortfolio initiative is given. 3.2

About the PDST

The PDST is a cross-sectoral multi-disciplinary support service of the Teacher Education Section (TES) in the DES. The PDST is Ireland’s largest single support service, offering professional learning opportunities to teachers and school leaders from primary and postprimary schools according to their self-identified needs and national policy priorities. The PDST aims to provide high-quality CPD support, to empower teachers and school leaders to provide the best possible education for all students. Because teacher educators are committed to the development of teachers and school leaders as reflective lifelong learners, the PDST team members provide a range of professional development experiences and models that support professional learning, reflection, collaboration and evidence-based practice in relation to the following: • • • • • • •

Primary curriculum and pedagogy Post-primary subjects and programmes Learning, teaching and assessment School improvement and school self-evaluation School leadership, culture and leading learning Pupil/student and teacher welfare Embedding the use of digital technologies in teaching, learning and assessment

The guiding principles of the service are the following: • • • •

An emphasis on quality professional learning provision for schools, teachers and school leaders The empowerment of teachers and school leaders in the context of developing schools as self-sustaining professional learning communities Flexibility in responding to the self-identified needs of individual schools and teachers, as well as to national system priorities A focus on school improvement with the aim of improved learning outcomes/learning experiences for students



3.3

PDST Supports Available to Teachers

CPD support is provided through a variety of models at the national, regional and local levels. These include national seminars and workshops, professional learning communities, bespoke support for whole staff at the school level, modelling of classroom methodologies, summer course provision, extensive website resources and online courses through PDST Technology in Education (PDST TIE). PDST’s Digital Technology team also works with schools in embedding the use of technology into teaching, learning and assessment. In addition, this team works closely with PDST TIE in relation to its service provision for Internet safety, online CPD, the development of digital content through Scoilnet.ie3 and the provision of technical advice. The PDST in recognising the importance of meaningful integration of digital technologies across all areas of teaching, learning and assessment, aims to ensure that these technologies are an authentic component of the PDST’s CPD provision across all areas of priority. In particular, they are focused on ensuring that the key focus of all of these interventions is on pedagogical outcomes rather than the technology itself. Regarding formative assessment, the team promotes in their work with teachers how effective practices can be facilitated and enhanced through the use of ePortfolio as a tool for teaching and learning. 3.4

Why Transition Year?

TY was chosen as the focus for this initiative for a number of reasons. In particular, the absence of a formal curriculum and examinations ensures a fertile experimentation ground for schools seeking to explore and adopt innovative approaches: in this case the use of ePortfolios. TY also offers pupils space to learn and develop in the absence of high-stakes examination pressure and according to DES (1994), ‘provides a bridge to help pupils make the transition from a highly-structured environment to one where they will take greater responsibility for their own learning and decision making’ (1). The TY also encourages a self-directed, individualised approach to learning that students can bring into senior cycle. Such ownership over one’s learning and taking responsibility for personal learning and performance is a central principle of using ePortfolios as an assessment tool.

3

Scoilnet is the Department of Education and Skills (DES) official portal for Irish education. Originally launched in 1998, the website is managed by PDST Technology in Education on behalf of the DES. Scoilnet collaborates with practising teachers to maintain and manage the content on the website. The Scoilnet website can be accessed at www.scoilnet.ie



Models of CPD

Assessment within TY is school-based and includes a range of formative approaches through ongoing assessment of performance, feedback, guidance and affirmation. It is argued that the use of ePortfolios can facilitate and enhance such approaches by providing electronic learning records that students can use to demonstrate, regulate and take responsibility for where their learning goes, thus aligning with the various functions of an ePortfolio system. According to the DES TY Guidelines for Schools: Pupil participation in the assessment procedure should be facilitated. This form of assessment which involves dialogue with tutors and self-rating on various performance indicators should lead to greater self-awareness and an increased ability to manage and take responsibility for learning. (DES, 2010: 4) In line with the above statement, it is suggested that ePortfolios should allow students to be actively involved in the assessment process. In practice, it is the students who make specific decisions about what they share with others subsequent to having engaged in the process of selection and reflection on a range of self-generated evidence for presentation and discussion. 3.5

Overview of the PDST ePortfolio Initiative

The PDST engaged in the EUfolio project from 2013 to 2015 and were charged with a number of associated deliverables. One of the more significant of these deliverables was the development of an online ePortfolio course for teachers. This was designed by the PDST and appears on the PDST online learning platform www.teachercpd.ie. In response to the bottom-up demand for professional development in the use of ePortfolios from teachers working in TY, the PDST issued an expression of interest to schools to ascertain interest and readiness to engage in a suite of professional development supports harnessing what was produced as part of the EU project. The initial conceptualisation for the various phases of the initiative was based on a PDST working paper titled Effective CPD: Background Paper. A Background Paper to accompany the PDST Proposal for the Integrated Language Curriculum CPD Framework (PDST, 2014).



3.6

Phases of the PDST ePortfolio Initiative

The ePortfolio initiative went through a number of phases from its inception in 2015. These will now be considered chronologically with a view toward highlighting the key pedagogical and theoretical constructs underpinning each. In Phase 1, the ePortfolio initiative required participating teachers to complete the PDST’s online synchronous course titled Ongoing Assessment using ePortfolios that was produced as part of the EUfolio project4 (Table 4). Module 1 – Understanding Assessment Module 2 – Understanding ePortfolios Module 3 – ePortfolio Functions Module 4 – ePortfolios to Support Assessment Module 5 – ePortfolio Implementation

Table 4: ePortfolios for assessment - online course (PDST Technology in Education) Along with completing the online course on ePortfolios, teachers also participated in a twoday seminar entitled Unlocking the doors to ePortfolios in December 2015. The purpose of this seminar (Table 5) was to build the foundations for teachers’ appreciation of the potential for ePortfolios usage in schools, as well as outlining the levels of digital literacy required of teachers and students.

4

Further details of the PDST’s online ePortfolio course Ongoing Assessment using ePortfolios are available at: http://www.pdsttechnologyineducation.ie/en/Training/Courses/ePortfolios-UsingDigital-Tools-for-Content-Creation-Collaboration-and-Assessment-Post-Primary-.html



Models of CPD Topic 1: Finding and selecting information for ePortfolios Topic 2: Creating and using digital resources. Introduction to digital storage (ePortfolios) Topic 3: ePortfolio functions Topic 4: Understanding assessment and ePortfolios Topic 5: ePortfolios as product and process Topic 6: ePortfolios as showcase. Considerations on the use of ePortfolios Topic 7: ePortfolios - Moving forward

Table 5: Outline of two-day seminar: Unlocking the doors to ePortfolios Phase 2 of the PDST ePortfolio initiative consisted of a one-day seminar in February 2016 entitled ePortfolios: Opening the doors to ePortfolios at Transition year level: Moving forward with the project. The purpose of the seminar (Table 6) was to further teachers’ understanding of the use of ePortfolios and to facilitate discussions around ePortfolios in a TY context. In addition, teachers were encouraged to consider the potential for ePortfolios at the Leaving Certificate level. Prior to the second part of the seminar, the PDST also developed generic templates and accompanying guides for the storage, workspace and showcase functions of an ePortfolio that could be created in both Google and Microsoft environments (PDST, 2016; 2017). These guides were used for the second part of the seminar, which consisted of a workshop on how to set up student ePortfolios and how the items contained in a student ePortfolio could be transferred from an institutional to a personal account at the formal end of a student’s post-primary education. Topic 1: Key messages, discussion and guidance on the updating of AUP Topic 2: Facilitated discussion around what an ePortfolio can be in a TY context. Storage workspace and showcase; progression of ePortfolio into Leaving Cert, LCA, LCVP Topic 3: Creating a school ePortfolio template Topic 4: Transferability and practical steps for TY students to begin using ePortfolios this academic year Topic 5: ePortfolios moving forward: Next steps toward embedding ePortfolios in your school

Table 6: Outline of two-day seminar: Unlocking the doors to ePortfolios



Models of CPD

In preparation for Phase 3 of the initiative, the PDST also provided training to a number of PDST advisors across all teams to build capacity internally and provide follow-up for the school-based support that was offered in March 2016. A small number of early adopter schools were selected to provide on-the-job training for these advisors and to inform a model of sustained support in 2016/17. Phase 3 of the PDST ePortfolio initiative consisted of proportionate support that was made available to all participating schools and began, as previously mentioned, in March 2016. An outline of the framework of individual supports provided to schools is detailed in Table 7. Item 1: Pilot implementation and planning in 2015/16; 2016/17 and 2017/18 Item 2: TY planning for ePortfolio (storage, workspace, showcase, and ongoing and final assessment) Item 3: Technical issues Item 4: School policies and plans (Acceptable Usage Policy, TY Plan, Assessment Policy) Item 5: CPD requirements for staff (Principles of ePortfolios in a TY context, ongoing assessment, integrating ICT into teaching and learning) Item 6: Experiences of students, teachers and school leadership to date

Table 7: Outline structure of PDST advisor school visits Phase 4 of the PDST ePortfolio initiative consisted of a shared learning day in May 2017. Within this teachers, students and school principals from participating schools provided attendees with a presentation on their experience of using ePortfolios and of the PDST ePortfolio initiative more generally. 5 Phase 5 consisted of an evaluation of the impact of the PDST ePortfolio initiative and an exploration of the potential for this model of sustained CPD support. This phase ran concurrently with all preceding phases of the PDST ePortfolio initiative and will form the basis of the remainder of the report. First, however, a description of the research methodology employed will be presented.

5

Further details of the shared learning day together with the views of student and teacher experiences of the initiative are available at the following link: Pdst.ie/eportfolio




POLYCENTIC INSPECTION EXPLORED

Chapter 4 Research Methodology


RESEARCH METHODOLOGY

4.1

Introduction

The purpose of the current study was to evaluate the impact of the PDST ePortfolio initiative. To gain as complete a picture as possible of the impact of the ePortfolio initiative, the researchers collected data using a combination of qualitative and quantitative techniques. This chapter describes the methodology used in the current study. The first section provides a description of the research design, including the methodologies used and the theoretical connections that linked each phase of the study. Finally, the chapter concludes with the acknowledged limitations of the evaluation. 4.2

Research Design used in This Study

This research used a multi-phase convergence research design consisting of four distinct stages (Figure 3). Each stage of the research consisted of concurrent levels that were sequentially aligned with other stages of the evaluation to build upon and provide an overall interpretation of the study.

Figure 3: Multi-phase research design used in the study. Adapted from Youngs, H., and Piggot- Irvine, E.,2012. (Source: Youngs, and Piggot-Irvine, 2012: 190, fig.2).



Models of CPD

Stage 1 of the study involved a single exploratory level (Figure 4) that consisted of a systematic literature review of the literature on ePortfolios (Chapter 1) and CPD, as well as a review of the various evaluation frameworks that are used to ascertain the impact of CPD provided. This stage of the evaluation resulted in the development of a questionnaire to ascertain principals’ and teachers’ perceptions of the PDST ePortfolio initiative and the choice of a professional development evaluation framework to be used in the evaluation (Figure 4). From a data collection and analysis perspective, it also formed the basis for the classification of interview data in the subsequent stages of the study.

Level 1 Process

Systematic literature review of ePortfolios, CPD and CPD evaluation frameworks

Level 1 Product Conceptual Framework (Guskey’s Evaluation Framework) Development of questionnaire for analysis of perceptions toward CPD provided

Figure 4: Exploratory phase 1



Models of CPD

Stage 2 of the study consisted of three exploratory levels that built on the work engaged in at level one in the previous stage (Figure 5). Models of CPD Levels 2 and 3 Process Survey to participants at the beginning and end of the first three stages of the PDST ePortfolio initiative

Levels 2 and 3 Product Distribution and analysis of questionnaire responses

Level 4 Process Collection of ePortfolio resources produced by the PDST from the beginning to the end of the PDST ePortfolio initiative

Level 4 Product Analysis of PDST ePortfolio resources

Development of interview schedule for Phase 3 of the evaluation

Development of interview schedule for Stage 3 of the evaluation

Figure 5: Exploratory phase 2 The purpose of this stage of the research was to gain an understanding of the following: (1) participants’ overall reactions to the training provided; (2) participants’ learning as a result of participation in the initiative and (3) the change orientation regarding organisation and support in participant schools. Levels 2 and 3 consisted of the distribution of surveys to participants at the beginning and end of the first four phases of the PDST ePortfolio initiative (Chapter 3). Level 4 also consisted of an analysis of ePortfolio resources provided to participants. Combining the various levels in this stage of the study resulted in the development of an interview schedule for Stage 3 of the study. Stage 3 of the study consisted of two concurrent exploratory levels (Figure 6). This phase of the research was carried out to ascertain the effects of the PDST ePortfolio initiative on participants’ use of new knowledge and skills, as well as student learning outcomes. The first level of this stage of the study consisted of school visits and semi-structured interviews with a sample of teachers (n=12) involved in the PDST ePortfolio initiative. The second level analysed a sample of student ePortfolios and teacher assessments and a sample of ePortfolio-related school policies.



Models of CPD Level 5 Process Interviews with a sample of teachers involved in the PDST ePortfolio initiative

Levels 5 Product Transcription of interviews (n=12) Classification and analysis of interview data

Level 6 Process Collection of student ePortfolios, feedback provided by teachers and ePortfolio policies in teachers’ schools

Level 6 Product Analysis of student ePortfolios and teacher assessments and ePortfolio-related school policies

Figure 6: Exploratory phase 3

Finally, stage 4 of the study consisted of merging the previous phases to form an overall interpretation of the impact of the PDST ePortfolio initiative (Figure 7).





4.3

Exploring Guskey’s Framework for Evaluating Professional Development

Bodkin (2013) argues in a review of CPD in the field of art education that ‘research evidence about the evaluation of CPD practice is limited at present’. Citing Guskey (1999:2), she suggests that ‘historically professional developers have not paid much attention to evaluation’ (Bodkin, 2013: 37). When they do take place in a CPD context, Guskey (2000:810) argues that they are methodologically limited, programme snapshots that seek to provide a delimited analysis of the short-term impact of the programme. Edmonds and Lee (2002) echo this when reporting on CPD evaluation experiences in England and Wales. They suggest that for the most part, evaluations are only interested in ‘.......delivery, content, relevance, applicability, whether the course met the stated objectives, whether it was costeffective and should be attended again’ (28). Although undoubtedly useful, these types of evaluations tend not to discuss issues of wider impact and are mostly silent on how interventions change practice for both learners and teachers in a school setting. Citing Mujis et al. (2004: 292), Bodkin (2013: 38) makes the point that much CPD evaluation occurs summatively rather than formatively, where the evaluation would be used to enhance the experience. The evaluation strategy underpinning this report took this reality into account. Seeking to ensure that the model of evaluation was contextually sensitive and methodologically robust, the current study uses Guskey’s (2000) approach (Figure 8).

Figure 8: Five Levels of professional development evaluation. Adapted from Guskey, T., 2000.



Models of CPD

This model of evaluation is designed specifically for CPD situations and argues that innovations such as the ePortfolio initiative need to be analysed under the following categories: 1. 2. 3. 4. 5. 4.4

Participants’ reactions Participants’ learning Organization support and change Participants’ use of new knowledge Skills and student learning outcomes Questionnaire Development, Distribution and Analysis

Questionnaires were developed following a review of the literature on CPD and ePortfolios (Chapters 1 and 2) and distributed to participating teachers at the beginning and end of the first four phases of the PDST ePortfolio initiative (Chapter 3). The questionnaires consisted of nominal, ordinal and open-ended response questions. All ordinal response questions received a score for bi-polar response alternatives, which did not apply to nominal data asking for yes/no responses. 4.4.1

Data accuracy

Although there are many stated benefits to the use of questionnaires as a data collection instrument, ‘in many respects the potential disadvantages of questionnaires go hand in glove with the potential advantages. You can’t have one without the other’ (Denscombe, 2010: 170). Reservations surrounding the use of this type of research instrument revolve around the below issues. Pre-coded questions can be frustrating for respondents and, thus, deter them from answering. Although the ‘tick box’ routine is less demanding on participants, closed-ended questions might encourage participants to respond. Denscombe (2010) goes on to state that predetermined answers could also cause participant frustration. Consequently, ‘this same routine might be experienced as negative and put people off co-operating with the research’ (170).



Models of CPD CPD

Pre-coded questions can bias the findings toward the researcher’s, rather than the respondent’s, way of seeing things. Because of the structured nature of questionnaires, answers may stifle participant responses and reflect the viewpoint of the researcher as opposed to the participant. Indeed, criticisms relating to scaled measurements of attitude have been well documented within the field of social science, where according to Foddy (1994): The most common criticism of the survey researchers is that their pre-set response categories determine the way the respondents can answer a question, making it impossible to evaluate the validity of the answers. Put another way; it is argued that the provision of sets of response options may cause respondents to give answers which they would not think of if they had to supply answers themselves. (16) Questionnaires offer little opportunity for the researcher to check the truthfulness of answers given by the respondent’s way of seeing things. Because respondent answers are quite frequently anonymous, the researcher is unable to determine the authenticity of the responses. Indeed, and particularly in relation to this study, ‘this is all the more true if the questionnaires are anonymous’ (Denscombe, 2010: 170) While recognising the positive and negative issues relating to the use of questionnaires as a data collection strategy for this evaluation, the concept of validity formed the basis for the development of the questionnaires, but also the decision to carry out a series of semistructured interviews with a sample of teachers at the end of Phase 4 of the PDST ePortfolio initiative. This approach was also in line with Guskey (2002), who states: Enough time must pass to allow participants to adapt the new ideas and practices to their settings. Because implementation is often a gradual and uneven process, you may also need to measure progress at several time intervals. You may gather this information through questionnaires or structured interviews with participants and their supervisors, oral or written personal reflections, or examination of participants' journals or portfolios. (6)



4.4.2

Pilot study

A pilot study was carried out prior to the distribution of the questionnaire. The importance of piloting questionnaires is further emphasised by Oppenheim (1998), who states: Questionnaires do not emerge fully-fledged; they have to be created or adapted, fashioned and developed to maturity after many abortive test flights. In fact, every aspect of a survey has to be tried out beforehand to make sure that it works as intended. (47) Four teachers were asked to participate in the pilot study, and responses and comments made in relation to the content, layout and the administration of the questionnaire were incorporated into the final questionnaire revisions. The participants involved in the pilot study logged onto a website address to make comments about how the questionnaire could be improved regarding language used, format, content and question order. Apart from the correction of typographical errors and terms of reference, all the teachers who participated in the pilot study felt that the pilot questionnaire was restricted by the fact that there were no open-ended responses available to clarify answers given to ordinal and nominal questions. Taking the views of participants in the pilot study into consideration, a final iteration of the questionnaires was developed to allow participants to comment with open ended choices if desired. 4.4.3

Descriptive and time interval comparison statistics

All questions, although interrelated, were classified according to Guskey’s (2000) framework for professional development evaluation, and the data were then inputted using SPSS Version 23. From this, a series of statistical tests were used to analyse the data by using a combination of both parametric and non-parametric tests based on the time interval of questionnaires. Descriptive statistics were used to examine the mean and standard deviation of the responses given. Non-parametric tests were also used to see if there was a statistical difference by time interval for each categorical response statement.



4.5

Interview Coding, Participant Selection and Analysis

4.5.1

Introduction and background

Fidel (2008) is of the view that ‘the use of inherently different methods fosters flexibility in the research process. This may create new insights and possibilities that one method alone could not produce’ (267). Furthermore, and in parallel with the objective of Stage 3 of the study, Seidman states: The purpose of in-depth interviewing is not to test hypotheses, and not to “evaluate” as the term is normally used (See Patton, 1989, for an exception). At the root of in-depth interviewing is an interest in understanding the lived experience of other people and the meaning they make of that experience. (9) Seidman goes on to state that ‘at the heart of interviewing research is an interest in other individuals’ stories because they are of worth’ (Seidman, 2012: 9). This perspective resonates with Mears, who states that ‘you ask participants about related matters and to tell their experiences, share their feelings or thoughts, and reflect on decisions and events. From their narratives, you will be able to analyse the information and answer your research question’ (Mears, 2012: 172). As part of the evaluation, the researchers wanted to gain a greater understanding of participant’s use of new knowledge and their opinions about and use of student learning outcomes (Guskey, 2000). From this perspective, it would not have been possible to ascertain teachers’ use of new knowledge and student learning outcomes with a single method. To facilitate dissonance-reduction and further elucidate the impact of the ePortfolio initiative, a series of semi-structured interviews were carried out with a sample of teachers. Prior to the interviews taking place, a letter was sent to teachers, stating the purpose of the study and asking teachers if they would be willing to participate. Ethical issues, such as the anonymity of interviewees, and the purpose and intended outcomes of the research, were also explained.



4.5.2

Selection of participants for interviews

The selection of participants for this phase of the study was based on a stratified sampling strategy. The purpose of a stratified purposeful sample is to capture major variations rather than to identify a common core, although the latter may also emerge in the analysis (Patton, 2002a: 240). In terms of sampling size, Patton suggests that a frequently asked question in qualitative studies relates to that of sample size: My universal, certain, and confident reply to these questions is this: “It depends.” There are no rules for sample size in qualitative inquiry. Sample size depends on what you want to know, the purpose of the inquiry, what’s at stake, what will be useful, what will have credibility, and what can be done with available time and resources. (Patton, 2002: 244) Taking into consideration the various issues mentioned, for this stage of the research, 12 teachers were interviewed between October 2016 and January 2017. This represents 44% of the total number of teachers who participated in the ePortfolio initiative. 4.5.3

Interview coding and analysis

According to Miles and Huberman (1994), coding is analysis: ‘To review a set of field notes, transcribed or synthesised, and to dissect them meaningfully, while keeping the relations between the parts intact, is the stuff of analysis’ (56). The coding and analysis strategy for the current study used a combination of Creswell’s (2008) data analysis process (Figure 9) and Miles and Huberman’s (1994) ‘Components of Data Analysis: Interactive Model’ (Figure 10), and it consisted of six stages.



Models of CPD Interpreting the meaning of themes/descriptions Interrelating themes/description (e.g., grounded theory, case study)

Themes

Description

Coding the Data (hand or computer)

Validating the Accuracy of the information

Reading through All data

Organising and preparing data for analysis

Raw data (transcripts, field notes, images, etc.)

(Source: Creswell, 2008, ch.9, para.23, fig.1) Figure 9: Data analysis in qualitative research

Data collection

Data display

Data reduction Conclusions: drawing/verifying

(Source: Miles and Huberman, 1994: p.12, fig.1.4) Figure 10: Components of data analysis: Interactive model



Models of CPD CPD

Stage 1: Transcription of Interviews Interview data were initially coded according to the participant and where the unit of information was located. For example, when referring to the code T1/23, the first letter identified the participant, and the number after the forward slash symbol was where the unit of information was located. Stage 2: Data immersion Before each statement was classified, interview data were read, reread and examined through a process of data immersion/crystallisation that according to Borkan (1999), ‘provides a means to move from the research question, the generated text and/or field experience, and the raw field data to the interpretations reported in the write-up’ (1999: 180). Stage 3: Coding and analysis According to Creswell (2008: ch.9, para.28), ‘One further issue about coding is whether the researcher should (a) develop codes only on the basis of the emerging information collected from participants, (b) use predetermined codes and then fit the data to them, or (c) use some combination of predetermined and emerging codes’. The interview analysis for this phase of the study used predetermined codes in the first stage of the analysis by assigning and reassigning a conceptual label to each statement’s location in Guskey’s five levels of professional development evaluation (2000). Following this, sub-themes began to emerge. For example, within the level of analysis organisational support and change, a unit of the text was initially classified as ‘support’, followed by the emergence of a number of themes such as ‘infrastructural support’ and ‘internal CPD’. 4.6

Ethical Considerations

According to Moyles (2007:245), ‘particularly because of the potential power relations, the researcher needs to be clear that the subjects fully comprehend the nature and outcomes of the research and that anonymity and confidentiality are assured’. A cover letter explaining the purpose of the research was developed to make clear for the participants, the aims, procedures and purpose of the study. In line with DCU’s ethical guidelines on carrying out research, this study was considered low level, and an assurance of confidentiality was given to all participants. Furthermore, all personal identifiers, place names, and school types were removed from the interview transcripts, and confidentiality and anonymity were assured.



4.7

Limitations of the Study

A significant limitation of the current study is that the researchers did not formally interview other key stakeholders involved in the process, such as parents, students and other members of the school community. Furthermore, although analysis of student ePortfolios and feedback provided by teachers formed a significant part of the evaluation, save for students and teachers presenting their ePortfolios at the PDST Shared Learning days, the researchers did not observe teachers and students using ePortfolios in a classroom setting. 4.8

Conclusion

This chapter described the research design and methodology used in the current study. The questionnaire development, analysis and design were discussed, including, for example, descriptive statistics, and aggregated questionnaire responses. Interviews that formed the third stage of the research were also discussed, including the selection of participants and how the interviews were coded and analysed. Finally, ethical considerations and the limitations of the research were described. The methodological approach described in this chapter formed the basis for the research findings and the discussion that is outlined in the next two chapters. Models of CPD



26| Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios


Chapter 5 Presentation and Analysis


PRESENTATION AND ANALYSIS

5.1

Introduction

This chapter provides an analysis of the qualitative and quantitative data obtained during the evaluation. Data were initially collected through the use of self-completed questionnaires. To gain a better understanding of the aggregated questionnaire responses, semi-structured interviews were also conducted with a sample of teachers involved in the ePortfolio initiative. The first section of this chapter, section 5.2, provides an overview of the participants’ profiles. Sections 5.3–5.6 provide an analysis of the impact of the ePortfolio initiative using Guskey’s (2000) five levels of professional development evaluation (Level 1: Participants’ reactions; Level 2: Participants' learning; Level 3: Organization support and Change; Level 4: Participants' use of new knowledge and skills and Level 5: Student learning outcomes). Finally, the chapter concludes with a description of peripheral barriers and supports that impact the integration of ePortfolios in participant schools. 5.2

Participant Profiles

This section describes participant profiles. The first sub-section describes participants prior experience with the use of ePortfolios. The next sub-section describes participants’ motivation to participate in the ePortfolio initiative and their prior experience with CPD. The last subsection provides a representative description of ICT facilities in participant schools. 5.2.1

Prior experience with the use of ePortfolios

Given the open invitation by the PDST asking schools if they would be willing to participate in the ePortfolio initiative (Chapter 3), it is unsurprising that participants’ prior experience with ePortfolios varied. In fact, 65% of the teachers participating in the initiative had either no experience or less than a year’s experience with ePortfolios (Table 8). Answer Choices

Responses

None

26.09%

3

Less than a year

39.13%

14

1-3 years

21.74%

5

3+ years

13.04%

3

Table 8: Participants’ prior experience with ePortfolios



Models of CPD This difference is illustrated by the wide-ranging responses from participants when asked about their engagement with ePortfolios, as follows: Absolutely zero. We did portfolios here in TY with kids, but it was paper-based. So, I was interested in how we migrate from paper-based to electronic-based portfolios. We were doing that in a way, but it just wasn’t pulled together. It would have been very thin, that’s a place to put stuff. That’s the only information I knew about, to be honest. Very little, very limited apart from storing files in my Drive. We are an iPad School, so all teachers would have been trained on how to use the storage functions of an ePortfolio. We had started ePortfolio 3 or 4 years ago, and every student in TY must produce an ePortfolio of evidence, and they then have to present their ePortfolio as part of the Assessment in TY.

5.2.2

Prior experience with CPD

All participants involved in the initiative engaged in CPD either through formal post-graduate studies or with various sectoral support services such as the PDST and Junior Cycle for Teachers (JCT). I’ve done a lot of CPD courses with t46 over the years. The most recent one was Tóraíocht. 7

6

t4 is a Department of Education support service whose purpose is to prepare and support teachers to implement the revised syllabuses, Architectural Technology, Design and Communication Graphics and Engineering Technology and Technology at Leaving Certificate level. 7

Tóraíocht (Post-Graduate Diploma in Educational Leadership) was offered in joint partnership by PDST and Maynooth University to aspiring leaders throughout Ireland. The course encouraged colleagues to engage with issues of school leadership together through learning and dialogue, to achieve an overview of the current education system and to recognise the unique contribution that can be made to the lives of the students in their schools.



Models of CPD On or off I’ve been doing 1 or 2 per year whether its subject based or whether its leadership based or TY based. I’ve also done some courses with JCT so, I’ve done the whole gamut. All very good and it has changed so much over the years. It’s a lot more targeted and focussed. 5.2.3

Motivation to participate in ePortfolio initiative

Almost all participants had interlinked motives to participate in the ePortfolio initiative that centred on the desire to improve the quality of teaching, learning and assessment, not only in their classrooms, but also in their schools. The following comments illustrate this point: So, we had basically our TY, and they were getting tests three times a year and reports going home, and I felt that we were going nowhere with it and student’s that were absent a lot were getting as good a mark as a student that was in all the time. So, we could have a student that missed a load of weeks and was getting 90%. So, we wondered was that a reflection on what was going on, so I was trying to find a different way. First, when we have our portfolios at the end of the year, they’re hard to manage. So, the kids come in with these big folders. So first of all, portability was an issue, and that was ePortfolios. I was at a seminar by the JMB, and they were talking about Google Classroom and was at another, the EUfolio launch in Dublin Castle. They described all of the other stuff you can do with it. So, from an educational learning point of view I thought, that makes even more sense. We had implemented it already, and we wanted to see if we could improve on it. Participants’ motivations to participate in the ePortfolio initiative also centred on the desire to become what one respondent described as ‘ePortfolio ready’, where it was envisaged that in due course, ePortfolios would become what another participant described as ‘part and parcel of what we will be using in the Junior and Leaving Cert’. Others expanded on this point: I had a personal interest. I like to be involved in initiatives that are going on nationally because you get on board with things a little bit earlier than other schools and you are keeping up to date with changes that are occurring. You know I think that ePortfolios will become central to the Junior Cycle in the future, and I think that we should start preparing for this now, so that’s the reason why I got involved in it.



5.2.4

ICT facilities

The quality and range of ICT facilities varied considerably in participant schools. For most, they consisted of a projector and computer in every class, as well as one or two computer rooms that frequently contained approximately 24 computers and an overhead projector. All the participant schools had wired Internet access in classrooms and other buildings. There is a computer in each class. Some teachers have iPads and Surface. We have two computer rooms one of which was refurbished with a set of PCs, and we have Wi-Fi throughout the school and its very, very strong. This is the third year that we’ve had Wi-Fi. We were kind of thinking of having one computer per student, but we’ve eased off on that and decided to ensure that all Wi-Fi issues are sorted out first. A minority of participating schools have also introduced mobile devices and have developed varied and innovative solutions for Internet connectivity: With one or two colleagues, we’ve tried to improve the whole infrastructure in the school. We’re now up to 140 PCs and 30 chrome books. A lot of teachers are utilising IT in their classrooms where they wouldn’t have before. Because we don’t have Wi-Fi in the school we have two routers on the top of the trolley, and you plug them into the wired point in the room, and that facilitates the issue of Wi-Fi. We have two computer rooms with 30 computers in each. The equipment in one room is 2 years old while the equipment in the other is almost 4 years old and requires upgrading. We also introduced ipads four years ago and all students up to and including TY year have ipads. All 3rd years have i-Pads, which were introduced three years ago now. There’s also Apple TVs in all classrooms. There are two computer rooms in the school, but these are somewhat outdated. 5.3

Level 1: Participants’ Reactions

This section provides an analysis of the participants’ reactions to the CPD provided by the PDST during the initiative. The first subsection describes participants’ perspectives on the knowledge and skill set of PDST advisors. The next subsections describe participants’ reactions to the quality of online and face to face CPD provided. The final subsection describes participants’ overall satisfaction with the experience of follow-up supports.



5.3.1

Knowledge and skill set of PDST advisors

Regarding the knowledge and facilitation skills of advisors, almost all the participants were of the view that advisors were approachable, supportive and challenging, as illustrated in the following comments: They were listening to the people in the room… they were working hard and putting in the hours. Well it was difficult at the start as some teachers had very few ICT skills, so there were many questions at the start and if they didn’t have the answer straight away, they would the next time they met you. Yes, the advisors were top of their game. The fact that they were teachers themselves helped a lot. Highly skilled, just in touch. Evidence of collegial teamwork between advisors was also described: The presenters were excellent, knowledgeable and enthusiastic with fresh ideas … a mix of different disciplines. There were a few of them [Advisors] presenting on the shared learning day. They were really professional and seemed to work off each other. 5.3.2

Quality of online CPD

Almost all the participants were of the view that the online CPD course was either very useful or extremely useful (Table 9).

Answer Choice

%

Count

Not at all useful

0.00%

0

Slightly useful

0.00%

0

Somewhat useful

9.09%

2

Very useful

54.55%

12

Extremely useful

36.36%

8

Table 9: Participants attitudes toward PDST online ePortfolio course



Models of CPD The usefulness of the online CPD related not so much to the technical aspects of an ePortfolio system, but rather that of learning about its purpose and functions: The five-hour course on ePortfolios online was really good, and the links you were given to the step by step guide to set things up was brilliant. It gave you the background information to learn about the learning, to learn about the teaching, to learn about ePortfolio as a collaborative process for learning, which is great. I thought that videos of teachers talking about how they used ePortfolios in their school was excellent. Very practical. Participants also commented favourably on the way that assessment was embedded throughout the course. Even though it’s more about assessment now, than it is about anything technical to do with e-portfolios, it opens your mind, and it really firms up your … you are 100% clear why e-portfolios are a good idea, because you can see the formative and summative and you can see the ‘for learning’ and ‘of learning’ and really, I thought that was a very good course. I came back and told everyone here they should do it. I already knew about what you can do with an ePortfolio, but the assessment part and the NCCA kit that they used brought everything together. The requirement for all participants to complete the online module to have a uniformity of understanding of ePortfolio functions was also seen as providing a positive baseline for engagement with the rest of the PDST seminar series: The online course I thought was excellent. Now, I thought it was a very good idea to make everyone do that, you know, before you got invited back to, say, the second day, you had to do the course. I think a lot of CPD introducing something new can be offered online and I liked the way that this was done, and time wasn’t spent describing the basics, especially for the likes of myself who already knew about ePortfolios. It was great as everyone knew what an ePortfolio is when we went back the next time. For participants who had previously used ePortfolios, the value of this element of CPD was less useful. Due to the introductory nature of the ePortfolio course, some participants were of the view that advanced modules on the technical aspects of ePortfolios could also be offered:



Models of CPD What I really would have liked were more advanced modules on how to deal with the technical side of an ePortfolio, how to set up and maintain an ePortfolio, how to set up different sharing levels, stuff like that. The online platform was fine, but at times, I got a bit bored and kept falling out. There appears to be a basic competence level required that I am way beyond. 5.3.3

Quality of seminar CPD

On average, more than 70% of participants were of the view that the PDST seminars were either very useful or extremely useful (Table 10). Day 1 Seminar 1 Answer Choice Not at all useful Slightly useful Somewhat useful Very useful Extremely useful

Day 2 Seminar 1

Day 3 Seminar 2

Day 4 Seminar 3

%

Count

%

Count

%

Count

%

Count

0.00%

0

0.00%

0

0.00%

0

5.00

1

0.00%

0

0.00%

0

8.70%

2

5.00

1

21.74%

5

13.04%

2

17.39%

4

15.00

3

69.57%

16

65.22%

15

47.83%

11

50.00

10

8.70%

2

21.74%

5

26.09%

6

25.00

5

TABLE 10: Participants attitudes toward the usefulness of PDST seminars on ePortfolios Analysis of the interview data suggests that in the main, the analysis of the quality of CPD provided related to the pace of training, the sharing of ePortfolio experiences among participants and the creation of informal professional learning communities: I really enjoyed the days, very good, well structured, nicely paced, very useful. I came home, and I was able to set up all the One notes for all the classes. The team were great. They knew how to take us from baby steps to full-on use of the system. For those of us who might be a little afraid of these things, it was very supportive and affirming. I found the CPD to be fantastic and very useful, and it was good meeting people from other schools and seeing what they were doing and what problems they were having. I got a lot of pointers from other schools that were more advanced.



Models of CPD The initial training was really good. We were lucky that there were a few of us who were very comfortable with technology and we sat together, sharing ideas and discussing how to solve problems. Although participants had a positive disposition toward the training provided, some participants were also of the view that self-directed learning was also required to obtain a level of competence necessary to implement ePortfolios in their schools: The training was very good, but unless you were sitting down with notes and material provided, nearly at home on your laptop that evening or the next day, it’s hard to keep up. The course was excellent, but for me, I had to put a lot of work into it after each day and go through all the steps again to become comfortable with it. It’s not something you can learn overnight. Considering the above comments and the different ePortfolio competencies of participants, it is evident that the learning trajectory during the ePortfolio initiative was steeper for some participants than others: I think in terms of the learning and assessment side I think that the PDST nailed it, but I think that maybe more emphasis on the technical side would be helpful if they rolled it out again. Very well spent because the training was accurate, it was good, it was useful. The only thing I would say is that I am not a techie. I’m not good at the technical stuff. A lot of the basic material I knew already so I would have liked more advanced training. 5.3.4

Follow-up support

Follow-up proportionate support provided by PDST advisors was viewed by participants as being essential to enhancing the various levels of ePortfolio readiness in participant schools. This point is illustrated in the following participant comments: They were the most beneficial by far because they really focused on the needs of the teachers and the school. I mean, it was great to see that there were so many schools mixed up together and some who didn’t need any more support after the seminars I think, but for the likes of our school who were starting from scratch, we wouldn’t have gotten to as far as we did without the advisor visits.



Models of CPD You need follow up support, and you need teachers to get together again to say what works, what doesn’t work, what are the students able to do? When I went online and saw what the other teachers had done I remember saying I have nothing like that so there was a big mix of teachers gathered together. So, when I contacted the advisor, and she came out it wasn’t that overpowering, and we took it from there, and now we are up and running. I mean if it was just based on the training days, I think I would have been in a bit of trouble. The follow-up support was excellent. The benefits to follow-up support did not only relate to an acknowledgement of the need for differentiated CPD. Participants also commented on the fact that advisor visits were beneficial to driving the collective ePortfolio agenda in participant schools: I really enjoyed the training – particularly the one in school. It brought us together and gave me a platform to call on people later on when I needed things done. I really liked the way when they came to the school that it gave us a chance to discuss issues around learning and teaching and our students in a way that we don’t often get. It led to a great buzz and people were ready to try new things. We had one visit, and the purpose was to bring other teachers with me to sing from the same hymn sheet, so we can spread the workload as much as possible. There’s a lot of work in it. 5.3.5

Quality of resources

Almost all participants agreed that the quality of resources provided by the PDST was beneficial for both the learning and technical aspects of the initiative as illustrated in the following comments: Everything was very well laid out, very clear, very easy to understand. The resources were excellent and allowed me to go through the whole thing again when I got back to school. The step-by-step guides [PDST, 2016a, 2016b] on how to set up an ePortfolio was very good.



Models of CPD Participants also commented on the ePortfolio templates provided: We all started off with the PDST e templates and then would have put in our logo and then students could change the background colour, etc. So, they changed the page template as opposed to the site template. This took a bit of getting used to, but I found it very useful rather than building an ePortfolio template from Scratch. When I started using the templates, I found them difficult to use. When I got the hang of it, I got rid of the material that wasn’t relevant to our school and just kept the core stuff and what I did then was to allow students to give suggestions as to what they could add in, so it makes it more relevant for them. 5.4

Level 2: Participants' Learning

This section describes participants’ learning during the duration of the ePortfolio initiative. The first subsection describes the digital literacies developed by participants’. The next subsection provides an analysis of the change orientation of participants to lead the teaching, learning and technical aspects of an ePortfolio system in their schools. 5.4.1

ePortfolio literacies

The development of digital literacies was viewed as being an important by-product of the initiative, and aggregated questionnaire responses showed significant development in this area. There were, as is perhaps to be expected, a variety of competence levels demonstrated by participants. Almost all could create digital files contained within an ePortfolio digital work-space (82%), and 44% of the participants were able to store digital files at the beginning of the initiative. This was not, however, the case for other ePortfolio competencies required to navigate an ePortfolio effectively. At the beginning of the initiative, only 22% of participants could set up various sharing levels contained in an ePortfolio. However, this value increased to almost 95% by the end of the initiative (Table 11).



Models of CPD

Question Are you and teachers involved in the PDST Transition Year Project in your school able to create digital files contained within an ePortfolio digital work-space? Are you and the teachers involved in the PDST Transition Year Project in your school able to store digital files contained within an ePortfolio digital work-space? Are you and the teachers involved in the PDST Transition Year Project able to set up various sharing levels (e.g. view only, share with teacher/peer only) for digital files contained within an ePortfolio digital work-space?

Beginning of Initiative % Count

End of Initiative % Count

Answer Choice Yes

82.61%

19

94.74%

18

No

17.39%

4

5.26%

1

Yes

43.48%

10

94.74%

18

No

56.52%

13

5.26%

1

Yes

21.74%

5

94.74%

18

No

78.26%

18

5.26%

1

Table 11: Digital literacies obtained by participants (creating, storing and sharing digital files) Furthermore, only 13% of those participants who were involved in the initiative were fully aware of legal issues relating to copyright and the acceptable use of, for example, images, audio, video and text for the purpose of creating, storing and sharing digital content. However, this value increased to almost 85% by the end of the initiative (Table 12). According to one participant: A lot of this learning is something I can apply outside of TY as well. On the first day, they spent a lot of time talking about various issues such as usage policy and copyright which I was surprised to learn. I didn’t understand the seriousness of copyright issues and how to turn on all the filters. .



Models of CPD

Question Are you and the teachers involved in the PDST Transition Year Project aware of issues relating to copyright and the acceptable use of, for example, images, audio, video and text for the purpose creating, storing and sharing digital content?



Answer Choice Yes

13.04%

3

84.21%

16

No

86.96%

20

15.79%

3

Table 12: Participants understanding of copyright and the acceptable use of digital files Finally, only a small number of participants could use the formative assessment function of an ePortfolio to respond to peer and student feedback. This value increased to almost 90% by the end of the initiative (Table 13).

Question Are you and teachers involved in the PDST Transition Year Project able to access and respond to peer and student feedback via a student/teacher digital workspace?



Answer Choice Yes

21.74%

5

89.47%

17

No

78.26%

18

10.53%

2

Table 13: Digital literacies obtained by participants (Accessing and responding to peer and student feedback) The benefits to participants of learning how to use the formative assessment features of an ePortfolio are illustrated in the following points: It is so much more convenient now; it’s handy and easy to give feedback on it, even on the app, in OneNote. But you have to make sure kids are taking feedback on board. You have to check in with them; it’s not just a case of leaving a textbox on something they have written and asking them to change it. You still have to monitor if they are doing that. It has been very good for assessment, so they have an interview at Christmas and its to give them a feel for it. So, the interview is based totally on their ePortfolio. So, it works when the other exams are going on in the school.



5.4.2

Digital leadership – assessment confidence

A core objective of the PDST initiative was to build the capacity and confidence of teachers to lead the introduction of ePortfolios in their schools. The following tables provide an analysis of the changing confidence levels of participants over the duration of the initiative. One noteworthy area in this context is the changing levels of confidence related to supporting assessment of learning with ePortfolios. At the beginning of the initiative, 39% of participants were either confident, very confident or completely confident in this aspect of their work using ePortfolios. However, by the end of the initiative, this value increased to 78% (Table 14). Beginning of Initiative Answer Choice Not confident at all Not very confident Somewhat confident Confident Very confident Completely confident

% 4.35% 39.13% 17.39% 26.09% 13.04% 0.00%

Count 1 9 4 6 3 0

End of Initiative % Count 5.26% 1 0.00% 0 15.79% 3 42.11% 9 36.84% 7 0.00% 0

TABLE 14: Confidence of participants to support assessment of learning with ePortfolios in their school Furthermore, at the beginning of the initiative, no more than 30% of participants were either confident, very confident or completely confident in supporting assessment for learning with ePortfolios in their schools. By the end of the initiative, this value had increased to approximately 79% (Table 15). Beginning of Initiative Answer Choice Not confident at all Not very confident Somewhat confident Confident Very confident Completely confident

% 8.7% 43.48% 17.39% 13.04% 17.39% 0%

Count 2 1 4 3 4 0

% 5.26% 0.00% 15.79% 42.11% 36.84% 0%

End of Initiative Count 1 0 3 8 7 0

TABLE 15: Confidence of participants to support assessment for learning with ePortfolios in their school



5.4.3

Digital leadership – technical confidence

Another area of significance was the perceived technical confidence of participants to set up and maintain an ePortfolio in their schools. At the beginning of the initiative, just over 30% of participants were either confident, very confident or completely confident at setting up and maintaining an ePortfolio system for teachers and students. However, this value increased to approximately 68% by the end of the initiative (Table 16). Beginning of Initiative Answer Choice Not confident at all Not very confident Somewhat confident Confident Very confident Completely confident

% 8.70% 43.84% 17.39% 17.39% 8.70% 4.35%

End of Initiative Count 2 10 4 4 2 1

% 10.53% 0.00% 21.05% 36.84% 15.79% 15.39%

Count 2 0 4 7 3 3

TABLE 16: Confidence of participants to set up and maintain an ePortfolio system for teachers involved in the PDST TY ePortfolio implementation project Further analysis of the interview data also revealed challenges and supports for participants in supporting and leading the implementation of ePortfolios in their schools. These challenges and supports tended to focus on the support offered by management and the time required to upskill other staff members: Do I have sufficient knowledge, yes but that doesn’t mean that it is going to be implemented? So, in order for it to be implemented you need cooperation and also to be honest, collusion. You need the principal on board; you need the teachers on board, you need the deputy on board.



Models of CPDModels of CPD Time and timetabling and year on year you might have a completely different set of teachers, and if you did bring them onboard and along with you, you might find that you don’t have them the next year and that’s a problem. So, all teachers need to implement and be trained which is a big job. Yes, we’re on top of it at the moment, but it’s not something you put in place in a year. We’d have between 80 and 100 in transition year and to get it implemented for that group. We launched in with everyone doing it. All or nothing. From the above comments, it would be reasonable to suggest that on the one hand, although most had the necessary competence to support the technical and assessment aspects of an ePortfolio, goodwill on behalf of the participants was also an essential element: I do have the skills now to lead it, but it’s the time to lead it which is the problem. Does it become like a post in a school? So, unless there is a time or monetary allocation you are back to goodwill. The amount of time that I had to put into the setting it all up after school was an awful lot, not to mention the weekends. 5.5

Level 3: Organisation Support and Change

This section provides a description of the levels of organisational support offered and changes achieved in participant schools. The first subsection describes the organisational planning and supports provided by schools during the ePortfolio initiative. The next subsection provides an analysis of the changing orientation of schools towards the implementation of the showcase, storage and assessment functions of an ePortfolio. 5.5.1

ePortfolio planning and support

One core objective of the PDST initiative was to assist teachers with the implementation of an action plan for the introduction of ePortfolios in their schools. At the beginning of the ePortfolio initiative, less than 9% of schools had an action plan on how ePortfolios would be used as part of the PDST initiative. However, this number increased to 75% by the end of the ePortfolio initiative (Table 17).



Models of CPDModels of CPD

Question Does your school have an action plan on how ePortfolios will be used as part of the PDST TY ePortfolio pilot implementation project?



Answer Choice Yes

8.70%

2

75.00

15

No

91.30%

21

25.00

2

Table 17: Percentage of schools that have an action plan for ePortfolio implementation 5.5.2

ePortfolio readiness

Table 18 shows that at the beginning of the ePortfolio initiative, almost 50% of those participants involved were of the view that their schools were likely or extremely likely to use the storage function of an ePortfolio. By the end of the initiative, this value increased to almost 85%. Beginning of Initiative Answer Choice Extremely unlikely Unlikely Neutral Likely Extremely Likely

% 13.04% 13.04% 26.09% 39.10% 8.70%

End of Initiative Count 3 3 6 9 2

% 5.26% 5.26% 5.26% 26.32% 57.89%

Count 1 1 1 5 11

TABLE 18: Likelihood of schools to use the storage functions of an ePortfolio as part of the PDST ePortfolio initiative Table 19 shows that at the beginning of the initiative, no more than 22% of those participants involved were of the view that their schools were likely or extremely likely to use the showcase function of an ePortfolio. However, this value increased to almost 79% by the end of the initiative.

Answer Choice Extremely Unlikely Unlikely Neutral Likely Extremely Likely

Beginning of Initiative % Count 13.04% 3 21.74% 5 43.48% 10 13.04% 3 8.70% 2

End of Initiative % 10.53% 0.00% 10.53% 21.05% 57.89%

Count 2 0 2 4 10

TABLE 19: Likelihood of schools to use the showcase functions of an ePortfolio as part of the PDST ePortfolio initiative



Models of CPDModels of CPD Table 20 shows that at the beginning of the initiative 39% of those participants involved were of the view that their schools were likely or extremely to gather information from TY student work samples to improve teaching and learning (Assessment of Learning). By the end of the initiative, this value increased to almost 79%. Beginning of Initiative Answer Choice Extremely Unlikely Unlikely Neutral Likely Extremely Likely

% 8.33% 16.67% 29.17% 37.50% 8.33%


% 10.53% 0.00% 10.53% 47.37% 31.58%

Count 2 0 2 9 6

TABLE 20: Likelihood of schools to gather information from TY student work samples to improve teaching and learning (assessment of learning) as part of the PDST ePortfolio initiative Table 21 shows that at the beginning of the initiative, no more than 9% of those participants involved were of the view that their schools were likely or extremely likely to provide feedback to TY students via a student-teacher digital workspace (assessment for learning). However, this value increased to almost 74% by the end of the initiative. Beginning of Initiative Answer Choice Extremely Unlikely Unlikely Neutral Likely Extremely Likely

% 8.70% 39.13% 43.48% 4.35% 4.35%


% 10.53% 5.26% 10.53% 36.84% 36.84%

Count 2 1 2 7 7

TABLE 21: Likelihood of schools to provide feedback to TY students via a Student/teacher digital workspace (assessment for learning) as part of the PDST ePortfolio initiative Although it is evident from tables 17–21 that the PDST initiative resulted in a higher likelihood of schools using all of the various functions of an ePortfolio, the analysis of qualitative data also highlights the fact that the likelihood of schools introducing ePortfolios also depended on internal support provided by senior management in participant schools:



Models of CPDModels of CPD The principal and deputy are definitely on board, and I think that everyone knows with a bit of gazing into the future, this is not only going to be in transition year, this is going to be everywhere, LC, JC its all going to be like this. The idea is trying to build capacity from here. No more than the PDST are trying to build capacity in schools, and I’m trying to build the capacity of teachers when those people have the knowledge of it. So, if you get a critical mass around the middle of the bell curve you might tip that over, and that’s what we’re all aiming for. So, we’re all trying to do the same thing. It must also be noted that for the many schools involved in the PDST initiative, the use of ePortfolios has been a steep learning curve and took a considerable amount of time and effort to implement all the various ePortfolio functions fully. Indeed, the analysis of the qualitative data reveals that more time was required for some schools to fully understand and become comfortable with not only the use of ePortfolios, but also digital technologies for teaching and learning more generally. This point is illustrated in the following participant comments: All TY teachers were aware of the project even though not involved. So, next year we are going to try it out not only in my subject but in all TY subjects. A lot of teachers still needed more training more than just ePortfolios. So, this year was all about training, we all went through the online ePortfolio thing and completed the course, and for next year, we have a core team who are interested in doing it with students. There’s great enthusiasm around as it’s something new because people want to find something new to do. To do this universally, it's going to take time. 5.6

Level 4: Use of New Knowledge and Skills

The analysis of interview data identified the ways in which participants made use of their new knowledge and skills. By the end of the initiative, almost all of the participants used the various functions of an ePortfolio with their classes. This work drew on the practical knowledge related to the use of the various ePortfolio functions developed during the PDST seminars. Participants commented as follows: I set up an ePortfolio template for students at the beginning of the project, and when they complete a piece of work, I use Google Docs to show them what needs to be done to improve. So, at this stage, I’ve set up all of the students with ePortfolios, and they are a lot more active at completing assignments. It is also a lot easier for me to give feedback to students.



Models of CPDModels of CPD Participants use of new knowledge and skills was not only limited to using ePortfolios with their own classes. They also provided training on how to use ePortfolios to students and, in many cases, also provided training to other TY teachers in their schools. So, we had a team that I put together to look at new forms of assessment. So, then we decided to expand, and we said look, our team, we will do it with ePortfolios. So, our team are now the ePortfolio Transition Year team. So, we have about 5 or 6 people on it, and they all said yes. This year I gave TY teachers and students some training, the same training that I got from the PDST. We started off with Google Docs and how to provide feedback, and after they got the hang of that, we then went onto how to set up sharing levels, how to access and upload material into a portfolio. 5.7

Level 5: Student Learning Outcomes

This section provides a description of the change in student learning outcomes since the introduction of ePortfolios. The first subsection describes the change in student skills and behaviour, and the next subsection describes interview participants, perspectives on the introduction of ePortfolios to enhance student performance and achievement. Finally, the last subsection describes the change in student attitudes toward learning. 5.7.1

Student skills and behaviour

Aggregated questionnaire responses show significant digital literacies obtained by students since the beginning of the initiative (Table 22). At the outset of the initiative, 61% of students could create digital files contained within an ePortfolio digital work-space, and 39% of students were able to store digital files. This was not the case, however, for other competencies required to navigate an ePortfolio effectively – a reality that matched the teacher experience. At the beginning of the initiative, only 17% of participants could set up various sharing levels contained in an ePortfolio. However, this value increased to almost 74% by the end of the initiative. As stated by one participant when asked about the digital skills obtained by students: Their skills are far better, they are more confident with the technology, even more confident than they would be with pen and paper, so they don’t mind engaging. I think they enjoyed that rather than to handwrite something, so I think it is useful for them.



Models of CPDModels of CPD Another participant also reported the steep learning curve for some students in their school and the need to provide training in digital literacies before embarking on the use of ePortfolios: I think they are much further on than a few weeks ago because when I started with them, one of the kids asked me how to log on, very basic ICT skills with some of them. Now the same guy has piles of things put into his e-portfolio. So, there’s definitely a need to spend 4 to 5 weeks training the students in ICT skills before starting on the ePortfolio.

Question Are students in your school able to create digital files contained within an ePortfolio digital work-space?



Answer Choice Yes

60.87

14

89.47%

17

No

39.13%

9

10.53%

2

Are students in your school able to Yes 30.43% 7 89.47% 17 store digital files contained within an No 69.57% 16 10.53% 2 ePortfolio digital work-space? Are students in your school able to set Yes 17.39% 4 73.98% 14 up various sharing levels (e.g., view only, share with teacher/peer only) No 82.61% 19 26.32% 5 for digital files contained within an ePortfolio digital work-space? Table 22: Digital literacies obtained by participants (creating, storing and sharing digital

files) In relation to the copyright and legal issues concerning the use of digital files, no more than 9% of students who were involved in the initiative were fully aware of the legal issues relating to copyright and the acceptable use of images, audio, video and text for the purpose of creating, storing and sharing digital content. However, this value increased to approximately 68% by the end of the initiative (Table 23).



Models of CPDModels of CPD

Question Have students in your school been provided with a description of issues relating to copyright and the acceptable use of, for example, images, audio, video and text for the purpose creating, storing and sharing digital content?



Answer Choice Yes

8.70%

2

68.42%

13

No

91.30%

21

31.58%

6

Table 23: Students understanding of copyright and the acceptable use of digital files Finally, only a small minority of students could use the formative assessment function of an ePortfolio to access and respond to peer and teacher feedback. This value increased to almost 74% during the middle and end phases of the ePortfolio initiative (Table 24).

Question Are students in your school able to access and respond to peer and teacher feedback via a studentteacher digital work-space?



Answer Choice Yes

21.74%

5

73.68%

14

No

78.26%

18

26.32%

5

Table 24: Digital literacies obtained by students (responding to peer and teacher feedback) The benefits of ePortfolios also extended beyond that of learning rudimentary skills needed to navigate an ePortfolio system. When asked about the impact of the initiative, interview participants reported that there was an increased motivation for students to improve their learning. As stated by one participant, ‘they have a self-motivated interest in making that as good as it can be as a result of participation in the course without doubt’. When asked about the effect of ePortfolios on students’ confidence, interview participants also reported positive outcomes: Definitely it gave them more confidence as learners, and they didn’t see themselves as failures. Along with that, they’ve tasted achievement whereas before they might never have achieved it was just through an examination. I think it has been great for our less confident students who are unsure about whether they are hitting the mark. I can drop in and give them feedback and support when they need it and a bit of a kick as well.



5.7.2

Student performance and achievement

Almost all participants were of the view that the introduction of ePortfolios has had a positive impact on student performance and achievement. However, some participants reported that it was too early in the initiative to fully ascertain the impact of ePortfolios on student performance: ‘I don’t know just yet what the impact is as we’re just about to get the students ready to be assessed’. Another participant also stated that it would be ‘too early to give an answer as they spent a lot of time just learning how to use an ePortfolio’. Other participants noted a considerable increase in the quality of student work compared to previous years when paper-based ePortfolios were used. According to one participant, ‘there is no comparison in the standard of work between students who used paper-based versus e-portfolios. Even better students didn’t put as much work into the paper-based portfolios’. Finally, other participants reported that students are now more inclined to review their work for various reasons, such as the dynamic and self-contained structure of an ePortfolio: I think they are much more inclined to implement it when it is in that format because they are reading back over it and taking things on board. Whereas often, even when you write in a copy, despite your own best will, it is often never seen again or often not redone. Its giving them more of an opportunity to do different drafts of their work, so I think it is improving the quality of stuff they are producing. It led to opportunities for students to display their work, showcased in a way they could not have done before previously and allowed opportunity to assess learning that would have been impossible to demonstrate before and given credit for it. 5.7.3

Student attitudes toward learning

Almost all interview participants were of the view that the introduction of ePortfolios had a positive impact on student attitudes towards learning, such as digital learning, as illustrated in the following comments: Now they’re doing it, and they’re saying, this has a practical application. If I’m building my own page through sites [the ePortfolio template provided] I need to be able to type, I need to be able to work with images, databases, so they now see a purpose to their ICT learning as opposed to just doing it, typing for the year. So, they’re learning life skills.



Models of CPDModels of CPD Like in ICT before this, we had had an ICT booklet, and they kind of followed it ad lib. So, they did assignment 1, assignment 2. So yes, they’re learning ICT skills, but it’s not really registering with them, why are we doing this? Other participants reported that the introduction of ePortfolios resulted in students becoming more reflective and appreciative of their learning, as illustrated in the following participant comments: The impact would have been that at the start they were up in arms that they had to do all of this extra work in TY so straight away they saw that this was extra work that they don’t normally have to do. They saw it as being easier to go in and have to do an exam. But the big thing is now the student has to think about what he is learning and evaluates it and see how beneficial it is to him or not to him as long as he justifies it. So basically, the student is thinking about what he’s learned. He’s not just accepting it, he’s thinking about it. It has given them confidence in terms of ICT for a start, and even, kind of, more of an appreciation of what they are learning and the experiences they are having. Participants also reported that students found learning with ePortfolios to be ‘more enjoyable than paper-based approaches and they worked more consistently and kept up to date, rather than a big rush at the end’. Evidently, according to one participant, this gave students ‘a sense of achievement and kept them on track as often students in TY forget all the activities and learning over the TY year’.

5.8 Peripheral Barriers and Supports towards ePortfolio Integration in Education There were a number of significant achievements in the course of the initiative that facilitated the move toward the implementation of ePortfolios by the PDST at a system level and most significantly, teachers and students at a school level. Whilst acknowledging this, it would be remiss to suggest that peripheral barriers towards the introduction of ePortfolios did not exist. For some participants, the central importance of state examinations in Ireland challenged the efficacy of introducing ePortfolios not only for teachers, but also for students: There needs to be a relevant output for students after they leave secondary school and third level education. So, what is the point? Definitely because of the intrinsic learning along the way but to explain to a 15, 16-year-old that ‘we’re doing this for the intrinsic educational value, well forget about it.



Models of CPDModels of CPD Furthermore, it is frequently suggested in the discourse of government policy implementation that there needs to be a change in the assessment practices of teachers. However in this initiative, it was found that while almost all teachers saw the value of providing formative feedback to students a number of teachers reported that some students struggled with the concept of submitting draft assignments for formative feedback. As stated by one participant: ‘I had one girl who is a bit of a perfectionist and who hated the idea of me being able to see her work in draft form. She was used to polishing it before she gave it to me and this was a big challenge for her’. Other participants reported that in the initial stages of the initiative, students disliked the fact that teachers could frequently monitor them. According to one participant, ‘some students hated the idea of being ‘monitored’ all the time. They felt it was like a “Big Brother” watching them’. Finally, it is unarguable that the PDST initiative was successful on many grounds and significantly exceeded what was initially envisaged. However, there were issues that arose, and one particularly noteworthy barrier that impacted the introduction of ePortfolios was the amount of time and resources that participants were required to give to training other staff members. Whereas the value of internal CPD and creating digital leaders in schools is to be commended, one must question the sustainability of this model of internal training given the limited amount of time that participants can dedicate to such activities. In this regard, it is recommended that the quality of existing resource hours, such as those available through Croke Park Agreement, be revisited to include a strong emphasis on internal and local school-to-school CPD. The following participant comment illustrates this point: The biggest challenge for me has been the internal training of the teachers. Because I was speaking to a teacher the other day and he said, will we be doing this in the morning or the afternoon as we simply didn’t have the time to do this between classes. So, the alternative was that I had to get teachers released from day-to-day classes and that took a lot of work for the principal. So, I think that’s a very big hurdle that needs to be worked on.



Evaluation of the PDST ePortfolio Initiative: Formative Assessment using ePortfolios |71

DISCUSSION AND CONCLUSION

Chapter 6 Discussion and Conclusion



6.1

Introduction

The current report presents the results of an evaluation of the PDST ePortfolio initiative, and within this, a number of interesting findings emerge that have implications for the future of CPD in Ireland and elsewhere. In addition, there are a number of general findings related to the use of ePortfolios in schools that should be of interest to a wider educational audience. The chapter begins with an analysis of the model of CPD that was developed, exploring how this impacted the success of the ePortfolio initiative. This is followed by a short discussion of the role of networks in the CPD framework. The role of the PDST is then examined in some detail, and following this, an exploration of the potential of professional forums for discussion will be presented. An analysis of the role of communication in the success of the initiative is then outlined, and finally, suggestions as to the orientation of future CPD are made. The second section of the chapter examines some of the general findings that emerged concerning the use of ePortfolios in schools. The broad issue of digital benefits will be examined from the perspective of both teachers and students. In addition, the relationship between ePortfolio adoption and assessment frameworks in the Irish education system is explored in some detail. Finally, the chapter concludes with some general conclusions and suggestions as to the potential for this model of CPD to be rolled out across other educational initiatives within Ireland and beyond. 6.2

Discussion

The study as a whole makes a strong case that there is a need to distinguish between onetime-seminar-based CPD opportunities and the CPD that was provided in this initiative. Support services can usefully provide government-priority seminars in areas such as digital literacies. However, it is hard to envisage that this kind of support would lead to improvement unless there is an ongoing asynchronous communication and support between the different actors, one that is overseen continually by an audit unit of the Department of Education. In other words, the model of CPD that was provided in this initiative, as opposed to face-to-face seminars, implies the existence and nurturing of a continuous discourse for improvement between support services and schools. This point emerges strongly from the research. There was widespread agreement among the research participants that although it is appropriate for support services to provide cyclical professional development opportunities to teachers, nothing concrete will happen unless schools, with the support of CPD providers, take ownership of what has been learned and from this, set in train the necessary improvement strategies.



Models of CPDModels of CPD In the case of this initiative, although it was expected that teachers would implement what had been learned in their classrooms, there was no requirement for them to provide, let alone to lead, professional development opportunities for colleagues in their schools. That it was decided to do so, was done in conjunction with the PDST, thus resulting in a genuine strengthening of the digital capacity of participating schools, is a welcome, if unexpected outcome of the initiative. All of this is only conceivable in the context of a carefully planned model of CPD. 6.2.1

The role of internal networks

Related to the above finding is the question of the importance of internal collaborative school networks that involve other teachers who were not provided with seminar CPD. As initially envisaged, the first series of seminars was more focused on training individual teachers. However, it was pointed out by the key research respondents that a major impact of the initiative has been the development of a much wider cooperative network in individual schools. Without this growth resulting from the internal leadership and training provided by participants in their schools, the success of the initiative would have been diminished. 6.2.2

The role of the PDST

All the respondents without exception stressed the importance of follow-up support provided by the PDST. As a result of this, relationships and trust were built up between schools and the PDST, which were seen as being a significant factor relating to the success of the initiative. Indeed, it was stressed time and again that a significant contributing success of the initiative was the follow-up support provided by the PDST, the gradual reduction of support required and the internal CPD provided by participants to other teachers in their schools. However, although the internal leadership and CPD provided by participants in their own schools is to be acknowledged and commended, particularly given the time pressures faced by teachers, it is questionable how sustainable this model of internal CPD can be. In this regard, it is suggested that available school resources, such as those provided by the Croke Park agreement, be utilised for the provision of internal and local school-to-school CPD opportunities based on the models developed in this initiative. This in turn will improve a wide range of targeted improvement areas for schools through the facilitation of wider access to CPD for teachers and will reduce the overall amount of money spent on releasing teachers to attend CPD seminars outside of their local area or school.



6.2.3

The creation of professional fora

Another important element to emerge from the current study is a recognition of the necessity of providing collaborative opportunities for teachers across a range of curricular areas. This particular initiative led to teachers to create what amounts to a professional forum for the exchange of ideas, materials and experiences. This outcome is contrasted with traditional single-school whole-staff professional development opportunities provided by support services, which, although very valuable in many ways, arguably tend to encourage schools to solve problems by themselves, reinforcing rather than breaking down barriers to improvement. This emerging reality is also significant in the context of recent policy decisions that are seeking to encourage schools to work in local ‘clusters’ or ‘networks’ (Government of Ireland, 2016). This initiative demonstrates that with proper support and judicious investment in school personnel, it is possible to create a meaningful professional learning network that draws on the strengths of individual schools and teachers in a way that enhances both the individual and the collective. 6.2.4

The importance of communication

All parties emphasised regular contact with and follow-up by the PDST. Respondents contrasted more traditional modes of CPD with long gaps between professional development events with the process of professional development in this initiative. This approach was perceived as growing the capacity of the schools as it becomes more mature, helping it to focus on using the school's available resources to examine issues around teaching and learning and teacher professional development. In this way, it was suggested that there should be a shared agenda for improved teaching between the PDST and schools that in many cases was constantly revisited and emergent. 6.2.5

Orientation of future CPD

As has already been suggested, an important question posed by the current study is whether learning activities such as online learning can in fact replace face-to-face professional learning programmes. It is always difficult to establish such related effects in educational research, and this study is no different here. However, what can be said is that most of the respondents in this initiative stated that although the PDST ePortfolio course provided a vital surface-level understanding of ePortfolios, there was an overall agreement that a deeper understanding of ePortfolios could have only been achieved via a structure that involved seminars and follow-up professional development opportunities.



Models of CPDModels of CPD If the views of participants involved in this initiative are to be taken as having wider applicability, then future iterations of system-level priorities should first be implemented with introductory online modules on the topic that professional development providers could then provide what might be described as challenging professional development opportunities for teachers in a range of modes. 6.3

Additional Discussion Points

In addition to the findings that were relevant to the particular model of CPD that emerged from the study, other interesting discussion points emerged regarding the broadly based ‘digital benefits’ that came from engaging in the initiative, as well as a range of outcomes exploring the relationship between ePortfolio usage and assessment. 6.3.1

Digital benefits

Among the more interesting issues to arise from the overall evaluation of the ePortfolio initiative was the extent to which participants – both teachers and students – became far more comfortable working with ICT in a range of educational settings. For teacher participants, these benefits ranged from enhanced confidence in the practical, technical aspects of their role to their increasing capacity to take on digital leadership roles within their schools. In the area of technical capacity, the evaluation shows that the level of comfort exhibited by teachers involved in the initiative increased markedly over the course of the evaluation. They moved, for example, from a situation where issues such as the creation of sharing structures for digital material was seen as a complex and challenging requirement of a new process to one where it was a common, everyday task that they were willing and able to complete. In addition, they exhibited a greater understanding of the broader pedagogical and legal challenges surrounding the use of digital material in classroom situations. Issues relating to the development of assessment strategies at, the department and school levels tailored to the particular needs of cohorts of students using ePortfolios became part of the everyday pedagogical practice of the teachers involved. They also developed an important expertise that allowed them to navigate the complex copyright issues surrounding the use of digital images and other media. As has been discussed in the previous section, it is significant that the participating teachers in all schools developed structures that allowed them to share these new skills and knowledge with colleagues within their school communities.



Models of CPDModels of CPD A broad range of digital benefits were also found in the analysis of student engagement with ePortfolios. They too enhanced their baseline skills in areas such as file creation and management, engagement in discussion and other reflective fora, manipulation of a range of media and development of recording mechanisms for use in an ePortfolio context. They became more literate in the areas of copyright issues and the appropriate use of private data in public spaces, a key skill for today's networked world. 6.3.2

ePortfolios and assessment

One of the more interesting series of discussion points that emerged from this evaluation is the link between ePortfolios and the assessment structures in Irish second-level schools. As shown, at a policy level at least, there is a recognition that ePortfolios have a clear role to play in the development and integration of formative assessment strategies in schools. In addition to this, there is a strong case to be made that the inclusion of ePortfolios in the assessment structure of schools fundamentally changes the teacher-student relationship. To quote the Eufolio programme, which was referenced earlier in the study, the use of ePortfolios in assessment contexts leads to the development of ‘collaborative, continuous discourse between teacher and student’ (EUfolio 2015b, 5). This represents something of a departure for many schools and demands, as an initial step, the reconceptualisation of the roles of the student and the teacher, moving away from more traditional, passive teaching strategies and instead moving toward more engaged, dynamic and fluid approaches to knowledge creation and mastery. In practice, this new conceptualisation required teachers and students to develop new ways of interacting. For teachers, there was a requirement to become more facilitative. They were asked to engage with ongoing, developmental feedback with students – challenging them where necessary, guiding them when asked and responding to them when needed. This mode of engagement was both time-consuming and different and placed significant demands on teachers – demands that they, for the most part, rose to. For students, the challenges were similar but focused on a different manner. The evaluation identified a change in their learning approach. Again, the concepts of activity, engagement, reflection and feedback were central. Students who engaged with the ePortfolio structure soon came to see learning as an iterative process – one developed over time and across a range of formats. In this conceptualisation, the key ‘skills’ of learning moved from memorisation to those of review, reflection, communication and collaboration. These were significant changes and ones that required a great deal of flexibility on the part of the student. As we have seen, not all were comfortable with the transition to a learning style that encouraged the sharing of incomplete or imperfect work, and this remains a challenge for those seeking to introduce this type of learning approach.



Models of CPDModels of CPD An additional and very significant challenge is the broad assessment framework within which the ePortfolio initiative was introduced. As discussed in Chapter 3, there was a deliberate decision to develop this initiative as part of the TY structure. TY was considered ideal because ‘it provides a bridge to help pupils make the transition from a highly structured environment to one where they will take greater responsibility for their own learning and decision making’ (2010: 1). Of course, the development of the new Junior Cycle assessment framework has, in theory at least, reduced the highly structured learning environment of lower secondary education in Ireland and encouraged greater exploration on the part of the student. Notwithstanding that, the decision to embed ePortfolio in TY is an explicit recognition of the assessment structure and philosophy that underpins that year. The absence of a high-stakes exam focus, so prevalent in upper secondary, gives the student and teacher an opportunity to innovate and explore. The evaluation confirms that this environment encourages students to experiment and take risks. In addition, it encourages them to take ownership of their learning and facilitates their providing evidence of this through the capacities embedded in the ePortfolio structures. The challenge, of course, is how to transition these new knowledge, skills and approaches to learning to the high-stakes leaving certificate environment. This is a challenge that participants at all levels were very well aware of, and it is one that will have to be addressed if the ePortfolio initiative is to have a wider systemic impact beyond TY. 6.4

Conclusion

One of the more interesting conclusions that can be drawn from this study relates to the impact on the work of the PDST as a whole. There was strong agreement by participants that the PDST, particularly the ongoing engagement of PDST advisors with participants, was vital in driving forward the ePortfolio agenda of their schools. A key theme that emerged was the gradual change of professional learning provision from the PDST to the teachers involved in the initiative. Respondents noted that from the beginning of the initiative, the PDST placed a great emphasis on school professional development provision and in response, teachers who were involved in the initiative sought to develop the learning capacity of colleagues in their schools. If this model of CPD, with the relevant range of the required supports, is followed by all professional development support services, the appropriate role of professional development support services then becomes one of facilitating, as opposed to providing, professional learning in schools. An important aspect to be considered is the extent to which what appeared to be a very successful initiative that resulted in the implementation of ePortfolios in participant schools, could be replicated across all schools.



Models of CPDModels of CPD This is a somewhat difficult question to answer. As indicated in participant responses, there are many issues that need to be resolved before ePortfolios become a standard tool for learning. These issues relate not so much to structural realities, such as the equity of access to broadband both in and outside of the school environment but rather, to the need to challenge assumptions relating to the benefits of ePortfolios as a powerful tool for learning, not only for teachers, but also, most significantly, for students. Finally, it must also be noted that the PDST ePortfolio initiative went way beyond that of providing professional learning for the use of ePortfolios in education. In this initiative, professional learning partnerships between and within participant schools also emerged. In summary, it seems that respectful, reciprocal, non-hierarchical and most of all challenging engagement within and between schools and the PDST is a core requirement for all professional learning. Finally, if the overall goal of the PDST is ‘for it to be widely acknowledged as an innovative, responsive and trusted provider of continuing professional development and support for teachers and school leaders’ (PDST,2015, P.13), then the the goal of the PDST was undoubtedly achieved in this initiative. Innovative in the sense that with all the varying participant capacities, a longitudinal initiative of this depth and scale has rarely been carried out to the extent that it has been with this initiative. Responsive in the sense that not only did the PDST effectively respond to system level priorities such as the enhancement of assessment for learning and twenty-first century learning skills, but were also responsive to the individual needs of those schools involved in the initiative. Trusted in the sense that the PDST sought to enhance various aspects of teaching and learning, such as assessment across all school types, in the face of external systematic challenges.


BIBLIOGRAPHY

Bibliography


BIBLIOGRAPHY

Akçıl, U., and Arap, I. 2009. The opinions of education faculty students on learning process involving e-portfolios. Procedia Social and Behavioral Sciences, (1), pp. 395-400. Argyris, C., and Schon, D.A. 1996. Organizational learning II: Theory, method, and practice. Reading. MA: Addison-Wesley. Austin, R. and Hunter, B. 2013. ICT Policy and Implementation in Education: Cases in Canada, Northern Ireland, and Ireland. European Journal of Education, 48(1), pp. 178-192. Barrera-Pedemonte, F. 2016. High-Quality Teacher Professional Development and Classroom Teaching Practices: Evidence from Talis 2013, OECD Education Working Papers, (141). Paris: OECD Publishing. Barrett, H.C. 2010. Balancing the Two Faces of ePortfolios. Educação, Formação & Tecnologias, 3(1), pp. 6-14.

Baylor, A.L. and Ritchie, D. 2002. What factors facilitate teacher skill, teacher morale, and perceived student learning in technology-using classrooms [Online]. Available from: http://www.motivationalexperience.com/PDF/factor.pdf [Accessed 23 January 2017]. Black, P. and Wiliam, D. 2010. Inside the Black Box. Kappan Classic, 92(1), pp. 81-90. Bodkin, P. 2013. Framework for the Continuing Professional Development of Art Teachers at Post-Primary Level. Doctoral Thesis, Dublin City University. Borkan, J. 1999. Immersion/Crytallisation, IN: Crabtree, B. F. and Miller, W. (eds.), Doing qualitative research (2nd ed.). London: SAGE. pp.179-194. Borko, H. 2004. Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), pp. 3–15. Borko, H., Michalec, P., Timmons M. and Siddle J. 1997. Student Teaching Portfolios: A Tool for Promoting Reflective Practice, Journal of Teacher Education, 48(5), pp. 345-357. Bubb, S. and Earley, P. 2009. Leading staff development for school improvement. School Leadership and Management, 29(1), pp. 23-37. Buzzetto-More, N.A. 2007. Advanced Principles of Effective e-Learning. Informing Science Press: California.


BIBLIOGRAPHY

Caena, F. 2011. Literature review: quality in teachers’ continuing professional development. Brussels: European Commission Directorate-General for Education and Culture. Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C. P., and Loef, M. 1989. Using knowledge of children’s mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26(4), pp. 499–531. Chang, C.C., Liang, C., Tseng, K.H. and Tseng, J.S. 2014. Using e-portfolios to elevate knowledge amassment among university students. Computers & Education, 72(1), pp.187195. Clarke, D.J. and Hollingsworth, H. 2002. Elaborating a model of teacher professional growth. Teaching and Teacher Education, 18, pp. 947-967. Clewell, B. C., Campbell, P. B., and Perlman, L. 2004. Review of evaluation studies of mathematics and science curricula and professional development models. Washington DC: Urban Institute. Collinson, V., Kozina, E., Kate Lin, Y.H., Ling, L., Matheson, I., Newcombe, L. and Zogla, I. 2009. Professional development for teachers: A world of change. European journal of teacher education, 32(1), pp.3-19. Consortium for School Networking. 2013. Rethinking Acceptable Use Policies to Enable Digital Learning: A Guide for School Districts [Online]. Available from: http://www.cosn.org/sites/default/files/pdf/Revised%20AUP%20March%202013_final.pdf [Accessed 18 January 2017].

Cramer, S.P. 1993. Navigating the Assessment Maze with Portfolios. The Clearing House, 67(2), pp.72-74. Creswell, J. W. 2008. Research design: qualitative, quantitative, and mixed methods approaches (3rd ed.). [Kindle Edition 1.10.1]. Thousand Oakes, CA. SAGE. Cyprus Pedagogical Institute. 2015. EUfolio: EU Classroom ePortfolios Training Booklet [Online]. Available from: http://eufolio.eu/docs/EUfolio_Training_Booklet.pdf [Accessed 17 January 2017]. Dam, G. and Blom, S. 2006. Learning through participation. The potential of school-based teacher education for developing a professional identity. Teaching and Teacher Education, 22, pp. 647 - 660.


BIBLIOGRAPHY

Day, C. 1999. Developing teachers: the challenges of lifelong learning. London: Falmer. Day, C., and Sachs, J. 2004. Professionalism, performativity and empowerment: discourses in the politics, policies and purposes of continuing professional development IN: C. Day and J. Sachs (eds.), International handbook on the continuing professional development of teachers. Berkshire: McGraw-Hill, pp. 3–63. Denscombe, M. 2010. The good research guide: for small-scale social research projects. Oxford: Open University Press. Department of Education and Skills. 1994. Transition Year Programmes – Guidelines for Schools[Online]. Dublin: Department of Education and Skills. Available from: https://www.education.ie/en/Schools-Colleges/Information/Curriculum-andSyllabus/Transition-Year-/ty_transition_year_school_guidelines.pdf [Accessed 17 June 2017]. Department of Education and Skills. 1997. Schools IT 2000, A Policy Framework for the New Millennium. Dublin: Department of Education and Skills.

Department of Education and Skills. 2008. ICT in Schools: Inspectorate Evaluation Studies [Online]. Available from: https://www.education.ie/en/Publications/Inspection-Reports-Publications/EvaluationReports-Guidelines/ICT-in-Schools-Inspectorate-Evaluation-Studies.pdf [Accessed 17 February 2017]. Department of Education and Skills. 2015. Digital Strategy for Schools: Enhancing Teaching, Learning, and Assessment. Dublin: Department of Education and Skills.

Department of Education and Skills. 2017a. Digital Learning Framework for Post-Primary Schools. Dublin: Department of Education and Skills. Available from: http://www.pdsttechnologyineducation.ie/en/Planning/Digital-Learning-Framework/DigitalLearning-Framework-Post-Primary.pdf [Accessed 2 November 2017].

Department of Education and Skills. 2017b. Digital Learning Framework for Primary Schools. Dublin: Department of Education and Skills. Available from: http://www.pdsttechnologyineducation.ie/en/Planning/Digital-Learning-Framework/DigitalLearning-Framework-Primary.pdf [Accessed 2 November 2017].


BIBLIOGRAPHY

Department of Interactive Media and Educational Technologies. 2015. EUfolio Process Specification [Online]. Available from: https://eufolioresources.files.wordpress.com/2015/03/del10_eufolioprocessspecification_e xtsum_en.pdf [Accessed 23 January 2016]. Desimone, L. M. 2009. Improving impact studies of teachers’ professional development: toward better conceptualizations and measures. Educational Researcher, 38(3), pp. 81 - 199. Edmonds, S., and Lee, B. 2002. Teacher’s feelings about continuing professional development. Education Journal, (61), pp. 28-29. ElfEL. 2016. ePortfolio for all [Online]. Available from: http://www.eifel.org/activities/campaigns. [Accessed 20 January 2016].

Eufolio. 2015a. ePortfolio Implementation Guide for Policymakers and Practitioners [Online]. Available from: http://eufolio.eu/docs/ePortfolio_Implementation_Guide.pdf [Accessed 19 January 2016].

Eufolio. 2015b. EU Classroom ePortfolios: Pilot Evaluation Results [Online]. Available from: http://eufolio.eu/docs/PilotEvaluationResults.pdf [Accessed 23 January 2016].

Eufolio. 2015c. Review of Existing ePortfolio Policies and Practices [Online]. Available from: http://eufolio.eu/docs/D.8%20Policy%20Practice%20Review_short_EN.pdf [Accessed 10 April 2016].

Eufolio. 2015d. EU Classroom ePortfolios: Training Booklet [Online]. Available from: http://eufolio.eu/docs/EUfolio_Training_Booklet.pdf [Accessed 22 April 2016]. European Commission. 2013. Key Data on Teachers and School Leaders in Europe. 2013 Edition [Online]. Available from: http://eacea.ec.europa.eu/education/eurydice/documents/key_data_series/151en.pdf . [Accessed 01 August 2017]. European Commission. 2012. Supporting teacher competence development for better learning outcomes [Online]. Available from: http://ec.europa.eu/education/school education/doc/teachercomp_en.pdf. [Accessed 15 July 2017].


BIBLIOGRAPHY

Fidel, R. 2008. Are we there yet? Mixed methods research in library and information science. Library and Information Science Research, (30), pp.265-272. Fishman, B.J. et al. 2003. Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, (19), pp. 643-658. Fitch, D., Peet, M., Glover-Reed, B. and Tolman, R. 2008. The use of ePortfolios in Evaluating the Curriculum and Student Learning. Journal of Social Work Education, 44(3), pp. 37-54. Foddy, W. 1994. Constructing questions for interviews and questionnaires: theory and practice in social research. London: Cambridge University Press. Garet, M.S. et al. 2001. What makes professional development effective? Results from a national sample of teachers, American Educational Research Journal, 38(4), pp. 915 - 945. Geijsel, F., Sleegers, P., Stoel, R. and Krüger, M. 2009. The Effect of Teacher Psychological, School Organizational and Leadership Factors on Teachers’ Professional Learning in Dutch Schools. The Elementary School Journal, 109 (4), pp. 406-427. Gibson, D. 2006. ePortfolio Decisions and Dilemmas IN: Jafari, A. (ed). Handbook of Research on ePortfolios. USA: Idea Group Reference, pp. 135-146. Goldsmith, L., Doerr, H., and Lewis, C. 2014. Mathematics teachers’ learning: a conceptual framework and synthesis of research. Journal of Mathematics Teacher Education, 17(1), pp. 5–36. Government of Ireland. 2016. A Programme for a Partnership Government [Online]. Available from: https://merrionstreet.ie/MerrionStreet/en/ImageLibrary/Programme_for_Partnership_Gov ernment.pdf [Accessed 21 April 2017]. Guskey, T.R. 1999. New Perspectives on Evaluating Professional Development. American Educational Research Association Annual Conference, 19th – 23rd April 1999. Montreal, Quebec. Guskey, T. R. 2000. Evaluating professional development. Thousand Oaks: Corwin Press. Guskey, T. R. 2002. Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), pp. 45-51.


BIBLIOGRAPHY

Guskey, T. R., and Sparks, D. 2004. Linking Professional Development to Improvements in Student Learning, IN: Guyton, E. M. and Rainer Dangel, J. (eds.), Research Linking Teacher Preparation and Student Performance, Dubuque: Kendall/Hunt Publishing Company, pp. 1122. Hargreaves, A., and Fink, D. 2000. The three dimensions of reform. Educational Leadership, 57 (7), pp. 30 – 34. Hattie, J. 2008. Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement. Oxon: Routledge.

Hattie, J. and Timperley, H. 2007. The power of feedback. Review of Educational Research, 77(1), pp. 81-112.

Hawley, W., and Valli, L. 1999. The essentials of effective professional development: A new consensus IN: Sykes, G. and Darling-Hammond, L. (eds.), Teaching as the learning profession: Handbook of policy and practice, New York: Teachers College Press, pp.127–150. Hislop, H. 2015. Advancing Learning, Assessment and Evaluation in Irish Second Level Schools. Address to Association of Community and Comprehensive Schools Annual Convention 2015, 19 March 2015. Galway Bay Hotel, Galway [Online]. Available from: https://www.education.ie/en/Press-Events/Speeches/2015-Speeches/SP15-03-19.pdf [Accessed 21 April 2016]. Ingvarson, L., Meiers, M. and Beavis, A. 2005. Factors affecting the impact of professional development programs on teachers’ knowledge, practice, student outcomes and efficacy. Education Policy Analysis Archives, 13(10), pp. 1-28. Jaquith, A., Mindich, D., Wei, R. 2010. Pockets of excellence. Journal of Staff Development, 31 (5), pp. 52-57. Jeanpierre, B., Oberhauser, K., and Freeman, C. 2005. Characteristics of professional development that effect change in secondary science teacher’s classroom practices. Journal of Research in Science Teaching, 42(6), pp. 668–690. Johnston, K. 2014. The Development and Implementation of ICT Policy for Schools in the Irish Post-Primary Context: A Critical Analysis, Doctoral dissertation, University of Limerick.


BIBLIOGRAPHY

Klenowski, V. 2000. Portfolios: promoting teaching, Assessment in Education, 7,(2), pp. 215236. Kok, I., and Blignaut, S. 2009. Introducing developing teacher-students in a developing context to E-Portfolios [Online]. Available from: http://www.puk.ac.za/opencms/export/PUK/html/as/oo/OO_Docs_2009/ePortfolios_NWU_Kok_Blignaut_ILearn_09_02-01-09.pdf [Accessed 22 January 2017]. Lorenzo, G., and Ittelson, J. 2005. An Overview of E-Portfolios [Online]. Available from: https://net.educause.edu/ir/library/pdf/ELI3001.pdf [Accessed 20 January 2017]. Johnson, C., Kahle, J., and Fargo, J. 2007. A study of the effect of sustained, whole-school professional development on student achievement in science. Journal of Research in Science Teaching, 44(6), pp.775-786. Kennedy, A. 2014. Models of Continuing Professional Development: a framework for analysis. Professional Development in Education, 40 (3), pp. 336–351. Kennedy, M. 1998. Form and substance in in-service teacher education. Arlington, VA: National Science Foundation. King, F., and Stevenson, H. 2017. Generating change from below: what role for leadership from above? Journal of Educational Administration, 55 (6), pp.657-670. Kwakman, K. 2003. Factors affecting teachers’ participation in professional learning activities. Teaching and Teacher Education, (19), pp.149-170. Lorenzo, G., and Ittelson, J. 2005. An Overview of E-Portfolios [Online]. Available from: https://net.educause.edu/ir/library/pdf/ELI3001.pdf [Accessed 20 January 2017]. Love, T. and Cooper, T. 2004. Designing Online Information Systems for Portfolio-Based Assessment: Design Criteria and Heuristics. Journal of Information Technology Education, (3), pp. 66-81. McArdle, K. and Coutts, N. 2010.Taking teachers' continuous professional development (CPD) beyond reflection: adding shared sense-making and collaborative engagement for professional renewal. Studies in Continuing Education, 32 (3), pp. 201-215. McCloud, R. 2004. Using ePortfolios for Engaged Learning [Online]. Available from: https://www.ctdlc.org/Evaluation/FIPSEdocs/EngagedLearning2.pdf [Accessed 6 April 2017]. McGhee, R. and Kozma, R. 2001. New teacher and student roles in the technologysupported classroom. Annual meeting of the American Educational Research Association, pp.1-10.


BIBLIOGRAPHY

McMillan, D., McConnell, B. and O’Sullivan, H. 2016. Continuing Professional Development-Why Bother? Perceptions and Motivations of Teachers in Ireland. Professional Development in Education 42 (1), pp. 150–67. Marshal, K. and MacNair, V. 2005. e-portfolios: reflection, collaboration and sustainability in early teacher education [Online]. Available from: http://bit.do/mmc2005 [Accessed 15 July 2017]. Mears, C. 2012. In-depth interviews IN: Arthur, J., Coe, R., Hedges, L. and Waring, M. (eds.), Research methods and methodologies in education. London: SAGE, 170 -175. Meyer, K. D. and Tusin, F. L. 1999. Preservice Teachers’ Perceptions of Portfolios: Process Versus Product, Journal of Teacher Education, 50 (2), pp. 131-139. Miles, M. and Huberman, A. 1994. Qualitative data analysis: An expanded source book (2nd ed.). Thousand Oaks, CA: SAGE. Moyles, J. 2007. Observation as a research tool IN: Briggs, A. and Coleman, M. (eds.) Research methods in educational leadership and management (2nd ed.). London: SAGE, pp.237-257. Mujis , D., Day, C., Harris, A. and Lindsay, G. 2004. Evaluating CPD: an overview. IN: Day, C. and Sachs, J. (eds.). International handbook on the continuing professional development of teachers. Buckingham: Open University Press, pp. 291-308. Mulkeen, A. 2003. What can policy makers do to encourage integration of information and communications technology? Evidence from the Irish school system. Technology, Pedagogy and Education, 12(2), pp.277-293.

National Council for Curriculum and Assessment. 2016. Assessment toolkit Available from: http://juniorcycle.ie/Assessment [Accessed 5 April 2017].

[Online].

Ochola, J.E., Achrazoglou, J. and Anthony, R. 2015. ePortfolio using the Power of Non-Linear Space to create and interlink a Repertoire of Skills Essential for Teaching. Electronic Journal for Inclusive Education, 3(4), pp.1-28. OECD. 2014. TALIS 2013 Results: An International Perspective on Teaching and Learning. Paris: OECD Publishing. Oppenheim, A. 1998. Questionnaire design, interviewing and attitude measurement. London: Pinter.


BIBLIOGRAPHY

Opfer, D. (2016). Conditions and practices associated with teacher professional development and its impact on instruction in TALIS 2013 (OECD Education Working papers, No. 138) [Online]. Available from: http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=EDU/ WKP(2016)12&docLanguage=En [Accessed 15 September 2017]. Patton, M., 2002. Qualitative research & evaluation methods (3rd ed.). California: SAGE. Paulson, F.L., Paulson, P.R. and Meyer, C.A. 1991. What makes a Portfolio a Portfolio? [Online]. Available from: http://web.stanford.edu/dept/SUSE/projects/ireport/articles/eportfolio/what%20makes%20a%20portfolio%20a%20portfolio.pdf [Accessed 02 January 2017]. Pelgrum, W.J. 2001. Obstacles to the integration of ICT in education: results from a worldwide educational assessment. Computers & education, 37(2), pp.163-178. Penuel, W. R., Fishman, B., Yamaguchi, R., and Gallagher, L. P. 2007. What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), pp. 921–958. Professional Development Service for Teachers. 2017. The Professional Development Service for Teachers Strategic Plan 2015-2020. Dublin: Professional Development Service for Teachers. Available from: https://drive.google.com/file/d/0BwubrsqYxsjISjZTNFR1R0Nqd2c/view [Accessed 18 September 2017] Professional Development Service for Teachers. 2016a. Google Sites for ePortfolio. Creating your Schools ePortfolio template. Dublin: Professional Development Service for Teachers. Professional Development Service for Teachers. 2016b. Google Sites for ePortfolio. Creating your Schools ePortfolio template. Dublin: Professional Development Service for Teachers. Professional Development Service for Teachers. 2015. ePortfolios: Using Digital Tools for Content Creation, Collaboration and Assessment (Post Primary) [online]. Dublin: Professional Development Service for Teachers. Available from: http://www.pdsttechnologyineducation.ie/en/Training/Courses/ePortfoliosUsing-Digital-Tools-for-Content-Creation-Collaboration-and-Assessment-Post-Primary-.html [Accessed 10 September 2017] Professional Development Service for Teachers. 2014. A Background Paper to accompany the PDST Proposal for the Integrated Language Curriculum CPD Framework. Dublin: Professional Development Service for Teachers.


BIBLIOGRAPHY

Putnam, R.T. and Borko, H. 2000. What do new views of knowledge and thinking have to say about research on teacher learning. Educational Researcher, (29), pp. 4-15. Rate, N. 2008. Assessment for Learning and ePortfolios: What are the formative benefits of ePortfolios? New Zealand: Ministry of Education. Regis University. 2003. Regis University Electronic Portfolio Project [Online]. Available from: http://academic.regis.edu/laap/eportfolio/basics_types.htm [Accessed 19 January 2017]

Roberts, G., Aalderink, W., Cook, J., Feijen, M., Harvey, J., Lee, S. and Wade, V.P. 2005. Reflective learning, future thinking: digital repositories, e-portfolios, informal learning and ubiquitous computing. Research Seminar at ALT Spring Conference [Online]. Available from: http://www.immagic.com/eLibrary/ARCHIVES/GENERAL/ALT_UK/A051118R.pdf [Accessed 23 April 2017]. Schleicher, A. 2016. Teaching Excellence through Professional Learning and Policy Reform: Lessons from around the World. OECD Publishing: Paris. Seidman, I., 2012. Interviewing as qualitative research: a guide for researchers in education and the social sciences (4th ed.). New York: Teachers College Press. Sleegers, P., Bolhuis, S., and Geijsel, F. 2005. School improvement within a knowledge economy: fostering professional learning from a multidimensional perspective IN: N. Bascia, A. Cumming, A. Datnow, K. Leithwood, and D. Livingstone (ed.), International handbook of educational policy, Dordrecht: Springer, pp. 527-543. Snow-Renner, R. and Lauer, P. 2005. Professional development analysis. Mid-Content Research for Education and Learning: Denver, CO. Stefani, L., Mason, R. and Pegler, C. 2007. The Educational Potential of E-Portfolios: Supporting Personal Development and Reflective Learning. Routledge: Oxon.

Strudler, N., and Wetzel, K. 2005. The diffusion of Electronic Portfolios in Teacher Education: Issues of Initiation and Implementation. Journal of Research on Technology in Education, 37(4), pp.411-433. Swennen, A. 2013. The Power of Policy-Makers to Make the Right – or Wrong – Choices for Professional Development of Teachers. Professional Development in Education, 39 (3), pp. 289–92.


BIBLIOGRAPHY

Teaching Council. 2016. Code of Professional Conduct for Teachers (2nd Ed). Kildare: Teaching Council. Theodosiadou, D. and Konstantinidis A. 2015. Introducing E-Portfolio use to Primary School Pupils: Response, Benefits, and Challenges. Journal of Information Technology Education: Innovations in Practice, 14(1), pp.17-38. Timperley, H. et al. 2007. Teacher professional development: Best evidence synthesis iteration. Wellington: Ministry of Education, New Zealand. Available from: http://educationcounts.edcentre.govt.nz/goto/BES [Accessed 12 November 2017]. Wayne, A.J., Yoon, K.S., Zhu, P., Cronen, S., and Garet, M.S. 2008. Experimenting with teacher professional development: Motives and methods. Educational Researcher. 37(8): pp. 469-479. Wilson, S. M., and Berne, J. 1999. Teacher learning and the acquisition of professional knowledge: An examination of research on contemporary professional development. Review of Research in Education, (24), pp. 173–209. Wolf, K. and Siu-Runyan, Y. 1996. Portfolios, Purposes, and Possibilities. Journal of Adolescent and Adult Literacy, 40(1), pp.30-37.

Yastibas, A.E., and Cepik, S. 2015. Teachers’ attitudes toward the use of e-portfolios in speaking classes in English language teaching and learning. Procedia - Social and Behavioral Sciences, 176(1), pp. 514-525. Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., and Shapley, K. 2007. Reviewing the evidence on how teacher professional development affects student achievement. Washington DC: Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Young, D. and Lipczynski, K. 2007. Transferability of ePortfolios in Education: Phase One Report. UK: ESCalate. Youngs, H. and Piggot-Irvine, E. 2012. The application of a multiphase triangulation approach to mixed methods. the research of an aspiring school principal development program. Journal of Mixed Methods Research, 6(3), pp. 184-198.


BIBLIOGRAPHY

