provides school, district, county, and state reports addressing students, staffing, ...... magnet high school, one continuation high school, and a developmental center for ...... Assistant Support System, was developed jointly with the union.
HOW DISTRICTS PREPARE SITE ADMINISTRATORS FOR DATA-DRIVEN DECISION MAKING
by Miriam Deborah Ezzani
A Dissertation Presented to the FACULTY OF THE ROSSIER SCHOOL OF EDUCATION UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF EDUCATION
May 2009
Copyright 2009
Miriam Deborah Ezzani
UMI Number: 3355411
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion.
______________________________________________________________ UMI Microform 3355411 Copyright 2009 by ProQuest LLC All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code.
_______________________________________________________________ ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, MI 48106-1346
ii DEDICATION To Baba, Mohamed Ezzani, for his unrelenting faith and stamina and Mama, Dabia Ezzani, for her enduring love and passion, all of which are the cornerstone of my spirit and will to continually pursue knowledge for the purpose of serving others. To my children, Jamila and Ali, who taught me patience and acceptance -- patience as I gauge my work with the rhythm of the world and acceptance of that which I am unable to control. To my sister Huda, brother-in-law Khalid, nieces and nephew, Heba, Sophia, and Yousef who are all a source of inspiration in that they are all in my sphere of influence. To Hasmik Danielian, an extraordinary human being, who has become my soul sister as a result of this once in a lifetime endeavor.
iii ACKNOWLEDGMENTS I would like to acknowledge the district administrators and principals at Buck and D’Angelo Unified School Districts for their openness during interviews and observations to evaluate professional development for school site administrators. Certainly, I am grateful to the Superintendents for their ability to value, appreciate, and concede to this research being done in their districts. I am truly appreciative of the support from the Rossier School of Education faculty at the University of Southern California. I am also grateful to my dissertation committee: Dr. Edmond Heatley, fellow Trojan, Superintendent of Chino Valley USD, and mentor, who providing the administrative opportunity which enhanced my professional and educational experiences; Dr. Kathy Stowe, the Executive Director of the Doctor of Education program for her generous sharing of knowledge, support, and feedback regarding my ability to lead; and, my utmost appreciation to my Committee Chair, Dr. Amanda Datnow, for her unrelenting commitment, impeccable knowledge in her craft, and timely support and feedback from the first course in my program to the end of this dissertation.
iv TABLE OF CONTENTS DEDICATION ACKNOWLEDGEMENTS LIST OF TABLES ABSTRACT
ii iii vi vii
CHAPTER ONE: Overview of Study Introduction Professional Development Research Questions Significance of the Study
1 1 1 3 10 11
CHAPTER TWO: Literature Review Introduction Accountability in K-12 Education The Role of Data-Driven Decision Making in K-12 Education Reform Professional Development to Build Capacity of School Leaders Summary of the Literature Review Figure 1: Data-Informed Leadership
13 13 13 14
CHAPTER THREE: Methodology Research Design Sample Overview of the Districts Buck Unified School District D’Angelo Unified School District Data Collection Procedures Data Analysis Ethical Considerations Limitations of the Study Summary
48 48 48 51 53 53 54 56 59 62 63 64
20 26 43 46
v CHAPTER FOUR: Data Analysis and Data Interpretation Introduction Framework Need for Professional Development Theme 1: Methods of Needs Assessment Theme 2: Leadership Capacity Building Training and Support for School Site Leaders Theme 3: Accessibility to Tools Theme 4: Process of Data-Driven Decision Making Evidence that Leaders Have Put New Practices into Place Theme 5: Effectiveness of Professional Development Theme 6: Culture of Data Use Existing Knowledge Gaps Theme 7: District Expectations Conclusion
65 65 65 67 69 70 85 99 99 104 117 117 131 143 145 154
CHAPTER FIVE: Summary and Implications of Findings Introduction Connections to Prior Research Accountability in K-12 Education The Role of DDDM in K-12 Educational Reform Professional Development to Build Capacity of School Site Leaders in DDDM Implications for Future Research Implications for Policy and Practice Mastery of Leadership Strategies in Data Informed Leadership Conclusion
159 159 159 162 162 166
182 186
REFERENCES
187
APPENDIX A: APPENDIX B: APPENDIX C: APPENDIX D: APPENDIX E:
District Administrator Interview Protocol Administrator Interview Protocol Observation Protocol Preliminary Codes Salient Codes
172 180 182
192 195 198 199 200
vi LIST OF TABLES Table 1:
Interview Participants in Buck and D’Angelo USDs
57
Table 2:
Organizational Process for Coding, Themes, and Integration with Sub-questions
61
Table 3:
Organizational Chart for Integration of Sub-questions, Themes, and Elements of Data-Informed Leadership 62
Table 4:
California Department of Education, Accountability Progress Report for Buck Unified School District (2008)
84
California Department of Education, Accountability Progress Report for D’Angelo Unified School District (2008)
84
Table 6:
Accountability Progress Report for Buck and D’Angelo USD
145
Table 7:
Effective Professional Development Practices in DDDM for School Administrators
179
Mastery of Leadership Strategies in Data Informed Leadership
184
Table 5:
Table 8:
vii ABSTRACT The federal government’s No Child Left Behind Act and California’s Public School Accountability Act have mandated California to establish a system of accountability for their districts. Thus, California’s districts have understood the need for data-driven decision making (DDDM) to facilitate the monitoring and improvement of teaching and learning practices to increase student achievement. The intent of this qualitative study is to investigate the types of professional development that districts offer school leaders to increase capacity in DDDM. Five research questions were formulated to guide this study: (1) How do districts build the capacity of school leaders to engage in data-driven decision making? (2) How is the need for professional development assessed? (3) What types of training and support are provided to school site leaders? (4) What evidence is there that leaders put new data-driven practices into place? (5) What knowledge gaps still exist? Two districts were selected based on their focus to provide principal professional development in DDDM. The districts attributed some of their increase in student achievement to the utilization of data. Interviews of superintendents, district administrators, and principals were the primary source of information. Observations and document analysis were also utilized. The research was conducted through the lens of Data-Informed Leadership, a framework by Knapp et al. (2006). Professional development for principals at Buck and D’Angelo Unified School Districts lent to models that allowed for breadth and depth. According to
viii participants, principal professional development at these districts directly improved classroom instruction and student outcomes. The superintendents in both districts possessed strategies which made a difference in the preparation of principals in DDDM and in practices that gave way to Data-Informed Leadership. It was the mastery of leadership strategies that moved their organizations into a culture and cycle of inquiry based on data. While this study provides relevant information and a roadmap into the process by which districts provide school leaders with knowledge in DDDM, further research is necessary in the mastery of leadership strategies in Data-Informed Leadership. Mastery of leadership strategies will ensure an established culture and cycle of inquiry in the use of data.
1 CHAPTER ONE Overview of the Study If education is always to be conceived along the same antiquated lines of a mere transmission of knowledge, there is little to be hoped from it in the bettering of man's future. For what is the use of transmitting knowledge if the individual's total development lags behind? -- Maria Montessori (1870-1952) Introduction Background of the Problem The State of California, in response to a growing concern in the quality of education, passed the Public School Accountability Act (PSAA) in April 1999 – three years prior to the implementation of the federal government’s No Child Left Behind (NCLB) Act of 2001. California’s PSAA is a set of programs to assist the state, local education agencies (LEAs) or districts, school leaders, and the public to monitor and evaluate the growth of California public schools (Ed Code, 5205052050.5). The Education Code states, . . . pupils in California are not now, generally, progressing at a satisfactory rate to achieve a high quality education. To remedy this, the state is in need of . . . comprehensive accountability system to hold each of the state's public schools accountable for the academic progress and achievement of its pupils . . . (52050.5, b, c, and d) California later modified its accountability system in August 2003 in order to meet NCLB funding requirements, which was put into place in January 2002 to assure that all students by 2013-2014 can read, write, and perform mathematics in order to graduate from high school (U.S. Department of Education, n.d.). NCLB is
2 founded on four pillars: (1) stronger accountability for results, (2) more freedom for states and communities, (3) proven education methods and, (4) more choices for parents. Both NCLB and PSAA accountability systems are all-inclusive efforts that hold all stakeholders accountable for raising the level of education of the nation’s children and this requires “. . . support to devise an acceptable way to balance public, academic, and market concerns” (Burke, 2004, p. 24). At the local level are instructional leaders that set the pace of accountability at their school sites. But do the instructional leaders possess the knowledge and skills to implement, monitor, and evaluate an accountability system that produces student achievement? Have states and districts provided adequate professional development to school site leaders in order to meet the accountability demands as stated above? As a consequence of NCLB, district and school site leaders are expected to have the knowledge and skills to recognize and implement all the modalities necessary to meet the pressures of performance-based accountability to increase the quality of public education for all students. Moreover, it has become crucial for school site leaders to be proficient in data use and analysis to gauge academic progress (Elmore, 2002).
3 Professional Development Numerous scholars suggest that capacity must be built at the school site for data-driven decision-making to be an integral part of the organization (Datnow, Park & Wohlstetter, 2007; Knapp, Swinnerton & Copland, 2006; DuFour, DuFour, Eaker & Many, 2006). This, however, requires professional development and ongoing training of school site administrators. Elmore (2002) states, “Accountability must be a reciprocal process. For every increment of performance I demand from you, I have an equal responsibility to provide you with the capacity to meet that expectation” (p. 5). The State of California, in an attempt to build capacity in school site administrators to meet the federal and state accountability requirements, offers The Administrative Training Program (AB 430), authorized by Assembly Bill 430, and previously known as AB 75. AB 430 was introduced through the California legislature in 2005-2006. AB 430 provides funding to districts to train principals and assistant principals in the knowledge and skills needed for their convoluted roles as instructional leaders. This two-year program is divided into three modules and can only be delivered by a provider that is state-approved. This is the only state of California initiative that focuses on principal professional development. It consists of: Module 1, Leadership and Support - Student Instructional Program (five days), which reviews the core academic standards, curriculum frameworks and instructional materials aligned to the state standards, use of assessments, ways to master use of
4 Standardized Testing and Reporting (STAR) assessment data, and school management technology to improve student performance. Module 2, Leadership and Management for Instructional Improvement (two days), which includes school financial and personnel management, and knowledge and skills necessary to serve all students. Lastly, Module 3, Instructional Technology to Improve Pupil Performance (three days), which consists of strategies for instructional leadership and management utilizing instructional technology to improve pupil achievement. To fund this program, each district pays $1000 and receives $3000 from the State of California for each participant. An estimated 10,000 administrators participated during the first five years. The crux of this training is a didactic ‘one-size-fits-all’ training, which familiarizes administrators with curriculum and instruction as it relates to state-adopted texts and standards (CDE, 2006). The effects of organizational leadership that are not addressed in the Principal Training program are highlighted in an in-depth study by Policy Analysis for California Education (PACE), which investigates how California principals acquire and organize resources in comparison to other states (Fuller, 2006). It reveals that principals spend a considerable amount of time interacting with parents, attending to student discipline, interfacing with the district office, working on compliance requirements, and attending to teacher concerns. Less time is spent evaluating and/or coaching teachers, assisting teachers in aligning pedagogy, and attending to student work or classroom performance. An overall comparison of California
5 principals with principals of other states reveals that they spend more time on activities that are not related to instruction. Even though California principals reported that student achievement data are a valuable resource, only one–third reported using this information at least once a month, and rarely do they use this information to assess teachers’ strengths and weaknesses (Fuller et al., 2006). Responses to the PACE survey are indicative of the didactic type training California K-12 principals have received through the state and their districts. When California principals were surveyed regarding their prime educational goals as well as that of their superintendent, they ranked basic literacy skills among the top; although, they believed they were less focused on test preparation in relation to their superintendent. Four out of five principals surveyed also reported that they have shifted instructional time to place more emphasis on reading and language arts; twofifths and three-fifths of elementary and high school principals, respectively, have increased the time spent on test prep. Principals also espoused concerns regarding teacher dismissal, stating that barriers are greater than their peers in other states regarding firing ineffective teachers, with principals in Title I schools reporting lower levels of satisfaction in teacher quality. However, when answering the question regarding the number of teachers they would like to dismiss from their schools, 75% stated they would dismiss two or less. Although some of the concerns mentioned above are valid with regard to priorities and tools needed for standardbased reforms, only a small percentage of K-12 principals, according to the PACE
6 survey, appear to be utilizing data to drive decision-making to improve student achievement (Fuller et al., 2006). With respect to new principals, the accountability and compliance toward meeting the Interstate School Leadership Licensure Consortium Standards by university preparation programs has been hampered by the elimination of the accountability piece due to budget constraints. Accreditation visitations were discontinued and there is little oversight of program to guarantee standards and program alignment (Darling-Hammond & Orphanos, 2006). Training provided by university preparation and state programs is thought to be theoretical and unrelated to the everyday demands that school site administrators face. Furthermore, learning is arduous in that information received is ill-sequenced, thus making scaffolded learning extremely difficult (Hale & Moorman, 2003). So broad is the concern that the National Commission for the Advancement of Educational Leadership Preparation (NCAELP) was formed in 2001. This organization is composed of prominent scholars and leaders in the area of educational leadership. Their task is to evaluate and improve the caliber of educational leadership in the United States. There are some school districts, however, that have taken vital steps to establish system-wide plans for the improvement of instruction by: (1) public acknowledgement of unacceptable student achievement, (2) taking responsibility for the crisis, (3) stating that the solution will be the responsibility of all stakeholders, and (4) making long term commitments and providing novel supports (Togneri &
7 Anderson, 2003). Case studies of districts focused on reform, such as New York City District #2, found the “cognitive apprenticeship” professional development approach to include peer-learning and formal mentoring, study groups, support groups to build leadership skills, and monthly one day conferences and institutes, visitation to other school sites, and one-on-one coaching from the district office, which includes walkthroughs of school sites. The combination of all of the above was said to be more valuable than any other training these principals had experienced (Mac Iver & Farley, 2003, pp. 16-17). One foundational piece of professional development is that school site administrators accept and believe that data use is necessary and should be a part of the school culture (Mac Iver & Farley, 2003). Although research shows that district leaders accept and promote evidence-based school improvement, it is also known that obstacles such as lack of technology, lack of access to the right data, as well as engrained cultural norms about teaching and learning continues to exist at school sites (Colbert & Talbert, 2006; Ingram et al., 2004; Supovitz & Klein, 2003; Young, 2006). There is, however, a developing body of research that shows that school leadership is a needed element for successful implementation of evidence based decision making. Furthermore, those school site leaders who are successful possess the knowledge and skills, the commitment, and moreover the vision that is necessary (Kerr et al., 2006).
8 According to conclusions in the study of Mac Iver and Farley (2003), important content for capacity building in school leaders, by their school districts, are: advising on curriculum and instruction; recruiting and providing effective principals and teachers; assisting in the analysis and evaluation of data for the purpose of instructional changes; and, lastly providing support to administrators so that data-driven decision making dictates instructional needs. The accountability piece, or the hallmark, that school site leaders are using data-driven decision making to lead their schools in student achievement, is the Single School Plan for Student Achievement (SSP). This document is an annual requirement that school leaders formulate in conjunction with their School Site Council (composed of administration, teachers, parents, and community) and submit to their school districts. The SSP links resources to learning, in other words, goals are established based on student achievement data, then a budget which is directly linked to the goals is formulated to support student achievement and ultimately increase a school’s API. Four precise leadership domains were found to produce a higher Academic Performance Index (API), and they are as follows: making student achievement a priority, implementation of a coherent standards-based curriculum, utilization of assessment data, and ensuring availability of instructional resources (EdSource, et al 2007). A principal’s behavior in terms of making student achievement a priority is espoused through a clear vision in setting high expectations for teaching and learning. As a consequence, implementation of a coherent
9 standards-based curriculum is essential. The school leader must assure that there is curriculum coherence through school-wide consistency, both horizontally and vertically. Assessments must be timely and should correlate with curriculum and state standards. A district that supports their school sites allots resources to support principals in site level planning, and site level planning is constructed through the SSP (CDE, 2006). Formulating a SSP requires specific knowledge and skills by the school leader, namely: develop avenues to collect and interpret data in an effort to meet NCLB’s requirements, which includes meeting Academic Yearly Progress (AYP). This complex process requires disaggregating data, for the purpose of determining that all subgroups are meeting AYP. In addition to gathering and analyzing data, and developing an action plan for implementation and monitoring, school leaders must also be adept in gleaning the bigger picture. This means knowing the current data and statistics that are available through the state website and being aware of future developments. The State of California publishes data and statistics collected from schools in California so that trends and needs can be measured and identified. The state has a number of items under Data and Statistics, specifically: (1) DataQuest which provides API, AYP, test data, enrollment, graduates, dropouts, course enrollment, staffing, and data on English Language Learners; (2) Ed-Data, which provides school, district, county, and state reports addressing students, staffing, finances, and performance ranking; (3) pending (2009-2010) will be the Longitudinal
10 Education Data Systems which will allow maintenance of longitudinal data based on student demographics, program participation, enrollment, discipline, state assessment results, teacher assignment, and other data necessary to meet state and federal reporting requirements (CDE, 2007). With the growing demands and requirements of the federal and state governments, school districts and school site leaders are expected to have the knowledge and skills to recognize and implement all the modalities necessary to meet the pressures of performance-based accountability to increase the quality of public education for all students. Moreover, it has become crucial for school site leaders to be proficient in data use and analysis to gauge academic progress (Elmore, 2002). Although research efforts have been made in the area of professional development for school site administrators in data-driven decision making, the empirical research is less than exhaustive. Hence, the aim of this research study is to enhance the existing body of knowledge, with a particular focus on promising practices of district provided professional development to school site administrators in data-driven decision making.
Research Questions The proposed qualitative multiple case study endeavors to extend the understanding of effective district professional development in data driven decision making for school site administrators. The study will focus on two districts that are
11 using promising practices in professional development for their school site administrators in evidence-based instructional improvement. This qualitative study will speak to the following overarching question: How do districts build the capacity of school leaders to engage in datadriven decision making? In conducting the study, the below-noted sub-questions will also be addressed: a) How is the need for professional development assessed? b) What types of training and support are provided school site leaders? c) What evidence is there that leaders put new data-driven practices into place? d) What knowledge gaps still exist?
Significance of the Study Professional development of school site administrators in school-wide datadriven decision making, couched in the current federal and state accountability systems, is a new area of study; therefore, research in this area is yet to be fully developed into an exhaustive body of research. This qualitative study proposes to capture and contribute to this new body of literature, in an effort to assist districts in providing their school site administrators with appropriate and effective professional development. The intent is to offer rituals and routines in effective professional
12 development that is not necessarily a mere transmission of knowledge but a base of knowledge and skills that can be implemented to establish and sustain schools to incorporate evidence based practices to improve student performance. It also may be used to provide districts with a road map to support their school site leaders to confidently acquire the knowledge and skills to implement an evidence-based culture at their school sites.
13 CHAPTER TWO Literature Review There's nothing wrong with setting goals, but it doesn't mean a thing if you don't pay attention to the day-to-day details. --Don Shula, Coach, Miami Dolphins Introduction It is certain that the establishment of NCLB and PSAA dictates that an accountability system is clearly in place to improve student achievement and to hold those within the system, states, districts, and schools, to certain expectations. The federal government and the State of California’s publication of Adequate Yearly Progress (AYP) scores and Academic Performance Index (API) have induced educators to focus and pay attention to data (O’Day, 2004). As districts establish a system of accountability for their schools, the building or establishment of social capital amongst the site administrators and teachers is necessary to form a cohesive system (Spillane & Thompson, 1997). In essence, this translates into all schools within the system believing that their work is geared toward incremental gains from year to year, and that evaluation and monitoring is a matter of policy and procedures. At the core of the accountability system are data and the ability of individuals within the system to use them. With the district leaders at the helm and the school site administrators as the leaders at the school sites, it is incumbent upon the district to assure that the school site administrators have the knowledge and skills in datadriven decision making (DDDM) to meet the accountability expectations. The
14 essence of this study is to examine promising practices in professional development for school site administrators in the area of DDDM. Thus, an exploration of issues related to DDDM professional development for school site administrators will be addressed in the following literature review. Three crucial areas will be discussed: 1. An examination of accountability in K-12 education as defined by United States and California policy and the implication for data use at the district and school levels. 2. An evaluation of the role of data-driven decision making in K-12 educational reform. 3. An investigation of professional development to build the capacity of school site administrators in DDDM.
Accountability in K-12 Education Historically, the nation’s public schools have been held accountable for complying with federal and state regulations as well as financial management; however, around the year 2000 the United States government reframed accountability for public schools. In 1999, some states as well as the State of California began holding schools accountable for student performance by way of scores on standardized tests. It is important to note that the term “accountability” falls under the umbrella of “standards-based” reform, wherein each state is required to develop standards for each grade level that delineate what a student should know
15 and be able to do. Hence, in a standards-based system, the following policies are set: each state must have standards for each grade level in the various content areas, assessments should be developed to measure mastery of standards at year’s end, and rewards or sanctions based on success or failure are meted out. For the first time in the history of the United States, the federal government has outlined specific guidelines for states to follow in formulating their own accountability systems. States, however, vary in the implementation of the aforementioned reforms via their state testing, textbook adoptions, and professional development (EdSource, 2007). Federal Accountability System (ESEA & NCLB) The current system, NCLB, implemented in 2002, is the most recent version of the Elementary and Secondary Education Act (ESEA) of 1965. It requires all states that accept ESEA funding to put into place an accountability system that holds states and their respective districts and schools responsible for the academic performance of their students. The federal government expects each state to demonstrate adequate yearly progress with upward movement in order to meet the goal of proficiency for all students by the year 2013-2014. The aforementioned goals of NCLB are quite similar to the accountability system that California established in 1999; however, NCLB required that the State of California make more changes to its system and, subsequently, revisions were made in 2003. The consistencies between NCLB and California’s PSAA are that student performance is based on standardized test results, which includes specific line items
16 that show performance of numerically significant subgroups. Another commonality is that interventions are mandated to help those schools that are low-performing or program improvement (PI) schools. The federal government, however, is raising the bar for 2008. From the inception of NCLB in 2002, targets have been leveled twice. In 2008 and for each year thereafter until 2014, there will be a target increase in AYP goals. It is assumed that states and their districts have had ample opportunity to raise the capacity at the state, district, and school levels (CDE, 2007). California’s Accountability System As noted earlier, prior to NCLB, many states including California implemented their own performance-based accountability system. In 1999, the legislature in California approved the Public Schools Accountability Act (PSAA), which included three areas designed to hold schools accountable for improving student learning. Shortly thereafter, in June 2003, American Institute for Research (AIR) completed the first legislatively mandated study of the PSAA, with support from Policy Analysis for California Education (PACE) and EdSource. In 2004, the California Department of Education (CDE) contracted AIR to pursue a continuation study to resolve differences between the state and federal accountability systems (AIR, n.d.). Some of the respondents in the survey stated that having two accountability systems (NCLB and PSAA) was overwhelming district and school staff. Furthermore, it was generally stated that AYP linked to NCLB was a more challenging target (requiring schools to meet proficiency targets) versus California’s
17 API target wherein the focus is on growth. The report revealed that factors hindering school improvement were: “. . . the district role in school improvement, the influence of instructional coherence, and internal school capacity” (AIR, p. I-5). Other noteworthy findings in the study were: . . . differences emerged between the data use practices of growth schools and schools that made low levels of growth (including two schools that were also SAIT schools) . . . main difference . . . was the extent of data use to inform instruction. Growth schools reported using data frequently and extensively to guide instruction, while the other schools, more often than not, reported using it to generally identify students not performing well (p. IV-13). While it is true that districts must adhere to federal and state expectations, a focus on the interdependence between district and school sites’ is crucial for growth within a school system to occur. Hence, the onus to create a comprehensive school system falls on the district to build the capacity of those involved in DDDM (Leithwood, Louis, Anderson & Wahlstrom, 2004). Implications of Data Use at District and School Level Data have become the avenue by which accountability is ensured on the federal, state, and district level. Accountability is multi-tiered with various assessment systems, establishment of benchmarks of effectiveness and targets, and systems for monitoring and evaluating (Earl & Katz, 2002). Furthermore, numerous studies have noted significant factors eliciting growth in student achievement, as evidenced in studies that identified schools that have made consistent or high growth,
18 which include but are not limited to: capacity at the school level, instructional consistency, regular assessment and data-driven decision making (AIR n.d.; Coburn & Talbert, 2006; Mandinach, Honey & Light, 2006; Marsh, Pane, & Hamilton, 2006; Knapp, Swinnerton, Copland & Monpas-Huber, 2006). In 2004, the RAND Corporation began a study of data-driven decision making in the Southwestern Pennsylvania school districts. Their purpose was to study the districts in this particular area for challenges in data use in an effort to improve school performance to meet state and federal requirements. This study looked at the steps taken by the districts to develop capacity for DDDM for the purpose of identifying the process of improving capacity in developing its use. The study states that DDDM is not always successful, and key factors toward success are: powerful leadership, sufficient resources, and planning for data collection and its utilization. An emphasis is placed on the finding that empirical studies on DDDM reveals school leadership is a required element for implementation to be eventful in districts (Marsh, Pane, & Hamilton, 2006). RAND researchers noted that specific types of data are preferred. For example, administrators find summative tests as great tools for school improvement, whereas teachers believe classroom assessments are more useful for planning instruction than are summative assessments. Principals noted that data from state testing assisted in identifying areas of improvement. Moreover, due to high stakes accountability, test data are also used to identify “bubble kids,” those students who
19 are shy of reaching proficient on state testing (Marsh et al, 2006). The RAND work proposes that the majority of educators believe data are useful for informing improvements in teaching and learning. It is further suggested that DDDM does not lead to effective decision making; however, the following is suggested: •
promotion of various data at various times
•
triangulation of data to find a balance
•
analysis of data and actions based on the same
•
provide training on data analysis, identifying, and implementing solutions
•
allocation of time
•
partner with institutions that support data use
•
assigning people to filter data
•
user-friendly technology and data system
Lastly, a word of caution is made about the negative consequences of high-stakes state testing and data use, i.e. using data analysis to target bubble kids who are near proficient (Marsh et al, 2006). It is noteworthy at this point to state that a school district’s espoused priorities and emphasis play a huge role in the implications and role of data use at the district and school level. It has also been emphasized in the majority of the above studies that knowledge and training of leadership is important.
20 The Role of Data-Driven Decision Making in K-12 Educational Reform Data-driven decision making (DDDM) in education as defined by Marsh et al, 2006) is: “. . . teachers, principals, and administrators systematically collecting and analyzing various types of data, including input, process, outcome and satisfaction data, to guide a range of decisions to help improve the success of students and schools” (p.1). The inclusion of DDDM in the current K-12 education reform is a result of previously failed educational reforms (Fullan, 2000). However, Fullan (2002) also states, “Only principals who are equipped to handle a complex, rapidly changing environment can implement the reforms that lead to sustained improvement in student achievement (p.1).” One might then ask about the various perspectives that have hampered the process of reform. Data-driven decision making (DDDM) can be an integral part of district and school accountability, though it is also seen by many as simply the “latest fad.” It refers to policies and practices regarding student achievement and other data (such as demographics, attendance, and graduation rate) that guide school improvement at the state, district, and school levels. Studies have shown that DDDM is seen as having substantial potential to advance efforts in order to improve learning outcomes (Datnow et al, 2007; Darling-Hammond & Orphanos, 2006; Elmore, 2002). The purpose behind DDDM is that analysis of student data will enable schools, districts, and states to target areas where progress is needed. Furthermore, DDDM serves as a gauge to evaluate and monitor success or failure of the action plans put into place to
21 improve a given system whether at the district or school level (Elmore & Rothman, 1999). Datnow et al (2007), in a study of leaders in the schools and systems, argued that DDDM should be a norm via “. . . stating explicitly that data use is nonnegotiable” (p. 71). From an organizational perspective, the superintendent, as the head of the organization, can promote a culture of data use through professional learning communities. Such data discussions at the administrative level build capacity for school site administrators to continue the process at their respective school sites. This may entail school leaders forming distributive leadership at their school sites and providing the foundation for collaborative data discussions where grade level chairs lead their grade level in activities. Such activities include conducting common assessments and analyzing data in order to formulate action plans that allow for adjustments in instruction (Schmoker, 2006). “A principal who is data-driven or technically savvy can exert substantial influence on the faculty, communicating the importance and thereby stimulating use (Mandinach et al, 2006, p. 13).” A principal’s capacity to implement reforms, however, is a direct result of district bestpractices in building the capacity of its school site administrators. Best practices to build capacity within a school district include, but are not limited to, the implementation and training in data tools for effective DDDM curriculum and instructional practice, the effective recruitment practices of principals and teachers, supporting schools in analyzing data and developing action plans through the
22 offering of administrative support. One must also be cognizant of the fact that the use of DDDM is a tiered process to be used at the district, school, and classroom levels (Mac Iver & Farley, 2003). Consistency at all levels is necessary for the development of a comprehensive school system in DDDM. Some of the barriers to establishing a comprehensive system are espoused by Coburn and Talbert (2006). In a two-year study wherein research data were collected from a large K-12 urban school district, it was found that individuals respond differently to NCLB and PSAA policy pressures. The response is fashioned by beliefs related to evidence and research as well as previous reform policies. Furthermore, responses to accountability policy differ at the various levels of a school system creating different views among district leaders, school administrators, and teachers. In other words, the issue at hand is the coexistence of conflicting or different views at the various levels that lend to or hamper the use of data to make decisions that will ultimately improve the quality of education for students (Coburn & Talbert, 2006). A symbiotic relationship between the various levels within the system is discussed in a study, entitled How Leadership Influences Student Learning (Leithwood et al, 2004). It is stated that a comprehensive and coherent educational system is possible via: establishing directions by charting a course that is understood by all; establishing high expectations and utilizing data to track performance and progress; developing people in the system to succeed through support and training;
23 and, ensuring that all conditions and incentives throughout the system support rather than inhibit teaching and learning. Well-informed school site leadership in data use can be powerful. Supovitz and Klein (2003) examined how innovative schools systematically used student performance data to guide school improvement. Upon doing so, they discovered that discussions on student outcome centers and connects to issues of curriculum, teaching, equity, professional development, school organization, use of time, and role of parents/community. When deciphering what influences student gains, one must look at all the indicators. Furthermore, it is up to the instructional leader to bring forth the totality of the organization through the espoused vision. With the school at the heart of instructional improvement, it is safe to say that principals are those educational leaders who make the difference; however, it is not without distributive leadership within the school, such as lead teachers, coaches, assistant principals and department chairs that this effort can come to the fore (DuFour, 2007). Supovitz and Klein, however, conclude that the American system of education does not allow for the structures and avenues to engage in the in-depth inquiry necessary to improve student learning outcomes. In another study related to leadership and building support structures conducted by Springboard Schools (2007), entitled Bringing the State and Locals Together: Developing Effective Data Systems in California School Districts the following policy were noted: data are crucial to improving teaching and learning;
24 the State of California’s data system is fragmented; and, the state does not provide appropriate support and resources for its school districts to develop comprehensive data systems. Specific recommendations for California include, but are not limited to: •
Invest the resources to develop an integrated state data system.
•
Provide funding for districts to develop or maintain local data systems.
•
Develop web-based guidelines for best practices, and provide liaisons to work with clusters of individual school districts. Springboard made some of the following recommendations for districts:
•
Monitor progress in data-based improvement goals through appraisal of key district leaders and principals.
•
Set goals for development of district data system by including all stakeholders.
•
Include in district’s data plan - linking data from student information system to assessment data.
•
Invest in resources to attain technological capacity to collect, organize, analyze and report data.
•
Encourage data accessibility, ownership, and ease of use at multiple levels of the system (district, school leader, and teacher level).
•
Invest in human resources (teachers and administrators) who can help in developing capacity of effective data use.
25 In summation, the Springboard report stresses the importance in investing in human resources to build capacity and that a high quality data system is necessary to support school improvement. Hence, a recommendation is made for policymakers, at the local and state levels, to invest in policies that help to build such infrastructures (Oberman, Hollis & Dailey, 2007). With student learning as the pinnacle of school improvement, the American system, thru states and policymakers, have responded by targeting efforts that have an exclusive connection to student learning, such as: the recruitment, training, and credentialing of teachers; curriculum and instruction aligned to standards and textbooks; class size reduction; and, of course, testing. Consequently, these efforts have triggered a demand by the state and district for principals to implement as well as monitor the above without considering the capacity of principals’ knowledge and skills to manage such reforms. The role of the principal in these regards could become quite daunting and was essentially overlooked in the 80s and 90s (DarlingHammond & Orphanos, 2006). Based on the aforementioned, school site administrators are the liaisons between the district and schools in forming a comprehensive school system. Vital to this study is an exploration of issues related to professional development, by the school districts, for their school site administrators. In the following sections I will review literature on professional development for school site administrators based on the need for them to have a firm understanding of their school’s data, be able to
26 analyze them, and then reveal the evidence and produce the action plan necessary for school improvement. This includes the school site administrator’s ability to create a school culture and increase the capacity of teachers to use data in making decisions about learning and instruction.
Professional Development to Build Capacity of School Leaders We are on a long journey, and the trip has just begun. --Charles Savage, Leading Thinker The impetus of the NCLB and PSAA accountability systems has placed new demands on school site administrators. In essence they must lead organizations that effectively teach a spectrum of students (numerically significant subgroups), who have a variety of needs. Furthermore, they must simultaneously meet the targets set by federal and state government. Coburn, Honig, and Stein (in press) note that policy makers pressure districts to use evidence in decision making. They also note that in a number of studies, administrators have been noted to make most decisions without the use of data. In other words, data use has no role in decision making. In another study done by Corcoran, Fuhrman, and Belcher (2001) it was noted that none of the districts that were evaluated utilized data to make decisions about professional development. It should be noted that the routine of data use is not simply to analyze, interpret, and utilize data to formulate an action plan for the SSP, but to create a culture of data use at the school site (Datnow et al, 2007). Thus, the most crucial and
27 most difficult investment is the development of human capital, which is necessary for the development of data literacy and analysis skills. This requires school site administrators not to simply be managers but to be organizational leaders with highly developed skills to understand and fashion an organization and lead it in organizational change (Darling-Hammond & Orphanos, 2006). Opportunities to gain new knowledge, skills, and tools to develop the repertoire of a school site administrator’s effectiveness in instructional leadership are dependent on effective professional development. Butler (1992) states that effective professional development is dependent on three areas: (1) characteristics and needs of the learner, (2) characteristics of professional development vis-à-vis purpose, structure, content, process, and follow-up, and (3) characteristics of the organization in terms of support (p. 1). Furthermore, for professional development to be effective, six conditions must be met: •
Individuals must feel the need to learn.
•
Learner must be able to make a connection to past and present experiences to see the value of new learning.
•
Learning must relate to the individual’s stage of development.
•
Autonomy is calculated based on method or mode of new learning.
•
Climate minimizes anxiety and encourages experimental freedom.
•
Learning styles are considered.
28 In essence, school site administrators must be able to visualize the professional development as part and parcel of their professional life and not as a remediation with implications of inadequacy (Butler, 1992). School site administrators’ professional development, however, has for the most part taken the form of transmitting procedural knowledge which often times does not pertain to practical application. This type of professional development leaves recipients in a vulnerable position, one that insinuates dependence on external sources rather than empowerment and ownership (Schmoker, 2006). Schmoker goes on to state that professional development is “. . . often bad beyond hope” (p. 100). In a review of literature, wherein he summarizes several studies on school districts, he notes that successful districts are those which spend heavily in human, financial, and technological resources. The states’ funding priorities may affect districts’ resource allocations. The State Role in Professional Development Since the district is the major intermediary between the school and the state, the state must ensure that the district has the necessary resources (Blasé & Blasé, 2004). When considering the number of California schools that are underperforming, the state should provide the district the necessary support and financial wherewithal. This would enable districts to provide the knowledge and skills in state and benchmark assessment data use to determine the needs of students and ultimately to provide appropriate instruction. Resources should also be provided
29 to allow districts to supply its schools with the appropriate technology and userfriendly tools to decipher student results and specifically target areas of need. The state can also provide incentive structures for those low-performing schools that require experienced and motivated administrators, teachers, and coaches (O’Day, J., Bitter, C., Kirst, M., Carnoy, M., Woody, E., & Buttles, M., et al., 2004). The State of California has decided not to allocate all professional development resources to its districts. It has taken some responsibility for professional development of school site administrators by offering AB 430. It is the only state initiative focused at principal professional development with modules 1, 2, and 3 (a total of two weeks), but can be taken over a two year span. This didactic one-size fits all training costs the district $1000 and for each participant the district receives $3000. The bulk of the training acquaints administrators with curriculum and instruction as it relates to state-adopted texts and standards (CDE, 2006). Such learning is learning out of the environment where situations take place; hence, it is not the type of application learning that makes a true effect (Fullan, 2002). Also, given that the state is detached from the school’s situation, it is incumbent upon the state to directly provide monetary resources to the districts so they may provide its school site leaders with contextual organizational leadership knowledge and skills. In an attempt to study issues of leadership development in the State of California in relation to the nation, Darling-Hammond & Orphanos (2006), utilized data mostly from a national study that included a national survey of approximately
30 1000 principals, with an oversampling in eight of the states including the state of California. The study consisted of a review of policy documents, literature, and interviews of a wide array of stakeholders from policymakers to principals and superintendents, and individuals in professional associations, university preparation, and professional development programs. They also studied eight leadership development programs, which they deem exemplary, documenting their strategies and results. Findings revealed that in order for the State of California to reach its “ambitious aspirations for raising student achievement” that major systemic changes and investment in knowledge and skills must be made. Specifically the recommendations are: knowing the research-based practices in teaching and learning as well as clinical experience; appropriate allocation of resources -- time, human resource, and money; skill-building of leaders, teachers, and other staff through effective evaluation, feedback, and professional development; and, management of a successful school improvement process through the use of data (Darling-Hammond & Orphanos, 2006). Other principal professional development programs through universities, professional associations, not-for-profit organizations, and government agencies were analyzed for focus of program, purpose, curriculum, instructional strategies, coherence, and technology by Peterson (2002). Peterson states that at the very minimum a conceptual frame such as the state standards should serve as a template and a closer link should be made with preparation program in order to enhance
31 learning. Also the length and structure plays a critical role, wherein most professional development is a one-time workshop – all day and sequential meetings over a period of a year is a more productive approach. Many programs are described, but two in particular are reported to have a comprehensive approach with a welldesigned structure for principal professional development: (1) California School Leadership Academy (CSLA), a statewide program and, (2) CLASS, a program which serves Chicago principals. CSLA currently offers three primary programs: Foundation 3.0, School Leadership Team, and Ventures. The Ventures program deals more with experienced principals and focuses on skills and knowledge associated with transformational leadership. Through an ethnographic research method, principals study their own school setting for a three year period (Peterson, 2002). Through the direction of CSLA facilitators, principals document transformation in school culture and their roles as leader. Program design is in three phases: Phase 1, participants define their “theory of action” and their “field of action” and the focus of their change efforts. In Phase 2, participants begin implementing a transformation strategy in their schools based on their analysis of multiple sets of data. In Phase 3, participants present an exhibition of their achievement to colleagues. This includes a narrative analysis of the change process in their school and data that tracks (Peterson, 2002, p. 224). CLASS, the other program deemed comprehensive, is a joint effort between the Chicago Principals and Administrators Association (CPAA) and the Chicago
32 public schools. The three programs that are offered are sequential and provide breadth and depth. They are: (1) LAUNCH, for aspiring principals; (2) LIFT, for 1st-year principals; and, (3) Chicago Academy for School Leaders (CASL) for experienced principals and other administrators. The thrust of these programs are to assist principals to develop skills and knowledge that coincide with governmental reforms of the 1990s. Thus, it is standards-based and related to the ISLLC standards. CSLA and CLASS are comprehensive long term professional development, which allow for breadth and depth. These programs are unlike on-the-job in service trainings which are offered to administrators as a quick look at the requirements for the position. Also, on the job trainings do not offer ample time to learn everything about instructional leadership as it relates to DDDM and accountability. Only when school site administrators feel capable and secure in DDDM will they then ensure that this filters down to the classroom (Heritage & Yeagley, 2005). Districts along with professional educational organizations have been intermediaries for the state by collaborating to form long term professional development programs that would allow administrators to acquire the sustained learning necessary to conduct data-driven decision making. A smooth transition between the federal mandate of NCLB, the states, and the districts to effect significant change in the classroom has been a slow and arduous process. States have attempted, with great difficulty, to be the key players in this process. The districts,
33 however, must play a major role in the systemic change that would ultimately effect improvement in student outcome (Darling-Hammond & Orphanos, 2006). District’s Role in Professional Development If professional development is to have relevance, then it should be done tandem with activities that reflect instructional improvement and should not be detached from its organizational system – the district. Fullan (1990) confirms that the learning should not be in isolation, but rather to produce individual and organizational habits and structures which encourage learning on a continuum. The relationship between the district office and its schools with relationship to student achievement was the focus of a literature review by Mac Iver and Farley (2003). It was noted that professional development for teachers and principals is one of four important district roles. They found that there was consensus in the literature regarding professional development for principals and teachers in the interpretation of data in order to make sound instructional decisions. They further note that more longitudinal research is needed in the area of professional development strategies provided by the district to its school site administrators. The importance of the district role in developing principal capacity is also noted in an analysis of district funding applications by Glass (2002). In a descriptive analysis of what districts identified as specific needs, principal improvement was noted. Districts that included principal improvement as a need stated that professional development should be provided in the areas of: instructional strategies
34 (24%), team leadership (24%), data driven decision making (16%), and in alignment of curriculum (16%). Of note, however, is that districts must be aware of the need to build organizational culture simultaneously with professional development to assure a meeting of the minds. In order for district leaders to evaluate the culture of the organization, they must ask, Are school site administrators actions based on intrinsic reasons or because they must conform to external pressure? Furthermore, if districts expect data to be utilized in the improvement of service to students, then pressure for use must be accompanied by external mandates that are couched in a mind-shift that DDDM is part of systemic or organizational improvement that empowers educational leaders to have a collection of tools to plan and make decisions (Darling-Hammond, 2002). The immediate implications are that insights and developments are unveiled by the evidence of new knowledge via data, wherein it is shared, meaning is constructed, and school site leaders set the pace and cycle for data literacy, inquiry, and action. Also, an injection of reality is much needed – the reality that we live in an age of information technology and with this comes not only data but data that must be formed into knowledge (Earl & Katz, 2002). Creating this type of understanding requires skills of transformational leadership. Enhancing the quality of schools via school-based principal professional development is the focus of Leaders and Leadership by the National Staff Development Council (n.d). The following recommendations for district offices are:
35
•
Encourage principals to utilize distributive leadership and evaluate principals on their ability to establish a culture of collaboration;
•
Rethink approach for determination and development of principals, such as leadership certification through district academies that entail an academic base as well as superior learning experiences wherein districts and universities partner.
•
Apprenticeship model, such as The Aspiring Principals Program run by Dennis Littky of Rhode Island's Big Picture Company. Program trains aspiring principals through a one-year apprenticeship with a celebrated principal simultaneously with an academic component through a university.
•
Coaching to assist principals to remain focused on instructional goals while at the same time achieving balance in life.
•
Superintendents design and lead professional development that requires principals to spend more time in classrooms as well as demonstrate knowledge and skills needed to use with teachers.
•
Develop programs to help teachers become instructional leaders.
•
Districts reconfigure time of school day to allow for professional development that will provide opportunities for team meetings focused on instructional improvement, and time for teacher leaders to facilitate, plan, mentor, and coach other teachers.
36 The above is a collaborative model that forms interdependence between district and school site administrators – a model that is conducive to building a comprehensive system (Bloom, 2003). Eliciting and obtaining feedback from school site administrators are key in strengthening the relationship. Building the capacity of principals to be instructional leaders was studied by Fink and Resnick (2001) who examined districts' efforts to create leaders who could accomplish sweeping turns in language arts and mathematics. Essential capacity building strategies described in their study were: nested learning communities, principal institutes, leadership for instruction, peer learning, and individual coaching. Datnow (2005) suggests that institutionalization which leads to sustainability involves a layered process wherein structural routines and rituals are built into the organization. It was further noted that the quality of a site leader should be that of “a savvy political leader,” in an already well-institutionalized setting. Datnow also emphasizes that reform is a result of state, district, school, and classroom forces all acting together to shape long term sustainability. Sustainability ought to begin as part of the district plan. Elmore and Burney (2000) suggest that a district’s requirement for instructional leadership falls under its strategic plan for instructional improvement. In a study conducted of six new principals in Community School District #2 in New York City, Elmore states that the district’s plan for massive instructional improvement consisted of professional development that focused on instruction for teachers and principals. It is further
37 noted that a focus was placed on the need of furthering knowledge, skills, and professional development for principals. This district’s practices included: •
Principal Conferences - monthly day long conferences wherein agendas were established to provide principals with direction in which to lead their school sites.
•
School-Based Staff Developers (direct work with teachers) that address one content area at a time.
•
Principal Site Visits – one to four times per year district personnel visit school sites to monitor progress and provide guidance.
•
Principal Study Groups – topics of discussion include conducting effective staff meetings, long term planning, intervention practices, etc.
•
New Principal Support Groups – one type is a mentor/mentee relationship which falls under the guidance of district personnel and the other is to procure direct support or assistance from district personnel to assist with instructional and administrative issues.
Elmore concludes from this study that despite the background and experiences of the new principals, they all have the foundational pieces to carry out the agenda of the district on improvement of instruction. Strong organizational leadership with a vision that entails continuous learning through a comprehensive professional development plan will set the path toward improvement efforts.
38 Leadership and Learning Organizations In the Leaders New Work Building Learning Organizations by Peter M. Senge (1999), he states at a conference sponsored by MIT entitled, “Transforming Organizations,” that there were two reoccurring questions: “How can we build organizations in which continuous learning occurs?” and “What kind of a person can best lead the learning organization?” These questions are extremely relevant in a public education system that demands accountability wherein data analysis is continuous, requires constant evaluation, planning, and action. Senge notes that the leader of a learning organization requires new skills and tools. A leadership development model approach, fashioned after that of the business world, is showcased repeatedly in Senge’s work. He describes three types of leaders: (1) the local line leader, (2) the executive leader, and (3) the internal networkers. These levels of leadership are analogous to distributive leadership, wherein the executive leader supports the local line leaders, formulates the learning infrastructure, and leads by example in the steady process of developing the customs and behaviors of a learning culture. Senge (2001) has coined the term ‘learning organization’ to mean "a group of people continually enhancing their capacity to create what they want to create” (Senge, 2001, p. 125). Simply put, it is an avenue to an endless process of increasing improvement capacity – instructional improvement if you will in the case of public education institutions. However, one may wonder if this is at all possible in public education where districts and schools are constantly
39 reacting to external pressures? Is it possible to somehow disengage from these pressures and develop thinking patterns and collaborative interaction to advance an organization that is always learning? Leadership Development Models Fullan (2002) contends that “learning at work — learning in context occurs, for example, when principals are members of a district's inter-visitation study team for which they examine real problems — and the solutions they have devised — in their own systems” (p. 19). This type of situational learning fosters shared knowledge and commitment, improving the social context of an organization. It also allows selective preservation of ideas and a social milieu for performance monitoring. This type of support system within an organization via problem-solving and strategizing allows for interdependency between district leaders and school site leaders – it is contrary to the familiar district and school culture wherein individuals are isolated (Fink & Resnick, 2001). This would require relationship-building to foster trust amongst colleagues, an endeavor which requires time and effort. In an effort to place an emphasis on relationship-building and trust, but more importantly as a means to build around specific needs of school leaders, some districts have taken on the coaching model of professional development for their school site administrators. This is not to be misconstrued with mentoring, which is a senior coaching a novice. CLASS (Coaching Leaders to Attain Student Success), developed by the New Teacher Center at University of Santa Cruz collaboratively
40 with the Association of California School Administrators is a professional development specifically for coaching (Bloom, Castagna, Warren, 2003). It is based on the following: •
Coach brings a different perspective -- sees circumstances and possibilities.
•
Relationship in coaching is based on trust and consent.
•
Based on assessment of coachee’s needs, coach moves between instruction and facilitation coaching strategies.
•
Coach commitment is to student success and will push coachee with that in mind.
•
Goal-setting and ongoing assessment is based on professional administrative standards (ISLLC and CaPSELs).1 Bloom et al (2003) further emphasize that coaching could be cognitive,
transformational, facilitative, or instructional (for instance when a principal needs assistance in gathering, analyzing, and evaluating data). Coaching in this case can be one way of assisting principals with gaining the knowledge and skills necessary to make evidence based decisions to improve school quality. California principals’ perspectives on effective professional development are noted in a 2006 EdSource survey wherein California principals were asked to rank training needs. They indicated that utilization of assessment data is their priority for
1
ISLLC (Interstate School Leadership Licensure Consortium) and CaPSELs (California Professional Standards for Educational Leaders).
41 professional development. Other items that followed, particularly of principals in high priority schools, were evaluation in teacher instruction and a focus on the needs of English language learners. When surveyed and asked to rank the types of professional development experiences they’ve had to date, based on their effectiveness of their day to day practice, they responded: •
Workshops/conferences related to principalship, 76% reported great or moderate influence.
•
Individual or collaborative research on a topic of appeal 70%.
•
Partaking in a principal group 62%.
•
Mentoring or peer observation/coaching 59%.
Other items on the bottom of the list were district training, other school visits, university course, and AB 75 training, wherein 47% said it had a great or moderate influence on their practice (Darling-Hammond & Orphanos, 2006). These results corroborate previous findings of some of the unsuccessful state and district trainings (Darling-Hammond & Orphanos, 2006; Fullan, 2002; Schmoker, 2006). To understand the effects of a school site administrator’s leadership it is important to gain the teachers’ perspectives as well. Teachers’ perspectives on effective instructional leadership in a study by Blasé & Blasé (1999) reflect similarities to the above principals’ perspectives. Teachers note six strategies that principals use to promote teaching and learning in schools and to increase teachers’ professional development:
42 (1) Emphasizing the study of teaching and learning. (2) Supporting collaboration efforts among educators. (3) Developing coaching relationships among educators. (4) Encouraging and supporting redesign of programs. (5) Applying the principles of adult learning, growth, and development to all phases of staff development. (6) Implementing action research to inform instructional decision making. Of particular note is the “implementing action research to inform instructional decision making” (Blasé & Blasé, 1999, p. 135). Principals conducted professional development through action research projects with a focus on class and school-based data. This type of professional development takes place with the understanding that proper determination of the effects of teaching and learning need to be centered on careful collection of data in order to properly diagnose challenges. This research further states that the process continues with an exploration for different solutions, consensus to act, and the careful monitoring of what worked and what didn’t. The process and cycle continues on either the same challenge or addressing a new one. In conclusion, it was noted that such efforts by principals is in the early developmental stage and data did not reflect a strong effect on teachers (Blasé & Blasé, 1999). Hence, there is a need for districts to acquire comprehensive professional development for their school site administrators in organizational leadership that entails evidence-based decision making.
43 Professional development for school site administrators, based on the above research, is fragmented in the State of California with the State and the districts providing what they believe is best for administrators. Also cited in the research are some long term professional development models that allow for breadth and depth based on sequencing and continuity for a prolonged period of time. It is yet to be known whether such efforts directly affect classroom instruction and improvement in student outcome. Thus, there is a need for continued research in effective professional development for school site administrators for the purpose of increasing student outcomes. Security in the practice of evidence-based leadership includes many of the elements mentioned above, such as: leadership focus, values, action theories, and the availability of data and their attendant literacy (Knapp et al., 2006).
Summary of the Literature Review The public school system requires special kinds of leaders – organizational leaders – who possess the knowledge, skills, and motivation that are fundamental in carrying out the task of instructional leadership in today’s high-stakes accountability era. The district, although confined by funding allocations from the state, is responsible for implementing a strategic plan which is directly aligned with student performance and priorities, central to language arts and mathematics instruction. A district superintendent’s resolve to accomplish a strategic plan that increases student achievement requires a team effort, necessitating the support of district and school
44 site personnel. Further, the philosophy that rigorous achievement standards, curriculum, instruction, assessments, capacity development, management and organizational strategies are vital to raising achievement must be pervasive throughout the schools (Spillane & Thompson, 1997). Such ambitious endeavors will require school districts to take a comprehensive and systematic approach to developing habits of mind wherein everyone shares the belief that “. . . to make responsible decisions, workers in any organization must have a steady flow of information about their work and its outcomes and continual opportunities to build their knowledge” (Darling-Hammond, 2002). As cited in the literature review, a systemic, comprehensive, and well structured approach to professional development for school site administrators is necessary to meet today’s challenges (Peterson, 2002). There also must be an understanding that accountability happens at all levels ranging from the federal government all the way to the school site; hence, an approach that includes the attitude of “What gets monitored, gets done” can make a significant difference in evaluating what works and what doesn’t (DuFour & Burnette, 2002). In a search and review of the literature to find how school districts prepare their school site leaders to lead via data-driven decision making, it was discovered that various methods and approaches to professional development have been attempted. The bulk of the cited research concludes that indeed professional
45 development in data-driven decision making is necessary for school site leaders to effectively improve student outcomes. The federal government, through NCLB, will have a target increase in AYP goals beginning in 2008 and increasing each year thereafter until 2014 (CDE, 2007). School districts must not only raise the standards for principal performance, but assure that principals receive a highly skilled professional development program that is comprehensive and continuous (NSDC, 2008). Based on the above, I intend to study two school districts in the State of California who have comprehensive professional development programs in data-driven decision making for their school site administrators. I will view the professional development programs in these districts through the framework of Knapp et al. (2006), Element of Data-Informed Leadership, which offers ways of recognizing and understanding what is or what is not happening in a particular setting. The following is a visual of the framework:
46
Figure 1. Data-Informed Leadership, Knapp, Swinnerton, Copland, Monpas-Huber (2006) Furthermore, the districts being examined will attribute their success of evidence-based decision making practices at their school sites to their efforts in professional development. I anticipate finding strong leadership at the district and school site levels that value data use and its attendant effect on student achievement with the belief that “. . . schools need accurate and actionable information about what students know and can do so that they can plan effectively for student learning” (Heritage & Yeagley, 2005, p. 320). The intent of this study is to add to the literature as well as to provide a roadmap to understanding the comprehensive and systematic process in which
47 districts conduct professional development training for their school site administrators in data-driven decision making. The aforementioned literature review provided a broad perspective in such areas as models of professional development as well as some longitudinal studies in data-driven decision making or leadership training. This study aims to answer the “What works?” and “How does it work?” questions through in depth multiple case study of two districts, wherein an effective professional development plan in DDDM for the purpose of increasing student achievement has been carried out.
48 CHAPTER THREE Methodology This chapter explains the research design, which includes the proposed qualitative methodology, sampling, instrumentation, data collection plan, and proposed data analysis. The intent of the researcher is to spotlight districts that affirm promising practices in professional development for their school site administrators in datadriven decision making (DDDM) for the purpose of increasing student achievement. Two districts in the State of California were studied in an effort to find answers to the following research questions: How do districts build the capacity of school leaders to engage in data-driven decision making? •
How is the need for professional development assessed?
•
What type of training and support is provided school site leaders?
•
What evidence is there that leaders put new data-driven practices into place?
•
What knowledge gaps still exist?
Research Design The intent of this study is to employ qualitative research methods to provide an in-depth description of the policies and procedures deployed in carrying out
49 effective professional development for school site administrators in DDDM. Through a qualitative design, it is hoped that the, ‘What is effective?’ as well as the ‘How does it work?’ questions are answered. The strength of the study comes from district and school administrator interviews, professional development observations, and document analysis (Patton, 2002). The above forms of data collection and analysis procedures are for the purpose of: understanding the type of professional development program offered by the districts to their school site administrators; discovering specific district actions and strategies to promote data use at the school sites; ascertaining the outcomes of the trainings; and, to determine the type of monitoring and follow-up on the part of district leaders. The multiple case study design, one of various forms of qualitative research, is specifically used when studying educational innovations (Merriam, 1998). This type of approach is valuable when questions of “how” and “why” need to be answered and the recommended method is via contextual conditions which are specific to a phenomenon of study. Furthermore, the multiple case study design is conducive to answering research questions that require multiple data sources. This influenced the choice of multiple case study design as the best qualitative research method for this inquiry (Merriam, 1998). The choice of a small select sample to research, that of two districts, resulted from the opportunity to obtain sensitive and
50 descriptive data rather than that which would have been collected in a large-scale study (Merriam, 1998). Furthermore, the qualitative descriptive-analytic multiple case study research method encompasses the use of interviews, observations, and document analysis to collect data. Merriam (1998) describes the case study method to be well suited for inquiry in research that intends to examine a process in its actual context. The intent of this multiple case study is to provide an in-depth description of the processes involved in carrying out professional development to school site administrators in data-driven decision making from the perspective of select districts and school site administrators. Merriam (1998) further notes that case study is conducive to understanding a phenomenon from the perspective of the participants. My decision to examine two school districts and their selected school site administrators is based on my wish to acquire insight into the districts’ and school site administrators’ perspectives on the professional development instruction as well as the process followed to increase capacity in DDDM. Merriam (1998) states that case studies are the primary designs to gain insight, discovery, and analysis, rather than hypothesis testing. Thus, an acquisition of insight through discovery and analysis of two school districts and their respective schools in school site administrators’ professional development in data-driven decision making has been conducted. A case study approach has allowed me to discover the process (Merriam, 1998). Process includes a description of the context and the people in the studies, but most importantly, to
51 describe the development and practices by which data-driven decision making professional development began and continues to evolve within the districts and their respective school sites.
Sample This study focuses on two California school districts and selected school sites within the districts. Intensity sampling was sought in order to choose units of analysis that would yield the most information and have the greatest impact on the development of knowledge and skills of professional development for school site administrators in DDDM. A smaller number of districts has allowed for a more in depth study, particularly when the samples are information rich. The validity or meaningfulness of the study is dictated by the richness of the information procured rather than by the size of the sample (Patton, 2002). Since a multiple case study was used, the researcher can provide a detailed account of each. This allows for cross-case comparison, either in a chart summary or prose (Creswell, 1998). Multiple cases are preferable to single cases, specifically when cases may not necessarily represent the range of behaviors and/or profiles, experiences, outcomes, or situations desirable. However, the inclusion of multiple cases has limited the depth by which each case is analyzed and has also had connotations for the structure and length of the final report.
52 Through the utilization of purposeful sampling, districts that have an effective and comprehensive system-wide professional development in DDDM for school site administrators were selected. Information about districts that fit these criteria was solicited from county education offices as well as researchers in the field of education. Districts’ professional development information was also verified via school district publications. Four school districts were initially considered, and two districts were chosen as the units of analysis. Selected districts and their respective school sites were chosen because they attributed the utilization of DDDM by school site administrators in part to a successful professional development program. Selection criteria for districts deemed effective will include proactive rather than reactive measures in their strategic plan for professional development, which may include a designated individual who plans, organizes, and conducts professional development for the district’s school site administrators. Selection criteria for school site administrators within each district were based on their effectiveness in carrying out the district plan of professional development in DDDM. Considerations were also made in a district’s willingness to solicit and network with outside sources in the development of a comprehensive plan (Patton, 2002).
53 Overview of the Districts The two school districts, Buck Unified School District and D’Angelo Unified School District2, were asked to allow me to visit their district offices and school sites to collect information for this study on the districts’ professional development program for school site administrators in DDDM. Buck Unified School District Buck Unified School District (BUSD) was formed in 1965 when the Banfield, Universe, Cattle, and Kingston School Districts unified and became known as Buck Unified School District, and educates approximately 21,000 students. It is an urban school district that serves the communities of the cities of Amber, Coral, Hue, as well as portions of Lake, Lapis, and Naples. The District has received county, state, and national recognition for outstanding programs in counseling, alternative education, staff development, and labor relations. Staff members have been selected to participate on state and national educational committees and have been invited to make presentations at national, state, and local conferences. Students have been recognized as National Merit scholars, academic decathlon winners, and participants in the Model United Nations program. Approximately eighty-five percent of the graduating students go on to higher education. The Buck Unified School District is governed by a seven member Board of Education. The district includes nineteen elementary schools, five middle schools, 2
Pseudonyms are used for the purposes of confidentiality.
54 three comprehensive high schools, a college prep 7-12 school, a continuation high school, infant/children centers, extended-day care, and adult school. The diverse ethnicity of the community is a reflection of the district’s K-12 population which is American Indian/Alaskan Native, Asian, African American, Filipino, Hispanic, Pacific Islander, Portuguese, and White (CDE, 2008). To meet the diverse needs of its community, they have incorporated professional development as a key component in phase III of their strategic plan (2006-2010). Of note is strategic direction 5, professional growth for all employees. Within that component, specific attention is placed on encouraging a professional culture of leadership, providing research based professional development, customized professional development opportunities, promotion of collaborative learning communities, and recruiting and supporting high quality professionals (Buck Unified, 2008). D’Angelo Unified School District D’Angelo Unified School District (DUSD), formed in 1879, serves a culturally diverse student population of approximately 30,000 students. The district serves the entire community of D’Angelo as well as a small portion of La Cross Flamingo, Lake, and Rose Garden. More than fifty percent of its schools have received awards of excellence such as National Blue Ribbon and Distinguished School awards from the U.S. Office of Education and the State Department of Education, respectively.
55 The district has a Board of Education, a five-member body, which governs 20 elementary schools, four middle schools, three comprehensive high schools, a magnet high school, one continuation high school, and a developmental center for multi-handicapped students – a total of 30 schools. DUSD prides itself on an educational program which is innovative and states that it is part of a community that is committed toward public schools with more than 72% of voters in recent years approving a $186 million bond measure to renovate schools. Furthermore, DUSD has a comprehensive strategic plan which includes professional development for its employees. Professional development, direction 3 in the strategic plan, notes that it is based on ‘research and proven learning theories.’ It is offered to all employees and it is meant to ‘. . . inspire excellence; promote skills, knowledge and areas of expertise; and enhance services to students, ultimately resulting in increasing student learning.’ Professional development at DUSD is said to be tailored to the specific needs of individuals, allowing time for partnerships, peer-mentoring, analysis, reflection, and refinement. Such promising practices are also said to be communicated in groups district-wide. Professional development targets for individual school sites are based on state and national standards and are aligned to the school site plan. It is expected that sites produce evidence of meeting targets via site plans, survey data, and calendar of events. Accompanying those targets are developed district strategies to support schools in their effort to increase student achievement. District strategies include: professional development based on
56 site needs, research, and data; resources; pre-service, in-service, and administrative support; follow-up support for continual planning, reflection, and implementation of professional development activities.
Data Collection Procedures According to Merriam (1998), case studies usually rely on the three strategies of interviewing, observing, and analyzing documents. Usually one or two strategies are used more than the others. In this study, the primary strategy employed is interviewing. Observations of professional development and document analysis were used to support methods for collecting data as well as for the triangulation of data. The process of data collection began with individual semi-structured interview of school district leaders that included: Superintendents, Assistant Superintendents of Education Services, and Directors. Interviews were conducted with three principals (elementary, middle, and high school) at each district. A total of thirteen interviews (one hour minimum) were conducted at both districts. Please refer to table of all interviews conducted.
57 INTERVIEW PARTICIPANTS DISTRICT Buck USD Buck USD Buck USD Buck USD Buck USD Buck USD Buck USD Buck USD D’Angelo USD D’Angelo USD D’Angelo USD D’Angelo USD D’Angelo USD
POSITION Superintendent Assistant Superintendent of Academic Services Director of Child Development & Special Programs Director of Schools Supervisor of Curriculum & Professional Development/Magnet Programs Elementary School Principal Middle School Principal High School Principal Superintendent Assistant Superintendent of Educational Services Elementary School Principal Middle School Principal High School Principal
Table 1. Interview Participants in Buck and D’Angelo Unified School Districts (2008).
The interviews with district leaders focused on the strategic plan for professional development vis-à-vis data-driven decision making for school site administrators, the districts’ method of execution, its monitoring and evaluation process, and the attendant results. Each interview was a minimum of 60 minutes in length, tape recorded, and fully transcribed. The protocol for the district leaders’ interviews is included in Appendix A. The interviews with school site leaders, three from each district, focused on soliciting their understanding of the district’s strategic plan for professional development vis-à-vis data-driven decision making for school site administrators, the districts’ process or method of execution, their evaluation and attendant use of the
58 knowledge and skills gained from the trainings, and the results with respect to the implementation of DDDM at their school sites. Each principal interview was also conducted for a minimum of 60 minutes, tape recorded, and fully transcribed. The protocol for the principal interviews is included in Appendix B. Data collection included observations of professional development for school site administrators at the district office. Observations were for the purpose of determining the types and extent of discussions centered on data use, data-driven decision making, and knowledge and skills needed in building a culture of DDDM at the school sites. The extent of the observations was limited to observing and taking field notes. I was a participant observer; hence, I did not participate in any discussion. The purpose of the observations and their attendant field notes was to allow for triangulation with other data (interviews and artifacts). Lastly, the process of data collection included a third type of data, district and school site artifacts. Documents collected and analyzed include, but were not limited to, data binders that included district and school generated reports, state assessment reports, formal and informal documents from the district or school sites such as agendas, policies and procedures, and protocols that relate to the districts’ professional development for their school site administrators. When possible, I photocopied documents for the purpose of cataloging and coding. With all documents, content analysis was conducted.
59 Data Analysis Transcribed interviews, observational field notes, and district documents were coded and analyzed for the purpose of correlating findings with the conceptual framework of this study, Data-Informed Leadership (Knapp et al., 2006) and its research questions. More intricate codes were cultivated to ascertain common themes that span the units of analysis through the use of color coding (Appendix F). Since I have a variety of data from two units of analysis, utilization of tables and charts were necessary to organize the process of analysis. Data and document triangulation was necessary in order to perform a synthesis of lessons learned from the two sites which will hopefully contribute to the effectiveness in school districts’ professional development in data-driven decision making (Patton, 2002, p. 500). The process of data coding and analysis was as follows: Prior to reading interviews, observation notes, and documents, protocols were reviewed, identifying key phrases that relate directly to the research questions. A list of fourteen codes was formulated. Interviews were read and margins annotated to coincide with a number that was given to each code. Interviews were read sequentially from district to school administrators, one district at a time, comparing and contrasting responses between district administrators and school site administrators. Codes were streamlined based on content, as there was overlap in the content that related to each code. Codes were narrowed down to seven, which later served as the most salient themes identified in interviews, observation notes, and documents. All documents
60 were read a second time in the same order and this time were color coded based on the eight codes, which then served as themes. Organization of themes and integration of Data Informed Leadership framework (Knapp et al., 2006) were as follows: •
Themes corresponded to each of the four questions
•
Themes were correlated with interrelated elements of Data-Informed Leadership
•
Integration of data for each theme, as it relates to the Data-Informed Leadership framework, is exhibited in the written content of Chapter 4 The below tables diagram the organizational process for coding, analysis, and
organizing data and the integration of Data-Informed Leadership elements with the themes.
61
1
Protocol Codes and Assigned Numbers for Annotating Data Assess Need for Professional Development
Refined Codes (Color Coded) Needs Assessment for Professional Development
2 3 4
Building Capacity Facilitators of DDDM Resources-Training and Support/Plan
5 6
Process of DDDM Process of Data Analysis
7
Technology/Support/Tools Support in Process of DDDM/Accessibility to Tools
8 9 10 11
Culture of Data Use Types of Data Evaluation Tools Evidence of DDDM
12 District Expectations 13 Gaps
Leadership Capacity Building
Correlation of Theme to Sub-questions Methods of Needs Assessment: Sub-question #1 Leadership Capacity Building: Sub-question #1 Process of DDDM: Sub-question #2 Accessibility to Tools: Sub-question #2
Culture of Data Use
Culture of Data Use: Sub-question #3
Effectiveness of Professional Development
Effectiveness of PD: Sub-question #3
District Expectations
District Expectations: Sub-question #4
Table 2. Organizational Process for Coding, Themes, and Integration with Sub-questions
62
Table 3. Organizational Chart for Integration of Sub-questions, Themes, and Elements of Data-Informed Leadership by Knapp et al., 2006
Ethical Considerations I obtained informed consent from all participants prior to conducting any interviews, observations, and document retrieval to ensure that there was voluntary participation in the research study. During the entirety of data collection, analysis, and the reporting process, I complied with the University of Southern California’s procedures as well as that of the districts and schools being researched. The districts understood the nature of the study and that, at any time, they could withdraw from it. Because the study was conducted at the district and school site levels, anonymity will not be compromised. Data gathered during research, such as taped
63 interviews, interview transcripts, field notes, and documents will be kept confidential in order to guard the names of all participants from other participants and to honor anonymity. Information included in my final dissertation will be presented in ways that mask the individuals’ identities should they choose to remain anonymous after previewing the entirety of the study. Specifically, I was diligent in adhering to guidelines for ethical conduct in research. Institutional Review Board (IRB) approval was obtained through the University of Southern California.
Limitations of the Study The current study gathered qualitative data through interview protocol, observational protocol, and document collection in two California school districts. These districts are identified as having promising practices in professional development for school site administrators that elicit DDDM at the school site level, ultimately affecting student achievement. The study of these districts and their schools were conducted over a short period of time, spanning five months. Factors in the districts and their respective school sites may affect the pertinence and transmission of this study to other sites. However, hopefully the themes found in the study will generalize to other districts since the selection was purposeful and made with the hope that data gathered would be relevant to all California school districts based on the requirements of PSAA and NCLB.
64 Summary This chapter describes the research design and methods of data collection procedures and analysis that I used in my research. In the forthcoming chapters, I will present my research findings as well as an accompanying analysis of the same.
65 CHAPTER FOUR Data Analysis and Data Interpretation
Introduction In this chapter, I will analyze and interpret data collected from interviews, observations, and documents with respect to Buck and D’Angelo Unified School Districts. I viewed the districts’ professional development programs through the framework of Knapp et al.’s (2006) Data-Informed Leadership, which offers ways of recognizing and understanding the process or lack thereof by which professional development is conducted in a particular setting, district and/or school site. The arrangement of this chapter is organized by the research questions and their direct connection to the common themes discovered in the analysis of the research. The following themes were extracted and organized under the corresponding research questions: Research Question: How do districts build the capacity of school leaders to engage in data-driven decision making? The overarching research question was formulated to guide the study and the answer to this question will surface within each common theme, but is most prominent in the themes of Leadership Capacity Building, Support in the Process of Data-Driven Decision Making, and Culture of Data Use. The sub questions and their attendant themes are outlined below.
66 Sub Questions: (a) How is the need for professional development assessed? Theme #1: Methods of Needs Assessment Theme #2: Leadership Capacity Building (b) What types of training and support are provided to school site leaders? Theme #3: Accessibility to Tools Theme #4: Process of Data Driven Decision Making (c) What evidence is there that leaders put new data-driven practices into place? Theme #5: Effectiveness of Professional Development Theme #6: Culture of Data Use (d) What knowledge gaps still exist? Theme #7: District Expectations The answers to the proceeding sub questions via the evidence that was compiled through their attendant emerging themes were discovered during data analysis. Note that Data-Informed Leadership will not be prominently outlined throughout each section but will be weaved in through data interpretation. Therefore, before delving into the themes, I will discuss the framework.
67 A Framework - Data-Informed Leadership Knapp et al. (2006) define Data-Informed Leadership as a “more thoughtful and intentional approach to using data.” They contend that this phrase is a shift or a redefining of data-driven decision making, wherein data-driven decision making indicates making decisions based on data. Data-Informed Leadership insinuates that data affirms that an educational leader brings into the process specific core values, vision, and a contemplative and deliberate approach into the practice of leadership. The implication is that data do not necessarily drive decisions, but that the use of data acknowledges the complexities and ambiguities that play into data use in educational institutions. This lens will draw attention to why and how professional development by the district office for school site administrators in data driven decision making is a more complex endeavor, which entails raising the capacity within a spectrum of knowledge and skills in order for leaders to be effective in increasing student achievement. When applying the lens of Data-Informed Leadership, it is important to understand that it is utilized in the midst of governmental policy environments (see p. 46 for a visual of the framework). This policy environment is the catalyst for high stakes accountability throughout the country and propels districts into refining their practices to meet these expectations. The ‘anchors’ for data informed leadership are: leadership focus, core values, theories of action, data literacy, and available data. The anchors will manifest in the message by the superintendents and their respective
68 district and school site leaders as well as in the district strategic plan. In order for these anchors to be solid there must be an investment in developing leaders’ expertise and data literacy, both of which were used as screening tools in the selection criteria for the above-named districts, and they continue to be part of the learning process of both school districts. Data-informed leadership is also defined through the culture and cycle of inquiry loop wherein districts access or search for information, make sense of it, take action and communicate, learning from action, and then pursue a reframing of a problem. It is my intention to demonstrate that ‘why’ and ‘how’ of districts conducting professional development for school site administrators is through continuous engagement in the culture of inquiry loop. Districts and schools encourage and motivate the development of cultures of inquiry within its institution with the intention and actualization of making strides in student achievement. Furthermore, this culture of inquiry functions symbiotically with the implementation process and the attendant effect on students, educational professionals, as well as continuous learning within the institution. Leaders, specifically district administrators, who by nature are expected to provide the above-named anchors for effective data use, often times define the focus (directly or indirectly) for the data that they generate and use with their principals. Knapp et al. (2006) suggest that the following foci be a priority: (1) Focus attention and effort on improving student learning, (2) Guide the learning of individual
69 professionals, and (3) Guide what is known as ‘system learning’ – similar to the term ‘organizational learning’ coined by Senge (1999). Since this study focuses on how districts build capacity of school leaders in data-driven decision making, the above three priorities were clearly articulated by both units of analysis, and remained at the forefront when gathering and analyzing data. Data presented will be in support of the Data-Informed Leadership framework, and it is hoped that specific insights from the data will expand the framework. I will now turn to the themes that address the research questions.
Need for Professional Development The assessment of professional development through the Data-Informed Leadership framework begins with the mandated requirements of NCLB and PSAA. For district leaders to begin to evaluate professional development needs, they must assess not only the external pressure of governmental mandates but also the knowledge, skills, and intrinsic motivation of the school site leaders to implement new learning at their respective sites. Other considerations in assessing the need for professional development is a district’s expectation of school site administrators data use to improve student achievement and how they will establish the cycle of inquiry at their sites. The prominent themes that emerged in the data were Methods of Needs Assessment, how districts went about assessing the need for professional
70 development, and Leadership Capacity Building, how the need is measured relative to the level of leadership capacity of the school site administrators. Theme #1: Methods of Needs Assessment Buck and D’Angelo Unified School Districts see professional development for school site administrators in data driven decision making necessary and valuable in meeting the required government policy mandates. This is evidenced by their inclusion of professional development in the strategic plan, which dictates their obligation to probe deeply into the needs that exist within professional development for principals. Both district superintendents have established goals within their strategic plan in professional development that resolve to forward district efforts to increase student achievement via a district team effort. Both superintendents have the understanding that the support of district and school site personnel are necessary, and that these individuals should hold the belief that rigorous capacity building, management, and organizational skills and strategies are necessary to increase student achievement system wide. Moreover, they are both of the belief that principals share in this understanding by carrying the message back to their school sites, implementing the plan, making responsible decisions, and most importantly valuing that learning organizations are about having a steady flow of information regarding their work, its outcomes, and proactively seeking opportunities to build their knowledge as well as that of the teachers within their schools. Albeit, the discovery of the professional development needs are different based on their
71 organizational culture, the leadership focus and strategic plan reflect a concentrated effort in student achievement. Buck Unified School District Superintendent’s Message Buck has a vision that is simple and powerful: We believe students in Buck should be as well educated as any in the world. We believe all students have the capacity to be high achievers. We believe people are the cornerstone of our district and students are the reason we are here. D’Angelo Unified School District Superintendent's Message It should be clear that the mission and focus of the D’Angelo Unified School District is student achievement. Each of us should clearly understand how we support that process. Buck Unified School District Buck Unified School District utilizes a concept formation model when conducting professional development for their school site administrators, allowing for innovation in their approach toward responding to government and district accountability measures. The concept formation model allows creative and critical thinking, communication, and independent learning. Since the particular concept to be learned is data-driven decision making, school site administrators would utilize data, technology, and leadership knowledge and skills. Furthermore, creative and critical thinking extend from a common set of features within the district and/or school site, namely best practices. Professional development, however, is not being
72 identified on the basis of a single critical piece in raising student achievement via data driven decision making or through one specific framework. Buck Unified School District has a longstanding history of providing school site administrators with autonomy over their school sites. On the basis that each school site has their unique community and their set of complexities, school site administrators have room for innovation. This allows each school site leader to incorporate their beliefs, values, and experiences within their individual school site environment and within the parameters of the district strategic plan and the individual single school plan. The configuration and performance of the individual school also plays a role in determining professional development training. There is an expectation that school site leaders, in collaboration with district leaders, will make an assessment of their site’s professional development needs. The assistant superintendent shares: We have different levels of ability . . . progress varies in terms of how many schools’ entire staff have been trained and the number of principals trained and their various levels. . . . our roll out last year consisted of a couple of principals who are very tech and data savvy, using data for instructional purposes . . . principals share how to use the data from a principal’s perspective. Due to Buck’s philosophical spirit of autonomy and innovation coupled with the district’s desire to implement a more district wide systematic approach to curriculum and assessment, a decision was made by district administrators to take the opportunity of the new math adoption training to further streamline their process. The assistant superintendent continues by reporting:
73 Everybody is attending the new math curriculum training. A good part of that training is assessment and the data that comes with it. So everybody is looking at benchmark assessments, feeding those benchmark assessments into Data Director so they can be used to make judgments about how and what kids are learning. Through document analysis, it was noted that over five hundred teachers are slotted to attend in-depth training in the newly adopted Houghton Mifflin and Prentice Hall Math programs from now until mid-March 2009. The training emphasizes lesson design, pacing and planning, differentiated instruction, and assessment embedded in curriculum. Of essence are demonstrations in instructional strategies and student engagement. A key component to this training is incorporating Data Director3, which has been in use district wide for one year now. A needs assessment for professional development in DataDirector revealed, in district documents, the following: This data and assessment management system allows teachers and administrators to view, disaggregate, and analyze student assessment data. During the first year of use, approximately 920 teachers and administrators accessed DataDirector for information. Buck’s superintendent articulated that despite budget constraints, the need for professional development is vital, stating, “. . . dollar spent on staff development is the best dollar spent and so we’ve been strategic.” The specific school site needs determine the strategic approach for professional development and in such cases
3
DataDirector is an online data management system which serves as a decision-making support and tool.
74 require a plan to be fashioned that would include a deeper analysis of instruction monitored by benchmark assessments. Buck USD contracted with educational consultant Dennis Parker to provide professional development, Effective Schooling Strategies, for their lowest performing schools in order to increase students' achievement. Parker was contracted via the High Priority Schools Grant to assist in the implementation of effective schooling strategies for one middle school and one high school. Also DataWorks, a researchbased educational support provider, is providing support at an elementary school as part of the high priority program. Through this partnership the teaching staff receives over forty hours of intensive staff development in the areas of explicit direct instruction, assessment analysis, and curriculum design. Professional development is aimed at assisting teachers with tools to ensure student success in the classroom. Staff will continue to employ successful strategies including working with consultant Dennis Parker to implement Strategic Schooling best practices; sharing and discussing test results with students; posting and checking off the California standards on a weekly basis; using Smart Goals to focus on a particular standard for four weeks; and sharing data to assist students who need extra intervention including English language learners. In order to accommodate for the philosophical belief of autonomy and innovation within the district and yet meet the professional development needs of the remaining Buck USD schools, the district has provided other opportunities such as:
75 DuFour’s Professional Learning Communities; Marzano’s training in cultivating and sustaining effective instructional strategies in the classroom; and, other opportunities to seek knowledge. It was also noted by district and school leaders that professional development needs were addressed based on the level of knowledge and skills of the school site administrators. Compass program is really for the new administrators . . . the leadership academy meetings are what in most school districts are called the principals meetings . . . Buck is a little different because the leadership development component speaks to all leaders in the district, not just principals . . . a component of the leadership development academy has always been looking at data through a data driven or results oriented organization. The Director of Schools added all principals come in the morning and then the new principals will stay throughout the afternoon, giving them a full day of professional development. New principal training includes use of DataDirector. Buck administrators also noted the need for professional development for school site administrators to be balanced with structured and unstructured time. The Director of Schools noted, “The idea is that on the leadership academy day there would be a structured time for collaboration, and by structured I mean we would provide the purpose, what we want them to work on collaboratively, talk about what’s working.” She noted that topics for professional development structured time are gleaned from discussions that materialize during these meetings. Planning is consistent with what principals need to know, which is revealed through a cycle of inquiry during the professional development at the Leadership Academy meetings. The district sees this
76 as being flexible and open to the needs of their school administrators. It was discovered through the interviews of district and principals that the goal has never been to have a conventional didactic approach to professional development but rather to determine the need during capacity building time based on voiced needs. Buck, however, is gently moving toward being more systematic in the area of benchmark assessments, the process of which will be discussed in the section for the second sub question, which delineates the process in supporting data-driven decision making. The Director of Schools noted, “The planning committee’s goal is really to revisit topics and issues over and over rather than use the whole shotgun approach of let’s throw everything out there and see what sticks.” Revisiting issues establishes a cycle of inquiry that allows for not simply learning, but opportunities to reframe problems, implement, assess, and allows for communication and collaboration in the process. For Buck this is a priority when determining the need for professional development. The Supervisor of Curriculum and Professional Development indicated that many considerations are made when introducing new concepts, stating: My experience at the district level is that you must take into account that people change and people move at different rates . . . when you plan for something big, you must plan for people who are ready to go, people who are going to look at it for a while, and people who are really resistors, but even they come along eventually, but you just have to keep that in mind when you do something. In an effort to move school administrators to implement systematic instruction and assessment practices at their school sites, district administrators along
77 with select school administrators meet to establish future needs for system learning, the implementation and effects on professional learning, which directly affect student learning. Via a cycle of inquiry in the district infrastructure, which allows for ongoing adjustments in the professional development, a beginning of the year meeting allows for a skeleton outline of professional development for the year. The team is made up of the Assistant Superintendent of Educational Services, Director of Schools, Director of Child Development and Special Programs, the Deputy Superintendent, and select principals throughout the district. The year will begin with a three to four hour meeting, which sets out to discuss issues as they relate to the strategic plan and the superintendent’s initiatives. During the course of the year, this committee meets monthly. The committee conducts ongoing monitoring of the professional development program with its primary concern this year to meet the need of establishing a common language across the district. Thus, the committee will tie professional development into the new math adoption in order to support schools in the use of data on a more frequent basis. An elementary school principal, stated: They’re (district) continually investing in programs and technology to make it easier for us, like Data Director, it’s a very expensive program. It was not an easy choice for our district to spend that money, but the willingness for them to do that so that we and the teachers have that access to data shows their willingness to support us.
78 A middle school reports that the need for professional development is based on district needs. . . . 19 new principals . . . the COMPASS program helps through training and support . . . making data driven decisions through teaming . . . an experienced administrator and two new principals. There is coaching from district, and retired administrators. The needs assessment for professional development at Buck, viewed through the lens of Data Informed Leadership, takes into consideration the complexities in the knowledge and skill set necessary to be an effective educational leader. Buck believes that an investment in the data infrastructure (through the purchase of Data Director) and access (through professional development focusing on instruction and assessment) are an integral part of the ongoing learning required to improve and produce results. With this understanding, the superintendent, in an observed professional development meeting in the beginning of the year, made clear that there is direct correlation between what is expected and what will be provided in principal’s professional development based on CPSELs, the California Professional Standards for Educational Leaders, indicating that expectations and evaluations will be aligned with standards for administrators just as students and teachers are evaluated based on standards. Thus, the need for professional development directly correlates with what school site administrators are expected to know and do.
79 D’Angelo Unified School District District expectations for learning and continuous improvement are prevalent at the D’Angelo Unified School District as well. D’Angelo USD utilizes the Focus on Results framework to guide professional development for district and school site administrators as well as teachers. It has been consistent in following this framework for the last four years. Focus on Results is a process within a framework to actualize improvement in student achievement, which is D’Angelo’s primary goal. A vital component in the process is the establishment of an instructional team of teachers at each school site; utilization of student data to form an instructional focus and to implement research-based instructional practices drawn from each school’s teaching staff; ongoing student assessment; setting and monitoring of goals; and intervention for students not reaching goals. D’Angelo’s superintendent clearly stated that SEAM, a set of questions based on instructional reflection, should be at the forefront: S: Is Instruction StandardsBased? E: Are students actively Engaged in the lesson? A: Are students Assessed regularly to inform and drive instruction? M: Are students reaching Mastery of standards? These reflective questions allow for an ongoing cycle of inquiry, which allows for the reframing of problems; thereby, determining the need for professional development. Rather than the process being school site leader driven, it is teacherdriven, dictated by teachers’ needs to actualize school improvement. Hence, the process of professional development for school site administrators includes 90 lead
80 teachers at district-wide meetings, and a total of 291 lead teachers at thirty schools. These teachers meet monthly for training, engaging in the sharing of best practices, and evaluating progress along the way. The superintendent’s philosophy is to make clear to the community the area of challenge, articulating that it is important for “. . . all D’Angelo children to find success—especially those that have lagged behind in achievement. Approximately 55% of our Hispanic students, 40% of our Caucasian students, and 20% of our Asian students are not proficient in Language Arts and Math . . .” With Focus on Results selected as a means for improvement, and keeping in mind the limitation of financial resources, a needs-assessment was made based on which schools would receive Focus on Results professional development first. Similar to Buck USD’s need to prioritize, a high school principal noted: The first schools that went on Focus on Results were going to have to contribute to the cost and we have no Title 1 money and I was not able to contribute to the cost of anything. . . some of those schools had additional pressures of beating state mandated improvement. It was very wise for the district to start with them because they needed all the help and support. The needs assessment for professional development was tiered – while the district began its implementation of Focus on Results with the first cohort of 13 schools, other school leaders were taking the initiative to engage in the cycle of improvement through organizational restructuring. In other words system learning was happening at some schools without the guidance and direction of a formalized
81 professional development. The schools engaged in Focus on Results professional development and the other schools, not engaged in the process, established very clear anchors for Data-Informed Leadership. The superintendent established leadership focus through asking all schools to produce a “Good News Statement” and an “Urgency Statement.” All schools were expected to work toward improving student achievement. On the second year of Focus on Results implementation, the second group was rolled out, and then the third and last group was rolled out by the third year, 2007-2008. The district is currently in their fourth year of implementation with cohort one having completed their three years of formal professional development through Focus on Results. The professional development roll out process for each group takes a total of three years. The entire process is being led by Focus on Results consultants who guide the principals at monthly meetings. The end goal is for existing administrators, and now for administrators in cohort 1, to exercise lateral capacity building via mentoring, coaching, and training of other administrators in the knowledge and skills they have gained in their three-year roll out. While engaged in this process a clear focus based on each school’s urgency statement is formulated. An assistant superintendent stated, Each principal establishes their goal. We work with them on the goal, looking at their strengths and weaknesses . . . so district wide if you walk down our hallways, we have a good news statement at all schools and we celebrate what we are doing . . . and then we have an urgency statement.
82
Assessing the need for professional development also happens through the central office instructional leadership team (CILT). After thinking through their delivery of support this year, CILT will identify the two highest needs schools, one elementary and one secondary. The team is made up of the assistant superintendents and directors. The goal is to target two specific schools identified through data to assist them in anyway they request, whether it is to get them more data, conduct more data analysis, or provide them with extra support. The assistant superintendent also noted that one of the measures used for assessing the need for professional development is also the schools’ API standings. We’ve done tentative runs on Data Director, all our high schools grew, and three out of four middle schools . . . two elementary schools that dipped had over 900 API. He also stated that administrative makeup at school sites is another consideration, i.e. a school site administrator’s effectiveness and those of their teachers is another element in assessing professional development needs. Principals and teachers attend the Focus on Results meetings together, but district administrators also establish their needs assessment through school site visits where they engage in discussion and cognitive coaching to see what supports can be offered and what avenues to pursue. An elementary school principal notes: Well they (district administrators) were trained sitting right next to us. (They) help us . . . coach . . . come once a month and see our sites . . . they push us and say, tell me how you’re using data, show
83 me what you’ve done this month. Tell me how your teachers are doing . . . I happen to have the Director of HR as my coach. Building a culture of inquiry requires district administrators to be a part of the learning process, where learning becomes a system endeavor. Through conversations, needs outside of Focus on Results may surface, such as was shared by a middle school principal, “Focus on Results gives you the primer, to decide the focus, for instance we chose writing. The district will provide a level of support . . . if the data shows we need more, then we visit the superintendent and he or cabinet will come up with ways they can support us.” Support, as a result of the methods for needs assessment for professional development, is ultimately measured by state expectations. The tables below show the Academic Performance Index for Buck and D’Angelo as well as their respective subgroups, and measurable growth from the previous year. Data indicate Buck made a growth of 11 points, which places it within 5 points of being an 800 district. D’Angelo USD also shows a growth of 11 points with an overall API of 818. Both districts have exceeded the State of California’s average API score of 742. Furthermore, both districts are aware that schools receiving Title 1 funds are subject to identification as a Program Improvement school based on its inability to make its AYP in a particular area for two consecutive years. Buck has 13 Title I schools including 5 in PI status. Four of the five schools met or exceeded their 2008 API growth scores. Both districts have growth in API for all their subgroups.
84
BUCK USD API
Number of Students in 2008 Growth API
2008 Growth
2007 Base
2007-08 Growth
15,478
795
784
11
Subgroup API
African American-not of Hispanic origin
2008 Growth
2007 Base
2007-08 Growth
733
715
18 9
1,471
Yes
33
No
Asian
4,820
Yes
915
906
Filipino
1,543
Yes
858
850
8
Hispanic or Latino
6,018
Yes
696
681
15
145
Yes
775
760
15
White (not of Hispanic origin)
1,324
Yes
796
786
10
Socioeconomically Disadvantaged
5,962
Yes
697
685
12
English Learners
4,155
Yes
697
683
14
Students with Disabilities
1,457
Yes
564
554
10
American Indian or Alaska Native
Pacific Islander
Table 4. California Department of Education, Accountability Progress Report, Buck (2008).
D’ ANGELO USD API
Number of Students in 2008 Growth API
2008 2007 Growth Base
20,311
818
807
2007-08 Growth 11
Subgroup API
20072008 2007 08 Growth Base Growth African American (not of Hispanic origin)
260
Yes
761
746
15
American Indian or Alaska Native
39
No
Asian
2,713
Yes
919
908
11
Filipino Hispanic or Latino
1,269
Yes
858
845
13
4,528
Yes
735
725
10
24
No
11,390
Yes
822
811
11
Socioeconomically Disadvantaged
8,630
Yes
755
742
13
English Learners
10,598
Yes
768
749
19
Students with Disabilities
1,747
Yes
581
576
5
Pacific Islander White (not of Hispanic origin)
Table 5. California Department of Education, Accountability Progress Report, D’Angelo (2008).
85 As evidenced from the above tables, both Buck and D’Angelo made from 5 up to as high as 19 points growth in various subgroup APIs from 2007 to 2008. Both districts are meeting the performance based accountability challenge set by the state of California. Theme #2: Leadership Capacity Building Knowledge and skills to meet the challenges of performance-based accountability in public education to increase student achievement is of prime concern in Buck and D’Angelo USDs. Both districts are acutely aware of the need and urgency for school site leaders to be proficient in data use and analysis, but more importantly for teachers so they can gauge academic progress and instructional practices. The need for professional development is also measured by the school site administrators’ level of leadership capacity to make the above materialize. Buck Unified School District The superintendent of Buck USD approaches capacity building of school site administrators and teachers as multi-pronged. The district’s approach must be one of empowerment for school site administrators and teachers. Just throwing the data out (there) without training principals on how to deliver would be a recipe for disaster. You have to know your audience, you have to walk in their shoes, and you have to do it in the most effective way without shutting them down. An effective way, we discovered, is to not give them the data, but give them the ability to go after the data. They will bring themselves to that gut wrenching moment, how did I do? So we’re not training the principals on the mechanics, but rather how to do it as a presentation to the staff.
86 With delivery being just as important as data use, the superintendent notes, “The ultimate goal is to awaken the collective intelligence. We don’t expect a principal to know everything. We expect them to unleash their teachers. The teachers will automatically go where you want them if it’s presented the right way.” Professional development at the district level has not always been at its peak; however, now the format for principals’ meeting is centered-around lateral capacity building -- colleagues presenting to colleagues. Buck’s superintendent notes, “We need to be selective about the principals we choose to present and worry about egos some other time. You know, maybe after I retire, I’ll worry about egos.” Relationship building, however, is a conduit to increasing the capacity in principals. Buck’s superintendent notes that frequent school visits is an intrusion, obligating school sites to treat you like a celebrity. He stated he sees principals all the time, and they do not mind coming to the district office for meetings, stating “We are informal.” The level of comfort was noticeable through the first half-day professional development observation at the district office, where the superintendent arrived prior to the PD and interacted with the principals. Conversations consisted of personal and work related topics. This was in line with the evidence gathered from district administrators that relationships are built due to the fact that they invite principals to be an integral part of the planning committee. Buck superintendent stated, “So there is little imposed except that which we impose on ourselves and we see ourselves as a collective group. Everybody’s idea has merit and we all gravitate
87 to a good idea.” He also noted that high expectations for student achievement pushes the agenda for leadership capacity building, “My job and the district’s job are to be the conscience . . . we won’t let you ignore those things even though there is comfort (in the relationship).” Along with increasing the leadership capacity of principals is the level of expectation espoused in the superintendent’s leadership focus, “. . . we do have a bottom line . . . a principal needs a game plan and needs to work with his or her staff to improve student achievement. We expect that.” The assistant superintendent indicated that leadership capacity building takes place at the school through providing additional human resources. Through a grant, there will be a coordinator at the site level who conducts frequent walk-through and provides support for staff. This additional monetary resource can put key people in to facilitate the day to day walk, “because this is really providing accessibility to teachers in the classroom, allowing for development and refinement in instruction.” There is also differentiation in capacity building for principals. COMPASS Coaching Our Management: Peer Assistant Support System is a program for first and second year principals and other administrators who may need it. PASS, Peer Assistant Support System, was developed jointly with the union. A district administrator stated, “It sounded better than PAR, Peer Assistance Review, and then we just expanded that type of program to administrators by adding COM (Coaching Our Management).” Principals receiving COMPASS are assigned to retired
88 principals who are COMPASS coaches. In addition to the monthly meeting, a couple times a month, retired principals will go to the site for one-on-one coaching. With close monitoring, guiding, cognitive and reflective coaching from district administrators, progress is made. He continued by reporting, “We’re definitely working with our program improvement schools, working with them on the data, and asking the hard questions.” This year we had two of our PI five schools meet all their AYP goals,” indicating the level of support is commensurate with the need. A principal of a school which has gone from PI 1 to PI 2 status asks, “What are we going to tell the staff, how are we going to convey that message? More importantly what are we going to do about this trend?” Discussions ensue around these issues to see where the district can offer support. The district will work to relieve the principal by freeing up time spent crunching and preparing data. Capacity building for school administrators consists of district administrators offering services with the expectation that principals will be proactive and seize the opportunities to refine their knowledge and skills. For instance, Buck offered approximately twenty after-school workshops on a variety of topics such as writing a single school plan, categorical budget, or data analysis. New principals or anyone interested could come and sit around the table for a one-hour informal presentation. Other district administrators play a significant role in capacity building. The Director of Schools has the responsibility of supervising and evaluating the nineteen elementary school principals. She is also responsible for the COMPASS program.
89 A large part of what she does is design the monthly leadership academy meetings. She works with others to plan and design the leadership academy which is the district’s professional development for all of the leaders in the district, not just principals. Part of the plan for the coming year, for principals, will be devoted to additional training to help them be more comfortable with data. She gives an example of a recent problem that required support in the form of capacity building from district administrators, reporting: One of our schools dropped dramatically in its API this year and this became a challenge of course for the principal who has a high level of accountability . . . I mean it caused a lot of anxiety. Both I and Director of Special Programs talked with the principal, we first looked at . . . scores individually through Data Director to see if we could see any trends to point out to the principal. The Director of Schools indicated during this portion of the interview that many different reports were pulled through the district’s expert on Data Director in preparation to assist this principal in leading her staff to find the answers. The process they took the principal through was what they expected her to model to her staff, stating: What I didn’t want to do is to tell her what to do because her staff needs to see . . . it come from her . . . we spent a couple of hours and I pointed out some of the things that we had seen and with some possible suggestions about what we thought might have happened . . . then we talked about questions she could ask her staff as they are looking at the data (together). District administrators coach principals in the process of the cycle of inquiry so they can replicate the process with their staff. Other skills are needed in the
90 beginning of the year such as the Single School Plan, the guiding document of how schools will determine their work for the year as well as how they will allocate their resources. The Director of Schools stated that she structures the topics for new principals through the COMPASS meetings to meet their needs as the year progresses, stating next week’s COMPASS meeting will be about the Single School Plan and how to work with school site council, which comes up early in the year and principals need the knowledge and skills to conduct this successfully. The COMPASS meetings are also a venue for individual conversations. The Director of Schools reports this is a powerful way to building relationships, providing space for reflective coaching, collaboration, as well as counseling, stating, “. . . there is a lot power in just individual conversation . . . as a way to facilitate relationships between administrators . . . to discover strengths in order to put the pieces together, put people together . . . guide in the way you want it to happen.” Leadership capacity building also takes place at the individual school sites wherein a culture of trust and respect are built to lead the way to distributive leadership. One elementary principal indicates “. . .it’s important having the culture of trust . . . now that’s an ongoing thing, that’s not like something you can say, well we trust each other now, so we can just move on . . . I’ve learned it has to be rebuilt almost daily. . . to try and make sure that people trust each other and feel comfortable with each other.” She indicated that she also has an assistant principal, who is also a classroom teacher, reporting, “. . . she’s a good sounding board for me.”
91 A middle school principal in Buck USD shared the role a district administrator plays in role modeling facilitation, indicating that this skill is a necessary one with principals and teachers. Proper facilitation leads to productive collaboration, which ultimately raises the level of performance in the day to day work at the school. She reports: As administrators we sit in on meetings and coach . . . what I’ve discovered is that we send our teachers to collaborate and we’ve never taught them how to collaborate. At first they were (defensive), but now they understand if you’re doing most of the talking then you’re not facilitating, you’re directing. As soon as we trained people to facilitate correctly, collaboration and trust moved to a different level. Lateral capacity building is the ultimate goal within a district, wherein colleagues are collaborating and sharing their knowledge and skills with one another. The principal of a high school describes his journey from working in a large district and coming to Buck, a much smaller district, stating: I remember gathering the high school principals and saying this is not the way to do business. Let me show you some of the practices that I have been doing for a while. And we finally got the data in here and that made it very interesting. We created our own benchmarks and we used that data . . . a lot of what is being created now has been emulated from what we’re doing. A key element to pushing forward the agenda of improvement is for the district to allow a forum for shared practices among administrators in order to increase principals’ ability. Capacity building at times works from the school site level up to the district level, and a superintendent’s support and leadership focus
92 makes all the difference when working through the process of improvement via leadership capacity building. D’Angelo Unified School District D’Angelo’s leadership capacity building is a key component in the Focus on Results process, wherein the principal is accompanied with the leadership team to the Focus on Results meetings. The superintendent indicated that the entire instructional team comes down with the principal on a monthly basis to be coached in the process of bringing about improvement. The principal and the team carry the message back to their school. D’Angelo’s superintendent reports: The vast majority in the district is focused on literacy in some way . . . writing . . . reading comprehension . . . one high school is focused on higher level thinking skills. Everyone has a focus and everyone has a common language. Now as part of the process, each year we train a cadre of teachers and administrators to actually teach the modules as we move through them. The promise was after three years of being in the program, the facilitator, an outside consultant, would back out and we would have built enough capacity into the program. As with Buck, D’Angelo’s superintendent believes that role modeling is vital to capacity building. This was evident during a professional development observation, and during stated in an interview: Unless you as the superintendent are visible to support the process . . . there will be principals who will not take it seriously. I will tell you I am at 98% of those trainings for the instructional leadership team . . . to openly talk about the great work their doing, give a pep talk, encourage them because you have to walk the talk . . . if it’s that important, that the focus is truly student achievement, then why can’t the damn superintendent be down here to be a part of the process?
93 D’Angelo’s superintendent indicates the importance of providing principals with complete support. He noted that supporting a principal with problematic situations is providing them with tools to do their job. With the support, however, is accountability. “. . . on the other hand, if I’m dealing with a principal like on the graph I showed you (inconsistencies in performance), and I ask him, how are you going to change this? What’s your plan to address this? After giving all the training and giving all the support, they can’t articulate a plan to make the changes, then that’s when they’re done.” Capacity building develops through vision and leadership focus, but sometimes it needs to be accompanied from outside sources. D’Angelo’s superintendent notes that one has to be knowledgeable of the opportunities available whether it is Focus on Results, Springboard, or any other framework for improvement based on what one intends to get out of it. He also indicates that the superintendent must drive the process, standing back and delegating a major endeavor sends the message to everyone that the effort is of secondary importance, stating “ . . . if you’re not willing to stand up and spend the time to do it, they won’t do it. They will shrug it off, you won’t be taken seriously.” This point was emphasized several times during the interview, demonstrating that building the culture of inquiry through viable system learning requires anchors of Data-Informed Leadership – leadership focus, core values, theories of action, data literacy, and available data.
94 The assistant superintendent emphasized that building capacity is a process that needs to be given time. He noted that the support system within the district is both structured and unstructured and formed through triads, wherein three principals work together as peer coaches, the Focus on Results consultants work with all the principals, and a district level administrator is assigned to a triad. He stated that it is his job along with the other assistant superintendent to coordinate Focus on Results throughout the district. Upon assessing the situation, a suggestion was made to bring Focus on Results into the principal’s meetings, which would allow for consistency. He stated, “We actually lessened their time but made it more consistent . . . it allowed for individual coaching with the principals, the one I evaluate and the one that I work with on the Focus on Results triad.” Principals’ leadership skills are necessary to accomplish the expectations set by the superintendent to ensure effective implementation of Focus on Results. An elementary school principal notes that to meet the needs of students “. . . the superintendent is challenging principals to spend 50% of their time in classrooms, so that we know what’s going on, so that our feedback is much more directed, so we can have more courageous conversations with teachers about rigor and connection to the standards.” The Focus on Results process has facilitated capacity building with respect to taking a critical look at instruction and its delivery. Some of the conversations that
95 are elicited are what does engagement look like, how does one raise the level of engagement. She continues by saying, We had a two day training to be presenters . . . we wrote our own curriculum, what we believed we need to do next to deepen our level of understanding . . . having teachers unpack standards and discuss what it looks like as far as teaching to mastery. What are the lessons? Let’s go through and see what really matches . . . what’s a mismatch . . . not simply going through every activity in the textbook. The Focus on Results consultants train us on how to train our staff how to work. This elementary school principal corroborated the assistant superintendent’s statement that capacity building for principals has extended beyond Focus on Results meetings, but have extended into the principal meetings where best practices are shared and discussed. The expectation is that principals take the information and go back and utilize it. This elementary principal also noted that there is an accountability piece between all the stakeholders, stating: . . . accountability piece where you have to come back and say what you did with it . . . it’s the same with grade level meetings . . . what do you do that ensures they met during that time unless you are running from classroom to classroom . . . what are they accountable for . . . what are the four questions to be discussed so you can get feedback, and that’s modeled in our principal meetings as well. Leadership capacity building happens through peer-mentoring outside of district meetings. System learning is evidenced at D’Angelo, similar to what the high school principal shared at Buck. Learning is intrinsic because school leaders believe that learning is an ongoing endeavor. A principal shares: I go to another elementary school every Wednesday to meet one of the principals. . . she really knows what to do with data and how to look at
96 it in a much deeper way . . . we share, what are you going to do when you meet with your teachers? What are you asking them? What is the expectation and how do you follow up? A middle school principal shares similar efforts to build her knowledge and skills. “The other principal and I have a relationship, a professional relationship, where we could trust one another to be open . . . vulnerable with one another . . . and it is the same thing even with my boss the assistant superintendent. . . I trust him but I could be vulnerable with him and know that it isn’t going to come back to bite me.” She also emphasizes that capacity building happens through multiple skills: As you know, an administrator has to have different components in their make up. If you’re just going to use data and using data is your sole criteria for leading your school, you are probably going to end up dying somewhere along the way. You need different skill sets to be able to incorporate data in your leadership skills to lead your staff effectively. If you don’t have people skills, no matter how good you are in data, you’re not going to get anywhere. Data Informed Leadership and having the knowledge to establish the anchors of leadership focus, core values, theories of action, data literacy, and provide data coupled with DDDM play a significant role in carrying out the process. This example is set at the district level where coaching is modeled by district administrators, where in turn principals model that relationship with staff. A D’Angelo high school principal shares: Because of Data Director and Focus on Results, we entered last year and had a team of about four or five, we’ve expanded to eleven. Eleven gives you a much better chance of having those tentacles to draw in more teachers because I firmly believe that the most important conversations do not always happen in the staff
97 development sessions. They happen in the faculty room, at the copy machine, and it’s those exchanges that we now have eleven. This quite honestly was one of my biggest challenges . . . to bring this team to the table in a way that was open-minded. The district recognizes those that are proactive and represent the essence of system learning. They become role models for other principals. A proactive endeavor at one high school was distributive leadership through resource specialist teachers. Distributive leadership through special resource teachers (SRTs) was a choice, a strategic move, for this principal. She noted that department chairs are elected but an SRT is not an official position with an official description. She does what is necessary to move the school and to do what is in the best interest of students. She conveys this to her staff by stating, “. . . success buys independence. As long as we are successful, they don’t have to send someone in to tell us how to do it better, so if we can make progress, our chance of staying independent is greater.” In summation, the needs assessment for professional development at Buck and D’Angelo is influenced by the governmental policy mandates. Most importantly, however, it is based on the desire of district and school site leaders to live up to the leadership focus that the superintendents have espoused. During the observations of professional development meetings at both districts, there was a high level of engagement from the superintendents. Both superintendents remained for the duration of the professional development, and conveyed through their interviews
98 that through their presence and message it is clear that ‘continual learning’ is vital to increase student achievement. Relationship building begins with superintendents, district, and school site administrators meeting formally and informally during the course of the year. Formal meetings take place at the district office and at the school sites between district and school site administrators; however, the cornerstone that exemplifies that both institutions are learning systems is that school site administrators meet consistently throughout the year – meetings that are not mandated – meetings that are born out of the sheer need to engage in the cycle of inquiry. This cycle of inquiry involves taking action and communicating based on system learning, learning from the action (through implementation at the school sites), visiting or revising the problem, accessing or searching various avenues, making sense of the outcomes, and continuing the cycle. This cycle of inquiry which allows for ongoing collaboration and refinement dictates the need for further professional development at both districts with the understanding and support of all stakeholders. Buck gathers their needs assessment through inviting school administrators into the process of developing the year’s agenda for professional development. D’Angelo formulates its professional development based on a process that plays out through the Focus on Results framework. A variety of needs are assessed through collaboration at the district meetings as well as meetings with members of each triad. Both districts have
99 prioritized their professional development by assessing the level of need at individual school sites. With professional development needs assessed, districts must also have a well structured plan to train and support their school site leaders.
Training and Support for School Site Leaders According to Knapp et al. (2006) Data Informed Leadership, training and support requires an investment in (1) the development of data infrastructures, and (2) the development of leaders’ data literacy. Investment in the development of data literacy can be through a relationship with third-party groups, in-house experts, or certificated programs. Buck and D’Angelo have utilized third party vendors as well as in-house staff to provide principals with the training and support to meet their district’s mission and strategic plan goals. Training and support for both districts was quite prominent in two themes: accessibility to tools and the process of data driven decision making.
Theme #3: Accessibility to Tools Buck’s superintendent acknowledges that supporting his principals through training and support is a direct result of listening to their needs, “We purchased . . . DataDirector. It is pretty user friendly . . . you can do almost anything with it and we received extensive training in it. . . the principals came and said, ‘we need this’ . . . They were eager to have it. After reviewing several programs we decided to go with DataDirector.” Buck’s superintendent continued,
100 We set up our principals’ meetings as well as a support system beyond the meetings so they are good users of data . . . and eventually we create schools where teachers and the principal are engaged in the process . . . they are a learning community working on it (improvement process) together. The initial process, however, did not begin in this way. The assistant superintendent recalls her role as a principal seven years ago: we would take data that was the size of a phone book . . . it was this stack and I would look at it and feel somewhat overwhelmed . . . at this time a couple consultants from [the county office of education] presented at the principal’s meeting and talked about how to use the data. I don’t know if at the time we had a full understanding of how to use it. The assistant superintendent noted that prior to December 2007, Buck purchased DataDirector and immediately began a series of trainings. Training was divided into level I, II, and III. Level I was a big overview, level II covered exam building, and level III was the data analysis piece. The training was conducted by Buck’s program specialist and technology professional development trainer. With the intention of providing effective district wide training, a multi-pronged approach was used by training principals, site level administrators, and school site training for teachers. The focus shortly thereafter was to move away from district print out of data binders, which are given to principals. The emphasis was placed on how principals and teachers can access the data through DataDirector. The assistant superintendent noted that the focus will now shift to principals bringing their data with them and talking about how they’re using it. She also shared that accessibility
101 to tools has allowed some principals to be proactive and take data use to higher levels, reporting: This is a principal who knows how to access the information and use it in a meaningful way . . . he needs to play a key role in sharing that expertise with other principals . . . an elementary school principal is also very good with technology and data. There are some strong principals who I believe will play a key role as facilitators at our trainings . . . we try to set it up (professional development) where we look at the best practices within our district. Best practices are also learned through Dennis Parker, with an area of emphasis (being) data. In order to utilize data for improving instruction other tools had to be put in place in order to establish instructional targets, namely benchmark assessments; however, these significant tools were not implemented as a mandate. The Supervisor of Curriculum and Professional Development explains: We have not had any district level benchmark assessments in any subject. We made it an option and we set it up along with this year’s math pilot . . . We bought them a laptop and a scanner for DataDirector. We are using benchmark assessments provided by the publisher. We’re printing for them the first time around, all of the assessments and all of the answer sheets. We are training a person to use the scanner and paying for a day to scan. Our technology trainer is going to train staff, showing them (what to do) once we get the information into DataDirector, and what they can produce. The assistant superintendent and I are in the process of coming up with a protocol, some guiding questions, for grade level teams to use to evaluate their data once they see it. The receptiveness and level of involvement from school site leaders and their teachers has been a source of excitement for the district administrative team. The assistant superintendent noted that as the beginning of the school year approaches more schools are getting on board, stating “ . . . if all 29 decide to pilot . . . we will
102 find a way to give the support.” She indicated that piloting the math adoption simultaneously with advancing the process of data driven decision making was a strategic move, stating “. . . it was more natural to introduce a cause . . . we’ll do a three day math training for all teachers, and one full day on assessments. During that time they’ll go deeper into the assessments . . . how to use DataDirector, putting results into the system, pulling out reports, and looking for trends.” She stated that every teacher will have access to these assessments and will be encouraged to try it this year. In 2009-2010 the target will be district benchmark assessments. A similar approach and training will be used with the language arts adoption process. The assistant superintendent stated, “We will make the same statement. We will provide the incentives and we will have a similar if not better response because they’ll be used to it.” Utilizing the new math adoption coupled with incentives has allowed Buck to implement DataDirector and launch itself from a district with no benchmark assessments to a district with benchmarks in one year. With these new tools in place the district will quickly have to engage in a cycle of inquiry when trying to support a system of twenty nine schools. The assistant superintendent stated “. . . we’re also trying to test the system. What does it really take to do a district benchmark assessment? What type of support do we need to provide schools, but more importantly is really getting them prepared to look at the data, to look at their best practices. That’s really the goal -- to make sure curriculum is covered to a level of mastery and they’re using data to drive instruction.”
103 The superintendent of D’Angelo noted that upon entering the district, it considered itself to be a data-driven district, but notes, “It really was far from that.” He stated that it may have been at some schools and in some classrooms, but not district wide, and it was “. . . virtually non-existent in secondary.” He noted, that to help facilitate the process, DataDirector was the best data tool given to teachers, and it was the springboard for establishing an instructional focus at each school. An interview with the assistant superintendent reveals that the DataDirector program allows test material to be loaded, information by state, district, and school site level. It allows teachers, assistant principals, and principals the ability to access data at their fingertips. He noted that this year was a transition year where they intend to shrink the data notebook down so “. . . it is not a notebook that just sits on the desk.” Both Buck and D’Angelo recognize investment in the data infrastructure and access for principals and teachers is a necessity. They also understand that this investment should be followed up with the investment of developing leaders’ expertise and data literacy. Both districts are slowly weaning their school site administrators from data notebooks, with Buck being on a more accelerated track, yet offering on-site support if needed. With this in mind, both districts, albeit it in different ways, are simultaneous providing training and support in accessibility to tools and in the process of DDDM.
104 Theme #4: Process of Data Driven Decision Making The process of data driven decision making requires an investment in the development of data literacy. A crucial component in this investment is to provide ongoing support to leaders in process of using data. In this particular theme, answers to the overarching question are addressed. Both districts have implemented a plan to facilitate this process via third party organizations with Buck using a variety of resources based on specific school site needs and based on its desire to provide autonomy to their principals. D’Angelo is utilizing one outside source and they are currently entering their fourth year in the process. Both districts are utilizing DataDirector as a conduit to DDDM, with Buck ending its first year and D’Angelo entering its third year. Buck Unified School District To highlight Buck’s initiatives and to create awareness within and surrounding the institution, the superintendent is frequently featured on “The Unite Show” an appealing and effective communication tool for garnering the support of district stakeholders. In a recent show, the superintendent spoke of three initiatives, two of which were: RTI (Response to Intervention) and the state of English Language Learners (a district that has students who speak 39 different languages). Both of these initiatives require training and support in the process of DDDM. As noted earlier, a key training for high priority schools in the district is professional development offered by Dennis Parker, an outside consultant. Training
105 of Buck principals includes an emphasis on data walls, data charts, and the importance of how teachers collect, display, and utilize various types of data beyond CST and CELDT4, i.e. data on student participation, whether students know and understand the standards being taught. This outside consultant has worked with 4-5 schools throughout Buck on how to look at data at different levels and from different perspectives. The assistant superintendent is of the belief that principals who underwent this in depth process with the consultant coupled with DataDirector will take the lead and bring along other principals in the process. Stating, “Most principals are functioning at levels that span intermediate to advance.” Having DataDirector for almost a year, the assistant superintendent states ‘intimate conversations around specific data with site leadership teams’ are taking place. Last year, at the inception of the DataDirector implementation, conversations were broader, and the assistant superintendent would meet with the principals before actually walking their site. The conversation then was about their school plan and what data would be used to develop school plan goals. The Director of Child Development and Special Programs confirms by describing that the enthusiasm for such data tools comes from the fact that there is no longer a need to sit down with paper and pencil for hours crunching data. Now
4
CST, California Standards Tests, and CELDT, California English Language Development Tests, are two of several standardized tests given to measure progress in student achievement in the State of CA.
106 subgroup data can be accessed through punching a couple of buttons. He indicated practical application embedded in professional development is the most effective. Looking at English learners last year, we used DataDirector to print out a report on how many had stalled at the intermediate level on CELDT . . . there’s that glass ceiling and we showed them . . . here’s the table so you can see which kids are moving up and which kids are stalling. Most of the principals ran back to their sites, got on DataDirector, and shared it with their staff. So the key is giving them the tools and letting them take it back and use it. Data Director, which was purchased last year, enables not just the site administrators but the teachers to look at their current scores and also at longitudinal data. The principals have all been trained in how to use DataDirector and how to work with teachers to support them as they look at their own student data. There is more support for the program improvement schools to make sure they have their data, are looking at it, and are being supported in its use. Nonetheless, it is the district expectation that principals will be proactive in utilizing the tools as a prime resource for Data Informed Leadership. This was emphasized by several district administrators, one of whom noted: We are working with them; we constantly try to show them the benefits of using data. The power of using it to move a teacher who doesn’t agree with you, it’s really hard for them to make excuses when the numbers show what you are saying, the evidence doesn’t lie. As a result of district professional development training and support for principals in DDDM, one principal noted that the process of DDDM has been refined through access to tools such as DataDirector. She talks about measures used
107 at her school site to improve instruction and learning via the training and support offered by Buck. Teachers look at multiple sources of data, stating that they use the basic reading inventory (BRI) by Johns, a school wide measure put in place before they had technological assessment tools from the district. BRI is used school wide, K-6, to track progress but also to have a way to compare. This allows them during a grade level meeting or an SST meeting to have a common language. This is done through a tracking card that follows a student from kindergarten to sixth grade. She noted that the process of DDDM has evolved through the years and has heightened with district accountability measures, and some of the means in the past have been publisher tests or summative tests. She gave an example of DDDM by teachers: I was with 3rd grade yesterday. They pulled out their chapter test, and were in the process of re-leveling their groups. So every teacher came in with her test, all leveled out by number and type of questions correct . . . deciding which kids were struggling and needed to be bumped down, which kids were showing improvement and could go up a level. They made a shift right there during that 45 minutes . . . mostly formative assessments at this point, thank goodness because it used to be autopsy . . . let’s look back and see what happened. This approach to DDDM is most effective and is also aligned with the district’s response to intervention (RTI) initiative. Initiatives are still balanced with the need to provide principals with a voice in how the improvement process is carried out at their respective sites. A middle school principal shares the process of DDDM with respect to the training and support provided by Buck USD. At the top of every agenda are the school plan goals and then the collaboration team’s learning goal. We do not call anything PLC
108 because people will think it’s a program . . . it’s the way in which you approach things, as a learning community . . . most of our staff has been trained by DuFour on professional learning communities because we had it district wide. My staff received that training and then last year I took a team of 5 to Marzano’s training. This middle school principal stated that these trainings took her staff deeper into the process. She stated that upon their return, “They bring back the fire and passion for these things, then it spreads quickly where as if I’m the only one going and bringing it back then it’s top down and that doesn’t do us any good . . . because I can’t teach it all, I need five other people helping me out.” She also conducts informal assessments of instructional practices, and similar to principals presenting best practices at principal meetings, she asks teachers to present best practices to their colleagues at staff meetings. The sense of urgency, however, is greater at a low performing high school in Buck USD. The principal of this high school stated that after the purchase and training on DataDirector, teachers now isolate and segregate data, they know how many kids they need to move forward, and how many kids are sitting in the bubble. He stated, “The kids know where they’re going, how many benchmarks they have passed, and how they’re progressing.” A possible significant hurdle in moving a district quickly into the realm of accountability, via DDDM practices, is the relationship that exists between a district and the teachers’ union. The superintendent and other administrators interviewed noted that there is indeed a partnership in Buck:
109 There is a partnership where the union really cares about student achievement. It’s not a union that sits there and tells you they only care about teachers and kids are secondary. They actually tell you that they don’t want bad teachers among their staff. I mean they are still here to represent teachers. They will fight you, they’ll come back here to defend the teacher, but their core roots are research. They want you to look at data. They want you to actually do your job as an administrator and they tell you, do your job, if you have a bad teacher, do your job and get rid of them . . . but do it right . . . and they’ll support you. D’Angelo Unified School District Support is also a profound element at D’Angelo. Through interviews with the superintendent, district, and school site administrators it was noted that the process of DDDM is reinforced through training and support received through the Focus on Results framework, which is currently in its fourth year of implementation. The Focus on Results meetings are monthly where the principal and the school’s leadership team attend and receive in depth training on the use of data. A recent decision was made to eliminate principal meetings where district simply passed along information that could be given in a memo. The superintendent noted, “We’re trying to make them (principal meetings) meaningful training sessions with an instructional focus.” The Superintendent noted, and this was corroborated by other interviews with other district administrators and teachers: The instructional leadership team has become such an important piece of the improvement process, in the best of cases . . . the instructional team drives the process. The principal really almost becomes a facilitator of the process in best practices. So it’s interesting to watch, now that I have principals leaving for various reasons, with a strong instructional focus and a strong team (at a school) a new principal can
110 walk into a situation and immediately hit the bricks running because all that principal needs to do is support that team’s direction. The process of DDDM through Focus on Results has been streamlined. Principals and their instructional leadership teams, with the assistant superintendent, described the process as “very structured and organized.” A notebook shared for document analysis delineated the leadership expectations, process, and tools, funded through a best practices grant. In addition to the Focus on Results meetings, there are instructional walk through days, mentioned earlier by principals. The instructional walk through consists of an assistant superintendent and two other schools. He reports, “All our schools have defined their best practices . . . and address one of the weaknesses through data collection . . . the other schools are versed on what to look for in a pre-walk staff. Then we break up into teams and go through classrooms and look for evidence of best practices.” He also indicated that this past summer the district has continued to weave things together in their PGP (Professional Growth Plans), stating that they’ve integrated Focus on Result into those plans. The goal is to have plans driven based on data and the results, noting that “. . . we’re using interim and common formative assessments to evaluate both pre and post to see how well they’re doing. Now is that implemented district-wide? No, but we’re in the process . . .” To provide the training and support to establish uniformity in best practices across the district, district administrators are coached by Focus on Results
111 consultants. The district has structured staff development so that it will be self sufficient in about three years wherein the consultants work with a group of writers in the summer and through on-going training. They write the staff development for the cohorts, with three cohorts -- they’re all in different places. The trainings will be written and run strictly by the writing team and then it will be presented by district administrators. The writing team consists of teachers, principals, and teacher specialists. And the team has been trained in presentations. The process of DDDM, however, is tailor-made to each school site. In other words, it is decided by the instructional leadership team. For example one of the high schools has writing as their best practice. They’ve done staff development around writing and so the goal there is to raise the level based on Dr. Douglas Reeves’ research. School site administrators share how the training and support they’ve received from the Focus on Results professional development has furthered the improvement process at their school site. An elementary school principal notes that Focus on Results has allowed principals and instructional leadership teams to have common discussions at district meetings, to work together, and to “Borrow and steal, and ask what are you doing? How is that working?” She also noted that they plan their own professional development based on what teachers believe they need for their next steps in order to help them become more proficient in delivery of instruction.
112 Teachers at this principal’s elementary school get a folder with all their data at the beginning of the year. They see the standard based report cards which include all assessment results. They can identify their second language learners, their GATE students, how students scored on their benchmarks, and how they scored on their CST’s. She states, “They get the whole picture. We also have an intervention program, and they’ll figure out which students are going to intervention right up front.” They meet four times a year to look at grade level data, which are summative data. Grade levels do a great job planning collaboratively, every grade level, K through 6th. Similarity in classroom instruction and environment are prevalent. The school principal notes that “. . . a lot of it comes from sharing best practices.” This principal attributes success in her school and district wide to commitment, stating: “. . . our superintendent sits in our meetings with us. Board members come and sit. There is no question about the level of district office commitment.” A middle school principal also expresses the impact of the effort D’Angelo gives in providing training and support in the process of DDDM. She stated that when she began with the district, 10 years ago, they were already beginning to work with data and that it has just gotten better and more intense. She noted access to not only CST data, but district benchmark tests, grade level data, classroom data, wherein teacher and student growth can be measured, but also growth from school to school within the district. She noted, “Healthy competition never hurts anyone and you know very well as a teacher you want to know how well this other school has
113 done? Or how did this other teacher do?” She noted an example of uniformity throughout the district is in the writing benchmark at every grade level, which is normed, administered, and graded by each teacher at each grade level. Furthermore, the district math coordinator does analysis of CST results, benchmark tests by teacher, school, and district. These reports are provided to the teachers each year. The lower performing schools have priority to the data and receive their reports first. She adds that for her school the plan and process for DDDM is school-wide and by department. I give them three years worth of data . . . one year is not enough . . . could have been a group of students, two teachers left in the middle of the year… I don’t want to hear those excuses. I want them to look at the pattern. As a department they will come up with a plan and then it is their responsibility. Electives have a responsibility to help Math and English. They will look to see how they can incorporate math or writing into the PE curriculum. Nobody is going to take the praise for succeeding and no one’s going to take the blame. This middle school principal shared that Focus on Results professional development is rather structured, with planning time allotted in the afternoon. The team gets together using whatever data they have, using whatever training they have received, to sit down and plan the next month’s staff development and bank time. The information learned is taken back and shared with the ILT, Instructional Leadership Team, which consists of every department chair in addition to anyone who wants to be part of that meeting. Then the ILT disseminates the information to everyone else.
114 Another component in the training and support, in the process of DDDM, is the coach assigned to each triad, which could be an assistant superintendent, a director, or another principal. The coach would meet with the principal on a monthly basis to discuss the improvement process at the school as well as talk about the expectations for next month’s visit. The routine of following up on the process of improvement is consistent. This is also mirrored in the relationship between the principals and teachers. For example, a principal shared that she asks departments to take a look at last year’s data and see if they still glean the same problems in the same sub strands. Trend data reinforces the point that the challenge is not the students, but rather the instruction that is happening in the classroom. On the north side of town, near the mountains, is one of D’Angelo’s high schools, which was confident in its high performance. The high school principal shares, “In 1999 when we got our first STAR test results . . . a strong highly qualified school was 800. There was an assumption that we would come in at 800 or maybe 810 or 820, who knew, but there was this strong sense that we were the kind of school that an 800 would indicate.” She noted that the first results came in at 759, and the next year they went down one point. The superintendent at the time called the administrators together and asked, “What do we need to do to get these high schools moving?” This high school principal along with her co-principal put forward the idea of a standard resource teacher (SRT) who for one period a day would focus on academics, looking at the blue prints, and making sure that the
115 curriculum matched, that the instruction given to the students matched the state blue print. They began the process first with English and Math, now they have this in all four subject areas: English, Math, Social Science, and Science. Data analysis at this time was conducted via data notebooks provided by the director of testing. The principal would take a team of 6 teachers, the department chairs, down to the district office to look at the data, analyze them, and plot a course. In going through this exercise, they learned from the data that their students were not doing well on say for instance the genetics section of the life science exam. They may discover that genetics was worth a phenomenal amount on the test but was only covered briefly in the former part of the school year or perhaps the textbook didn’t cover specific material that was tested. Armed with specific knowledge and skills, they were equipped to gauge and carry out instruction effectively. As the district adopted Focus on Results framework and DataDirector for their data management system, the principals noted that they were able to delve deeper into what was happening, particularly with the subgroups. The high school principal reports, “We started this improvement process and last year our API was 861 and this year we’re predicting 870 or higher, so we have made substantial progress through a very deep look at data.” She noted that the process of improvement involves pulling together teams to discuss data that concern them and district administrators allotting time for school teams to look at data. She states the
116 Focus on Results process not only allows them to look at the data but to put together a course of action on how to use the data. The process of DDDM is embedded in practical application but in research as well. Focus on Results training for principals includes professional reading and discussion, such as the Marshall Memo. A principal notes, “. . . for the leadership team, I think it’s essential. I always try to find articles that are focused on where we are and what we are trying to do and that reinforces (the professional development at the school site). People who are studying this have determined that this is going to make a difference, we need to keep going down this road . . . it’s a reinforcer.” There was consistency in the interview, observation, and document data analysis from Buck and D’Angelo regarding the evolutionary process that occurs in the adoption of assessment tools as well as processes for DDDM. Both districts made an investment in the data infrastructure as well as in developing leaders’ expertise and data literacy. Through these investments, both districts were able to develop cultures of inquiry in their respective institutions. These investments consisted of both districts purchasing DataDirector and contracting outside vendors and using in-house experts to drive the process of DDDM. Both districts have principals who were proactive and advocate, pushing the process along. Some principals at each of the districts realized that distributive leadership was necessary to effect real change in building a culture of data use and analysis, with a process of ongoing inquiry. With the districts making a concerted effort to provide professional
117 development to increase student achievement, how do they gauge the level of implementation and its effectiveness?
Evidence that Leaders have put New Practices into Place In an effort to build data expertise by principals, Buck and D’Angelo demonstrate it is not enough to simply invest in data infrastructure and in training and support. Both districts realize that the crucial question is how can the effectiveness of such investments be measured? A foundation, for gathering evidence that tools are effective and that processes are being learned, is to create an internal accountability system within the institution. Interviews with district administrators shed light on how the effectiveness of professional development for school site administrators in DDDM is measured through creating a culture of inquiry and an ongoing cycle of inquiry. Answers to the overarching question are presented in themes 5 and 6. Theme #5: Effectiveness of Professional Development Effectiveness can be measured on a variety of levels, such as on a rubric of 1 to 4 with set criteria, according to the superintendent of Buck USD. He gives an example, “Principals are trained on how to present data to teachers. A principal goes back to their school site and makes the presentation. Is that effective staff development? Certainly on one level it is because it’s being utilized. Utilization can be checked off. Then if one were to ask if the utilization is effective, rating effective
118 utilization would be to say it is making a difference in student achievement.” He noted that looking at student achievement is a bit removed because now you go from the district to the school to the teacher and then to the students. He also noted that “long gone are the days of the Madeline Hunter model, where evaluation is based on teacher behaviors. There needs to be a relentless focus on student behaviors, asking the question: What are the students doing?” Furthermore, “Staff development that never reaches the students is like not having done any staff development at all.” Thus, a plan with action steps from a district level must be formulated to determine the effectiveness of the district’s professional development. Buck relies on achievement data to dictate discussions that probe or elicit action from principals. For instance, the superintendent notes attention will be paid to a particular grade level at a specific school which has excelled in a specific subject. The principal of the school will be asked about the instructional strategies utilized. On the other hand, there could be a school with a grade level that’s floundering, and then the questions would be: What do you think happened? What kind of staff development did you do? Was there a disruption? Once those questions are asked, district administration will pull back. If a principal conducted staff development, then the question would be: Why wasn’t it effective? He noted that it is not about laying blame because “we all have that gut ache” but there is a need to ask the staff development question, “What got in the way of student achievement?” As the interview proceeded, Buck’s superintendent stated that the ideal professional
119 development should be a trainer of trainers’ model, which he believes most only give lip service to, noting: “We never really do a staff development that says when you train people, you bring the box in, you set the projector up, you access adult learning file. Simply do the thing we are trying to get them to do.” He noted that this model of professional development should be at the principal and teachers level. In other words, how does one work a room of 30 teachers or 30 students? This idea revisits the notion that data alone does not ‘drive’ action rather that Data-Informed Leadership fostered from the experiences and wisdom of leaders is more likely to be effective in addressing the challenges of ineffective professional development or ineffective principals in carrying out the knowledge and skills learned. However, Buck’s superintendent also notes that there are other elements which serve as a hindrance. I think we need a longer school day and a longer school year. I think it’s ridiculous what we have now. Theoretically, the way we test is, I’ll say it, stupid. At best we have 180 days and we test 90 percent of that year, so what 10 percent do we get to take out of the standards. The answer is nothing. The state should say that we are going to test at the end of the year, a minimum of 4 days at the end of the school year, after everything is over. How about 100 percent of the year to teach 100 percent of the curriculum? That’s real simple . . . if the state has to pay for 4 more days of school well then so be it, but test after the instruction is completed. External elements play a factor; however, the assistant superintendent is convinced that the professional development she personally received, during her principalship, through Buck’s leadership academy was effective. She passed this
120 knowledge onto her assistant principal, who is now the principal of a Title 1 high achieving elementary school for the third year in a row. “We’ve evolved as a district over time and I would say that my capacity as a site administrator really came from the training that was provided to me during our principal leadership academy.” Building profound relationships within the district through a step within the cycle of inquiry, specifically ‘taking action and communicating’ goes a long way in district administrators accessing the information they need to measure the effects of professional development. The assistant superintendent notes that the principal of this high performing Title 1 elementary school conducts data chats with each student, calling in every student, one by one, and talking to them about the evidence of their performance level. She emphasizes to the students that the only score they need to beat is their own. Along with the data chats are the milestones where success is also celebrated. The assistant superintendent, however, noted that the cycle of inquiry is not simply about random inquiry that is elicited from a district administrator or shared by a school site administrator. She links the focus of her inquiry directly to initiatives and that invariably becomes a measure of professional development effectiveness. She gives an example, Last year we had two initiatives, English Learners and Response to Intervention. In terms of measuring the effectiveness of professional development, I went to the sites and asked them to share the percent of students who moved up a level? What does that (ELD program) look like? What do you need in terms of materials, professional development? So from collecting the data, it became apparent that we needed to do some site workshops. We take the professional
121 development on the road, to that site, and target what that site needs because each school in district has a different need and a different structure. We try to customize our professional development. The assistant superintendent noted that sometimes the district leadership team will walk through a school site. As they walk through the classrooms they are looking for best practices that are making a difference in the classrooms. Targets are set for the next visit, where the principal has a conversation with the teachers to get them on board to meet those targets. School site observations accompanied with data are used to look at best practices and to shine the spotlight on areas of need. More importantly there is an internal accountability between the stakeholders within Buck, which includes the district, school sites, and the union. This year they’ve introduced a new evaluation system for principals based on the CPSELS, but even with the old evaluation system the caliber of the principal’s efforts is demonstrated through their own efforts. Principals were expected to set up goals. These goals are connected to the Single School Plan (SSP). The SSP is married to the strategic plan so that district and schools are synchronized. Principals write goals not only for academic achievement but for other areas such as staff development, fiscal resources, and community support. This allows them to reflect on their achievements throughout the year prior to meeting with district administrators. The assistant superintendent notes, “You know who the top principals are . . . you know because they are taking schools higher and higher every year.” These principals set up goals and produce the evidence that they’re achieving those goals; however, “. . .
122 if you are a principal who’s not getting it done, the teachers are going to complain to the union. . . we have a good partnership with them, they’ll let the superintendent or deputy superintendent know, saying you know what, we have a problem here.” Principals who are effective and utilize best practices are proactive and furthermore take the step of sharing their knowledge with their colleagues. The principal of a Title 1 school (with an API over 800) was asked to present to her colleagues. This principal, according to the Director of Child Development and Special Projects, has “. . . taken data, created that sense of urgency, and then used it to show progress . . . she gets everybody behind her.” This type of leadership is extraordinary but requires a skill set that could be evaluated through the new evaluation system, connected to CPSELS, in that documentation and a portfolio will demonstrate evidence of each element. According to the Director of Schools, however, the district cannot forget about the leadership development aspect of professional development, those elements that are beyond CPSELs. She stresses the importance of having the knowledge and skills needed to manage change, innovation, dealing with issues of ethical leadership, and issues of communication. She notes that these are the ‘big umbrella’ issues that are not directly addressed in CPSELs. She notes, “I think often times educators forget that there are things to be learned about leadership or organizational theory from the business side.” She references Pullman and Bowman and Deal, noting “These are people who really speak to the power of leadership and
123 leadership with a big L.” She notes the fact that these can be taught or at least brought to ones level of consciousness so they are aware of the effect. So they can be strategic about things they do that are specific to their role as a leader, not necessarily an educational leader but as a leader. She states that this could be a huge pitfall and if not addressed, principals will end up being simply managers of schools, “That’s not going to help us to deal with all of the challenges that education is facing right now.” Evaluation of an abstract concept, such as leadership skills, is gleaned by Buck district administrators from collaborative interactions that take place at leadership academy meetings or observations of professional development held at the individual school sites. The Supervisor of Curriculum and Professional Development notes that access to the principals is always a challenge, noting they are not always the best at reading and digesting all the e-mail. To have them as a captive audience at leadership meetings allows for observation. It also is an opportunity to listen to their comments and ideas. She notes it is not that big of a group, there are 29 principals. This allows a more intimate setting to engage principals, but the most valuable observations, are those at the school sites. Through a culture of inquiry by district administrators they garner how principals take action and make sense of the improvement process. A principal describes how she carves out time for collaboration and intervention:
124 Some of the actions we’ve taken are providing library and P.E. back to back, two subjects that can be monitored by a credentialed teacher with the kids going 45 minutes to the library and 45 minutes to P.E. The whole grade level goes at one time so the teachers are all released during that hour and 45 minutes for grade level collaboration. Intervention is also done between the bells because we noticed for years students in need of intervention don’t come after school. I really have a between the bells philosophy because I feel that’s really the only thing I have control of. Her philosophy, which is observed by district administrators, is based not only on her philosophy as an educator but through her organizational leadership skills. The practice is effective because it has shown results in student performance. I think one of the reasons we are successful is we do look at each subgroup. We went over 800 two years ago, then to 801, then 815, this year 843, but the most exciting thing is our subgroups which are recognized by the state, which is our English Language Learner population and our low socioeconomic students, and our Asian students. All those groups are over 800 and that just happened this year. The low socioeconomic and the English Language Learners had 48 to 49 point gains this year and went over 800 so we are thrilled. I was more excited about having those subgroups go over 800 than I was about the overall API score. This principal goes on to explain the effects of a variety of professional development offered by the district have made a difference at her school site. She noted that a school must embrace the concept of professional learning communities. Several staff members at her school site spent several days in summer institute training, allowing for time to make plans on how this would look at their school by looking at pieces such as grade level planning and the grouping of students to help them at each level. At the same time as the PLC training, her staff was introduced to RTI, which
125 was fully embraced. It was a combination of these professional development opportunities that made things work at their school. She stated that training through AB430 made her view things in a different perspective: “I came into teaching at a time when there was really no curriculum. I walked into my classroom the first day. . . I was hired during the 20 to 1 year and I was given a white board, teacher’s desk, and a chair and I really didn’t have much more than that.” She believes that the shift to standards is impressive, but moreover the concept of fidelity is a stretch for a district which prides itself on innovation. She states, “This year we’re not going to deviate. We are not going to do lessons out of order . . . I said to them (staff), I can’t even believe I’m saying this to you right now, but because this is our learning year, let’s just do it. Let’s just follow it.” A middle school principal notes the district gauges professional development effectiveness by referencing the school plan, the district plan, and where data were used, stating, “. . . our district, to its credit, is very trustworthy and administrators will visit your school, and will bring up (issues). Let’s talk about this, but nothing is ever tangibly handed in.” She notes “. . . it’s easy for them to see if you are or are not implementing a plan” stating that all district administrators are highly engaged in the leadership academy professional development sessions. The district utilizes an informal measurement based on leadership academy discussions, guiding the discussion by addressing data use and then using the administrative discussions as a sounding board for what each site is doing. The data from principals mirrors the
126 informal data collection methods espoused through the interviews by district administrators. This middle school principal admitted that not handing in evidence and engaging in a professional dialogue allows for “a culture of trust.” As stated earlier by the superintendent, the effectiveness of professional development is measured by the effect on students. This middle school principal indicated “Because we’ve made data non-threatening, the kids are starting to log things and talk about their own scores.” She describes the culture of trust as ‘spreading’ with everybody logging everything and students charting their own progress and teachers are charting student progress and talking about it freely. “I am finding that the students aren’t as afraid to talk about their own data. Well how did you do on that test? Why did you answer that question that way? So I think the culture of trust in using data is a big thing because data are not about us, it’s about those kids in the classroom and if we train them to use it, then we know they have ownership.” She continues, “I know that the big push is to have teachers read data. Data is of no relevance until students can understand how it impacts them too and so I think we forget that in the equation and until we put it back (in their hands), data is not going to be as relevant for schools.” Like Buck, D’Angelo’s superintendent states that there are multiple measures that demonstrate whether a school site administrator is meeting the expectations of the professional development training. “There are multiple indicators of whether the principal is truly doing a good job. We have a unique evaluation process of the site
127 principals in that they are evaluated in October each year, rather than the end of the year and the state test results are part of the evaluation process. Everyone seems fine with it . . . But I can tell you it’s a norm now. I don’t feel like the principals are overly anxious. They’re going to have to do a study to ensure that what they’re doing is effective. And we actually have a model that they’ll use and we’ll take them through it.” An elementary school principal reports that one of the expectations from the district is to be in the classrooms 50% of the time. “I do. I cover their classroom, so they are in another classroom. I want them to be able to see best practice. I am not always in their room watching. Sometimes, I take their classroom while they go teach a lesson in another.” She continues by indicating that another expectation from the district is that each site determines its best practice. They provide their own professional development, which needs to be research-based. They may need help from the district for references or people within the district that can provide the professional development, but there is no one overseeing professional development like when there was a professional development center. Everyone is doing whatever they want. They all do something different based on their instructional focus. Furthermore, Focus on Results includes regular walk-through with a large group of people from different schools. Four times a year they have a walk through, a team composed of other schools to observe, focusing on specific questions. The
128 first year began with evidence of school focus and the following year evidence of best practices. Last year the walk through was evidence of SEAM. To role model district best practice, this elementary school principal conducts internal walk through at her school site, where they (teachers) all go together. “We’ve been doing internal walk through after school during a bank day, where they can go see one another’s classroom. Every meeting I have is in a different classroom.” During a walk through the principal will use evidence to ask hard questions. Part of the plan for this year is with teachers who are struggling with instruction. They will be required on a monthly basis to visit other classrooms in order to refine their instructional practice, look at pacing, and at the level of rigor. The middle school principal shares that the effectiveness of professional development manifests and is gauged by the level of ongoing dialogue between the principal and the assigned district administrator and/or the Focus on Results consultant. District administrators look for evidence during the walk through, which elicits more inquiry. The improvement process in D’Angelo mirrors the cycle of inquiry as demonstrated by Data-Informed Leadership. The process is supported and then witnessed by the district. One middle school principal shares: Every Wednesday, for 4 to 5 years, we (principal and teachers) met, talked, strategized. . . now district has helped and picked up the cost of two people . . . they are not pulled out, they are not at the district, they are at the school. We have been working with Marzano’s most effective instructional strategies. . . how is this different from what you learned last year, how is World War II different from World War I . . . you are building a wealth of knowledge.
129
Another mechanism put in place at this Title 1 middle school is a program for students with three or more F’s, wherein by the end of the year the total number of F’s for the entire group of students has diminished drastically. It is said to be “a highly successful program involving student tutors.” Student tutors are placed in twenty to one algebra classes and twenty to one intro classes. The school is unable to hire adult tutors so students tutor, who have a gap in their class schedule or study hall. The current staff consists of teachers who were once student tutors. The school offers a community service medallion to anyone who does 100 hours of service. Last year’s graduating class gave 55,000 hours of service to the community and a lot of it was service to fellow students, a tremendous help to low achieving students. Evidence of effective professional development is manifest in the form of student success. One high school principal states there are significant reviews in ninth grade. There is a district writing benchmark which is studied carefully and then in first semester of the tenth grade, all the tenth grade teachers give a former CAHSEE5 essay. Subs are brought in for the day and all English teachers score those essays. This high principal states, “When English teachers get together and score, it is amazing staff development.” After this, they do significant reviews in every tenth grade English class. An administrator or an SRT, special resource teacher, visits every tenth grade English class to explain the CAHSEE and what role 5
CAHSEE, California High School Exit Examination, a test which all public school students are required to pass to earn a high school diploma.
130 it plays in their educational process. For students who haven’t passed, they offer an intensive program in the summer. The test results and graduation rates attest to the effectiveness of this school’s improvement process. Furthermore, research based assessment strategies are conducted and deemed effective. The common formative assessments are teacher created at the school, given to the students, and returned quickly, then re-taught based on the results of the assessments. The principal attests that this is when “you get the gain.” She notes that staff development days this year will be focused on common formative assessments and the goal is to get four in place this year for each subject area. The principal praises her staff, “I think one of the keys to this school’s success has been teacher involvement which is one of the big pieces on Focus on Results. We sit and look at what’s strong and what needs to be fixed.” This high school principal utilizes the power of distributive leadership to assure effectiveness in the strategies and to assist in guiding the improvement process. The genuine efforts and resolve of district and school site administrative staff at Buck and D’Angelo are coupled with the recognition that research based professional development is the conduit to school improvement efforts. Professional development effectiveness is measured via formal and informal assessment (evaluation and discussion/observation), by formative and summative assessment (walk-through evidence and CST data). The measurement of effectiveness of principals mirrors the same accountability piece that is in place for teachers via the
131 professional teaching standards and students via the California State Standards. It must be noted that the effectiveness of the professional development manifests through the cycle of inquiry -- accessing (knowledge) or searching, sense-making, taking action and communicating, learning from action, problem (re) framing. This cycle is possible through districts expending great effort in building cultures of inquiry, and specifically in this era of accountability -- a culture of data use. \Theme #6: Culture of Data Use There is a sense of urgency to accelerate the competency of educators in the use of data, based on NCLB expectations for 2013-2014. Accessibility of data tools and its attendant knowledge and skills for administrators and teachers is important and necessary to begin the process of ultimately developing a sustained culture of data use. The culture of data must also be supported by system-wide learning based on results from the implementation and effects on student, professional, and system learning (an element of Data-Informed Leadership). Buck Unified School District At Buck system learning is not an option and is embraced by most. System learning, however, does not mean loss of autonomy. Buck’s superintendent highly regards the anchors of Data-Informed Leadership, core values and theory of action, when considering system learning for those within the district. Values by which he makes decisions have implications based on a theory of action that he shares with
132 key players in the organization. With the understanding of how Buck operates and a method for how it can intercede to improve, he shares: For example the reading program . . . we actually have three reading programs at the elementary schools. Now that can be a problem when you are trying to do overall staff development; however, the plus is that teachers and principal at that site, the educators, the colleagues at that site, feel empowered to make the best decision for their community. Schools are made up of two kinds of communities. The community they serve and the community they are themselves and if you get a program that the teachers work between the bells on . . . you will get compliance. So teachers and the principal that own the program are going to devote more to it and take it beyond compliance, we hope, through to greatness so that’s why we encourage, not allow, we encourage that kind of school by school decision-making. The core value of autonomy in the use of curricular research-based materials is an example of one of the anchors of Data-Informed Leadership, an extension of core values is a theory of action. For Buck’s superintendent, the empowerment of principals and their staff goes a long way in opening the door for establishing a culture of data use. At Buck this has been one way of building trust and respect between the district and teachers. Buck’s superintendent proudly describes how partnerships facilitate ease of use in building a data use culture: We have a partnership with our teachers union that is a decade old. We don’t fight the same way other people do. We don’t pay lawyers for grievances and neither do they. They devote tens of thousands of dollars -- I’m talking hard cash dollars, to providing staff development to the teachers in our district. It’s rather amazing. I meet weekly with the union president. He reminisces about an occasion when one of the union leaders came in from the national organization, looked at the teachers and said ‘I don’t know who’s been
133 talking to you but you’re not working hard enough,’ and this was a union leader talking to teachers, so that partnership provides incredible human resources but also absolute material resources. He went on to share that his predecessor, who came from outside of California, told him that “there are no prima donnas at Buck, just hard working people.” Their experiences dictated that most districts had islands, people who horded good news and that isn’t the culture at this district. “People want to share something that’s working. They’re freely and eagerly sharing.” He continues: Everyone knows we need to use data. We’re a district that wants to be honest and take a look at ourselves. But my role is to keep that in front of us and require us to take an honest look, focus on the gap with Hispanic and African American students. The gap is there, I’m going to show it (at the Principal’s Leadership Academy), and I am going to ask, ’Now what do we do? After adopting RTI, we analyzed the data after students had failed . . . even though we didn’t want to do that, and we have (received) wonderful responses to that. We achieved a level of improvement. What we neglected is the data that says we should be impacting the original classroom instruction. The superintendent is of the firm belief that data should drive classroom instruction. “One must ask, ‘How can African American kids be successful in this class? How can Hispanic kids be successful?’ Not after school, not on weekends, but in this class that you are teaching.” Data are a significant part of the decision making process. Data use has been an evolutionary process, where in the beginning it was prepared by the district and given to administrators. Now DataDirector allows data to be at administrators and teachers’ fingertips. This entire shift happened in a
134 year time period. Discussion about trends and looking for patterns within current subgroups of students are topics of conversation. If there are patterns within content areas, identification of those trends in the data are made. With this in mind, they began to look at data trends by students and tried to put a face to a name, to really personalize the data. The other thing they did was to pull out blueprints and to look at the percentages for the blueprints. Assessments focus on how did students perform in bands and then correlate that information and look for discrepancies. After that they look at best practices in specific areas, and how to duplicate a practice beyond one classroom. Such conversations became the culture of the school and the district. The assistant superintendent recalls the time she was a principal in the district, stating: Analyzing subgroup data with staff and noting the gap with the African American subgroup, and because I’m African American . . . they saw it, and a teacher was whispering. I told the staff, ‘let’s say it, because if we are not comfortable saying it, then we’ll pretend it’s not there.’ I told lots of personal stories. I really wanted teachers to see these students as if they were their own and I believe this is what began to change the culture. We began the language of adopting kids, having data chats with them, pulling kids one by one to our desk . . . making data number one, having them set goals. This was a result of district training . . . training with Dennis Parker. Culture of data use began prior to the installation of DataDirector. Teachers have received training such as Success for All6, DIBELS7, and other reading
6
SFA, Success For All, is a reading program which can be paid for by Title I and State Compensatory Education funding.
135 assessment tools as a source of data to measure student progress and make determinations about strengths and challenges, and methods of instruction and intervention. The Supervisor of Curriculum and Professional Development noted that sites have used local measures for a number of years. She anticipates that it will take time to get everyone to a level of proficiency where they are able to enter and access data through DataDirector. It is anticipated with time, training, and support this will become a norm district-wide. An elementary school principal notes that she only meets with teachers during grade level collaboration once a month. She states that it’s difficult for a principal to get to all those meetings, but also that she doesn’t want teachers to feel that she’s “looking over their shoulder.” She wants them to feel empowered not as if she’s a tyrant looking down on them. Furthermore, she is aware that many surrounding districts operate differently than Buck. She notes that this year is a pilot year for benchmarking in the district, and attributes the late timeline of implementing benchmark exams to the culture of innovation in the district. She notes that principals are really given a lot of freedom to look at their school site and say, ‘this is my unique population and what can I do to help them specifically, instead of a blanket approach. She notes that this has its good and bad, “. . . because you seem
7
The Dynamic Indicators of Basic Early Literacy Skills (DIBELS) are a set of procedures and measures for assessing the acquisition of early literacy skills from kindergarten through sixth grade..
136 like you’re out there on your own quite a lot. However, if you do come up with a great idea, you can get some support to back you up.” This year Buck starts math benchmark assessments, and this elementary school principal’s team has requested to be part of the benchmarking process. This step into to trimester benchmark exams begins the cross over from a culture of innovation to establishing a deeper culture of DDDM. Trust is a factor in the process. Through interviews and observations, the importance of trust and respect are evident. District administrators as well as school principals value a climate of trust and respect through relationship building. With one principal noting that building a climate of trusting relationships is important “. . . when I do propose something, or if someone comes with an idea, we can all say, hmm, you know, should we try . . . if she says it’s going to work, its going to, you know, we should try it.” She indicates that this is a big consideration that is often times overlooked, going on to say “The groundwork has to be laid in order to make big changes.” A culture of trust and respect opened the door for one principal to introduce the concept of fidelity in the Houghton Mifflin language arts curriculum, appealing to the teaching staff to participate in action research to see if indeed this will make a difference in student achievement. This suggestion was born out of AB430 inservice that the principal attended, which noted that a disservice was being meted upon some students by not allowing them to benefit from explicit and direct instruction. At the end of the day, after teaching the series with fidelity, the school
137 had a 31 point gain. This principal noted, “So I’m a believer because what it really does is give every kid a level playing field.” The proficiency level of students can be determined by how they learn, and may be similar to that of teachers. Each grade level chooses assessment tools and not all will use the same thing. For instance when teachers are clustered producing and analyzing data some teachers learn through a more interactive environment. Other teachers are more proficient and already have their data and have produced nice graphs through DataDirector, a tool which they were not privy to a year ago. The evidence of a culture of data use is prevalent when teachers are utilizing common assessments, and one sees students charting their own progress. A middle school principal reports, “If it’s just the teacher charting the progress, the student doesn’t know what it is and it doesn’t have any relevance.” The culture of data use is strengthened by the benefits by which it provides teachers to gauge instruction as well as by the way it allows students to regulate their learning. Another principal is of the belief that the culture of data use begins with an introduction from the district level, but ultimately there is one purpose and one purpose alone, student achievement. If utilized incorrectly it can be detrimental to building that culture. If its purpose is to improve instructional practices, then it will help in moving students to the next level. A principal reports that the district began using data based on surveys given within the community and at the schools. The superintendent presented the data based on these surveys, sharing community
138 responses, and administrators would learn to look at this type of data. Then, district data would be presented, noting that it did not score as high as expected, and the superintendent reporting that this is an area that needed work. He would ask the question, “How are we going to fix this so that the next year when we take this survey, we are going to do better?” This type of conversation brought data to the forefront and was the beginning of building a culture of data use. Data were then presented based on the strategic plan goals, such as the data on ELL students. He would ask questions such as, “How many students were being re-designated?” In turn principals brought that data back to their schools. Each principal would present the data that were pertinent to their school and staff. A school site administrator views the process of building a culture of data use as informal; although, it was introduced at the beginning of this year through CPSELs, which will be the basis of principal evaluation. Because of the ease of use of Data Director, it is the consensus of those administrators interviewed that everyone has bought into it in a short period of time. This air of acceptance was also noted through professional development observations conducted during this research. A high school principal described the culture of data use as synonymous with a culture of an internal accountability within each teacher . . . stating that this has always been the goal. He noted that data has no blame, but that everyone is held accountable for their data, which is a cultural change in his school, stating “Now, the culture of the place is that everybody looks at everybody’s data. This principal notes
139 that the ultimate message is that the principal is not going to come get us, but rather we’re not going to let each other down.” This high school principal describes how the culture of data use is evident on the walls of the classrooms. It can be seen with teachers, how many standards can be covered in a day, how many times a teacher is seen sitting at the desk when you’re walking through. Data are used to track the number of classrooms administrators visit each day. Data can also reveal student effort, such as how many times homework was not turned in, a test score, and what it takes to be proficient. Building a culture of data use requires use across the board in everything a school or district does, and in every conversation. “This year it finally feels as if it’s working and moving.” The first year the start was slow, showing data, not making much noise, emulating the district’s approach to introducing data use. Then the next meeting they showed a little more. Teachers became comfortable, and without telling them, they began sharing it with each other. D’Angelo Unified School District D’Angelo’s superintendent shares how a deeper culture in data use at his district was established. He indicates that a framework made all the difference in establishing a deeper culture allowing for consistency district wide. After searching around and being aware of the Focus on Results process, through various connections and success stories at another district, where he was close friends with the Superintendent. He began the process of trying to get the principals to buy into
140 the program. Also a funding source was available to make this process materialize to encourage and support entry into the program. The process then began with thirteen schools. The superintendent was told by an assistant superintendent that trying to develop a district wide program was going to be a waste of time and would never happen because the district had provided so much autonomy to school sites. With a complete focus on student achievement, the second year nine more schools were brought in, and then last year the remainder of the schools. The Focus on Results framework, which is research based, allows for differentiation from school to school. It helps each school identify the instructional focus by using its own existing data. The superintendent notes that there were challenges along the way, with the union attempting to sabotage the implementation process; however, because it is so teacher driven and teachers who love what they’re doing, loved being a part of the process. The process allows teachers the opportunity to provide leadership. Some schools have three to four teachers on their leadership team, with others having as many as twelve. The superintendent has given principals the room and space to do that. He noted, “They change behaviors as they trust you and believe what you are doing is good for kids and making their lives better. So, you’ve got to show them this truly is the case; otherwise, they’ll close the door.” The doors at school sites are open three to four times a year for walk through, which creates dialogue, builds climate of trust and respect, and creates peer pressure. A D’Angelo elementary school principal notes that culture of data use is observable
141 through the data posted in classrooms, with every single teacher having their graphs posted on how students are doing. Students will verbalize their goals, based on a pie graph that may be printed on a monthly basis, noting “We’re going for the green because green is proficient. Blue is advanced proficient.” Furthermore, every single student is on the office wall, which demonstrates level of proficiency in their school wide goal of writing. Culture of data use means providing time for teachers to meet and collaborate on data. Also time is allotted for celebration when students advance a level. Release time is provided for teachers to meet once a month to score writing and to have that collaborative time. After scoring is done, post-it notes get moved on the office data wall. Building a culture of data also means communicating with parents and getting them on board as well. “We talked to parents about it. Everybody has a script on back to school night. The message is exactly the same. The teachers have a common language as well.” The principal shared a story, “I had one parent that questioned the office data chart (indicating his opposition to the information being public). The conversation shifted later to, I’d rather my son was proficient.” Building a culture of data use is a result of deeper and wider involvement. “I have a Focus on Results Team that we call FORT. There is representation from every grade level. I have 17, from a staff of 29, 17 of them are on this committee.” This committee meets twice a month with the principal. The principal notes that the most powerful piece is for teachers to see a best practice that’s implemented by a
142 teacher who has out-scored everybody else, where teachers are able talk and share those practices. Like Buck, data use and goal setting must filter down to the classroom, to the student level, and should include parent involvement. Letters are sent home to parents, CST information is shared between teacher and students, and a plan of action is outlined. Each teacher does it differently, but ultimately each student has a goal. Building a culture of data use requires a school site administrator to know and understand the level of use in data with each teacher at each grade level. A middle school principal shares her perspective on the establishment and building of a culture of data use. She states that it is founded on the frequency of data analysis and planning. She notes that data are reviewed at the beginning of the year. Time is blocked every Friday with two of the four being used for a department meeting and the other a staff meeting. At this time common formative assessments are analyzed and discussed, which includes coming up with a plan. The other two Fridays they work independently or as teams on creating new lessons or strategies to help students. Department percentage goals are established as well as individual teacher goals. Common and expected practices to build a culture of data use include commendations of teacher work to build respect and trust, and also to confirm successes of research based practices. A culture of data use is made stronger by bringing more voices to the table. One D’Angelo high school principal notes, “ . . . this new model . . . involving more
143 people . . . gives us more voice and more people to think at the table and it makes the table a richer place.” She also notes that the culture and depth of data analysis can be bolstered by joining an initiative with evidence. For instance, if the emphasis is Algebra, to provide teams with discussion data on Algebra 1, so that a group of principals can all sit and discuss Algebra 1, or if the conversation is closing the achievement gap with Latino students to provide that data for discussion. Evidence of DDDM practices surface in the above themes of effective professional development and culture of data use. Evidence was noted from the data collected in the interviews and observations, and through triangulation of district, school, and observational data. Under the umbrella of core values and theories of action, effective professional development and a culture of data use are implemented. Ultimately the execution of the same provides focus and rationale for participating in a cycle of inquiry. The challenge is more than a technical one, limited to the use of DataDirector; it broadens to acquisition of knowledge and skills to assist students in academic achievement. Furthermore, and most importantly it facilitates capacity building amongst educators to share their effective strategies with one another. The end goal for both districts is to establish a district-wide culture wherein the capacity of school site leaders is such that each school within the district utilizes data conscientiously in daily practice.
144 Existing Knowledge Gaps Knowledge gaps, however, for principals exist and can result from a number of reasons. The following has been discovered through research: a fear regarding consequences of systematic data analysis; other beliefs, aside from instructional practices, that get in the way of student achievement; refusal to expose a lack of knowledge about analyzing and interpreting data; deficiency in investment in a data management system or professional development; and, a disinterest in making a change in practices that are necessary to develop a culture of DDDM. At Buck and D’Angelo, the district offices have utilized Data-Informed Leadership by investing in DataDirector, a data management system. They have also invested in human resources to facilitate use of the data system at the school sites as well as to offer training and support in implementation on an ongoing basis. Both districts have been forthright about the sense of urgency to increase student achievement, despite or perhaps in light of functioning in an accountability system that could lead to punitive action. Both districts are cognizant that a lack of capacity to engage in DDDM through a cycle of inquiry can hamper Data-Informed Leadership by district and school site administrators. Both districts have a goal or an expectation that to increase student achievement, a district must increase the capacity of knowledge and skills of principals to trigger a district-wide effect. To pursue this goal effectively, a district has to set clear expectations but also be able to recognize where the challenges lie. The below table demonstrates that challenge areas lie in
145 approximately 30-40% of students, in Buck and D’Angelo, who have yet to score proficient in language arts and mathematics. Students
District BUCK USD D’Ange lo USD
Performance
Enrollment
% English Learners
% Free or Reduced Meals
% Minorit y
Largest Ethnic Group
% Schools Making API Targets
Made AYP
% Proficient Lang Arts
% Proficien t Math
20,860
20.40%
39.50%
88.10%
Hispanic
68%
No
57.90%
61.50%
27,035
24.60%
41.30%
42.20%
White
72%
No
63.80%
68.50%
Table 6. California Department of Education, Accountability Progress Reporting (2008)
The absolute reality of the percentage of students, not scoring proficient, lends itself to the realization that professional development to positively affect instruction in the classroom is a necessity. The approach, in addressing and presenting this need, will determine the ease of acceptance. Theme #7: District Expectations The superintendent of Buck notes that a contractual feature that has existed for quite a number of years is that individual schools can make time allotments for professional development. Furthermore, it takes 75% of the staff vote and the principal to make it happen. The principal has veto power, noting that even if 100% of the staff votes and the principal opposes, the principal has the power. Of course, he states that this would not be a good practice. Generally speaking, the principal will introduce a concept to the leadership team with the evidence that supports it, and then follows this with a proposal. He notes that invariably if done right, there is no controversy. “If you get people voting no and you want to do it anyway . . .
146 Alexander Poe said, ‘A man convinced against his will is of the same opinion still.’ You can’t get people in a headlock and make them like what you’re asking them to do. The model we like to practice, and it’s contractual, is the leadership team gets together, suggesting to bank some time for collaboration with a purpose of student achievement. He also notes that he doesn’t know of any school within the district that has banked time and does anything else. They could be working on a particular focus for their school site, not necessarily the same focus, but almost all schools bank time. He did note that this is the case in elementary schools; however, at the middle and high schools it is a challenge and there is no set model for secondary. He also stated that he does not recall any negative votes on banking minutes at any school. Even with a leadership team consensus to bank time for collaboration, the superintendent stated that the expectations are clear, a school site administrator must have the following: a game plan for the year based on data analysis, leadership knowledge and skills on how to impact staff, know how to ask questions based on data, and be able to guide staff to delve into the answers for the question, ‘What do we do now?’ If a principal is unsuccessful, and the school is not prospering, he would make changes at the school. This change would not necessarily be stated as the principal’s fault, but from a perspective that both you and I have been unsuccessful in moving the school. If it appears that the principal is not going to leave voluntarily, then the superintendent would not hesitate in moving
147 administrative and/or teaching staff. Reconstitution has been implemented, particularly when a school is stuck at a low level. Buck’s superintendent does admit that test scores do matter, noting that beyond the politics, the California standards are good standards, and that the tests do reflect the standards. The assistant superintendent at Buck is also of the philosophy that buy in from teachers is an important factor. The minimum expectation is a review of CST data in order to make instructional refinement, but also a look at data for the current school roster in order to establish target goals for those students. She notes that each school is at a different level with data use with some utilizing it in professional development, grade level meetings, broadcast programs, and some go as far as to personalize it to help students gain awareness of their own scores. Expectations are also articulated during professional development in the leadership academy wherein an initiative is addressed, i.e. English Language Learners, subgroup achievement gap, etc. and principals are taken through the inquiry cycle of what will they do to work with their staff on this challenge. Sometimes you will find principals saying that they have three or four teachers who don’t even know what a computer is, and then others who have teachers that are savvy with using data. Nonetheless, district administrators, in charge of supervising and evaluating principals, have been given the charge of increasing accountability. The expectations were outlined in a professional development observation that was made
148 early in the year, wherein CPSELs, the California Professional Standards for Educational Leaders, was introduced as an evaluation instrument. One of the elements of CPSELS is using data to make decisions. Along those lines, the district expects that they would all have a level of facility and ease of use with DataDirector, to guide their teachers. This, of course, is with the expectation that school site administrators be proactive particularly since the district has invested in a data management system and provided training and support at the district and site level. During interviews with principals regarding their understanding of district expectations, they innumerate the following: One middle school principal states, “They have never said you have to do this. What they do is provide you the opportunities and you as a principal better take the leadership role. So they trust that we believe in the process and that we’re going to follow through with our staff with the training and support that they’ve offered in terms of DataDirector training and professional development. There are people at our disposal that can help guide us . . . but as far as giving directives, that’s just not the way we function.” District expectations at D’Angelo are also geared toward instructional focus and API growth. The principals are expected to attend district meetings twice a month, once with their instructional leadership team and one meeting with just principals. D’Angelo superintendent notes that the meeting with just principals is to “get them going down the right direction or path.” Thereafter, the district instructional team works with the principals. The superintendent notes, “I’m starting
149 to see and believe that even though they have an instructional focus, they all now know how to use the data to begin to change classroom teaching.” He goes on to explain that he provides them with “a great deal of latitude” with respect to their best practice, indicating various strategies or skills that teachers use to refine instruction. He points out that schools that have engaged in Focus on Results for three years will need to revisit and compare their best practices and investigate whether indeed the practice is truly best practice. During our interview, he displayed longitudinal graphs for each school. He pointed out the API growth for the elementary schools, noting “You can see consistent patterns . . . see how steep some of the patterns are . . . on this one there is no consistency. So . . . the pressure is on district wide to show a consistent pattern of growth.” The superintendent notes that principals are expected to utilize the resources that have been provided. He indicated that providing a data binder is ‘passé’, and that principals are expected to go online and procure data and be ready to discuss them based on the topic of the day. He stated that the district focus is to truly make principals instructional leaders. He stated that they are expected to spend at least one day a week in the classrooms with a goal of 50% of the week, stating “If you don’t want to do that, don’t come work here. That’s the way it is. I don’t want to hear excuses about parents. I don’t want to hear excuses about other issues. If you’re not going to spend the one day a week in the classroom, this is absolutely critical, you won’t make improvement.”
150 The superintendent notes that he role models involvement at principal meetings. He expects principals to do the same at their sites, “If you’re doing your job, then your team will do theirs.” He asks principals, ‘Do you want to trust the coaching process to someone else, or do you want to do it?’ Because if they’re (your designee) not doing it, you are going to be the one to suffer.” Furthermore, the open dialogue that ensues at the leadership meetings is expected to filter down to the sites to bring the schools along in the process. While information is being conveyed to the sites, there is strong union presence that also needs to be considered. In D’Angelo, there are two schools in particular that are very union oriented – one with no push back and the other with considerable push back. It was noted that leadership skills are an extremely important factor in bringing the school along in the process, and the majority of principals have been able to do this successfully. An assistant superintendent notes that district expectations are framed publicly with the good news statement being that the district is achieving at an 800 level, district wide, as well as the highlights of distinguished and blue ribbon schools. The urgency statement is that the district still has 60 percent of Latino population not achieving at grade level as well 40% of our Caucasians and 20% of the Asian population. He states, “We recognize this, we talk about it at board meetings, it’s out in the open, as you walk through the hallways, you will see it all.”
151 He notes that the superintendent has made sure that the message is prominent and that everything in the district centered on student achievement. The expectations of revolving all work around student achievement manifests in the ability for administrators to have difficult conversations when necessary. The assistant superintendent describes that the district uses improvement plans better than most districts. He notes that there are teachers on improvement plans with an expectation that they improve their teaching. At the same level, if an administrator is not doing what they need to do, there are some hard conversations that district administrators pursue. The superintendent through his leadership focus expects the instructional focus to be implemented without deviation. He attends to the principals, visits sites, and has conversations with them as well. The conversations sometimes revolve around keeping their eye on the ball, and not allowing anything to pull them away from the instructional focus, which is the priority. Principals understand this very clearly and keep an eye on the instructional focus. This translates into making sure time is being used wisely but also that time for instruction and student practice is not being wasted. One elementary school principal notes that she has dissuaded her teaching staff from giving book reports because of the time it takes to grade them. “If a book report is done, one must ask how much of it is instructional based, what is being graded, and what’s the purpose?” She constantly asks teachers to identify the purpose, expecting that when she walks into the classroom, a lesson or assignment is based on an instructional
152 purpose or a connection to something valid. The principal expects teachers are using results from DataDirector to guide the purpose. Although there is no particular mandate of how or when teachers use DataDirector, open dialogue and conversation raises the level of expectation. A principal states, “Principals that use it (DataDirector) more frequently, have the discussion with their colleagues, they are making much more progress . . . and those practices are shared with the superintendent. Just like at the principals’ meetings . . . we hammer . . . we model . . . how is goal setting helping in your school, look this one bumped up 54 points . . . tell us what you are doing. There is not a single person there who doesn’t want to do what that person is doing.” Thus role modeling at district professional development is emulated at the school site level to raise the level of expectation by highlighting successes. Even though it is a district expectation to utilize data, data use is not tied to teacher evaluation. One middle school principal describes a conversation she has with the teaching staff. “I think the district’s expectations for data use are simple, that we use it to help guide our instruction. To help students be successful. It’s a tool. It could be used wisely depending on the teacher or the principal. You’re always very sensitive to the fact that we cannot hold the teachers accountable by using the data in their evaluations, but that doesn’t mean we can’t have conversations with that in mind. “You know what -- we need to step it up.” She notes that the message is students need to be proficient and that the excuses of the past cannot
153 continue. She encourages data use to help find solutions in that the data are what they’re being held accountable for. The district expectations are intertwined with a principal’s ability to lead their school site. A principal confirms information from an earlier interview with a district administrator regarding union involvement and push back. She notes: They want people (principals) who are able to use data to lead, but they are also looking for someone who has empathy, compassion, and the professionalism to be able to get a staff to follow them into the trenches, because some of our schools are in the trenches. Right now we are having some union issues . . . staff and administration is unaligned . . . How are we going to cross the line and work together as a team? Data is there to be used productively and wisely, and the district expects you to have the skills to be able to use it wisely. With district expectations for principals to have the know-how to build a team, a middle school principal has a candid conversation with her staff, explaining that as the year progress growth should be evident. Using the data to show percentages, she states: From your 150 students, I’m expecting that maybe you have 10% in the far below basic, you’ve moved 30% into the below basic, and moved another 40% here into the basic, and then you moved the other 30% into proficient. If you can show me the growth, I am going to be happy with you, and you’re going to be happy with yourself. A high school principal sums it up by stating that data utilization is important to making progress and guiding the instructional focus to meet target goals. Bottom line, however, “You have to hire great teachers. The one thing that I always do, I always look at their college transcripts and if I’m hiring them to teach English or
154 history, I want to see A’s and B’s and if I am not seeing them I do not hire them.” The knowledge and capability of the teaching staff is commensurate with the high expectations that the district sets for principals to increase student achievement at their site. This high school principal notes that teachers provide instruction to a spectrum of students and within a range of classes. “You (as a teacher) not only teach honors 10, you teach literacy for success. So we don’t have the situation in which all of our best teachers are teaching our high level kids and they are not working with our struggling students, so for the most part that’s across the board.” Levels of Data-Informed Leadership are apparent at the district and school levels in Buck and D’Angelo. There is strong evidence from interviews and professional development observations that principals are provided with tools and support to meet the expectation that data use, analysis, and interpretation are a nonnegotiable to promote data driven decision making activities to improve instruction at their respective schools.
Conclusion This chapter represents the discovery, analysis, and interpretation of a host of data on district provided professional development for school site administrators in data driven decision making in the Buck and D’Angelo Unified School Districts. This method was useful in studying educational innovations within the district and a valuable approach in the quest to answer the “how” and “why” questions. The
155 research questions focused on the identification of need for professional development, the choice and implementation of professional development, and its effectiveness and attendant challenges. After collecting and analyzing the three types of data (interview, observations, and documents) essential themes surfaced and these themes were subsequently associated with the sub questions formulated for this study. The sub questions answered how the districts assessed the need for professional development, the types of training and support provided school site leaders, evidence that leaders put DDDM practices into place, and what knowledge gaps exist to date. The common themes within each sub question represented the most dominant and significant data which connected to the research questions. The professional development offered to school site administrators at Buck and D’Angelo Unified School Districts was formulated through a thoughtful process, which has had impact on the culture of the organizations, permeating school sites, albeit at different levels, with regard to data driven decision making. Evidence was discernible in the models of ‘best practice’ at school sites within each district. Of note were well-established routines of Data-Informed Leadership. School leaders and their staff modeled elements of Data-Informed Leadership garnered gains in student achievement through DDDM by the superintendents establishing and making clear the anchors for Data-Informed Leadership. Anchors include: a well outlined and pronounced strategic plan and initiatives that are communicated to all stakeholders; core values are reinforced by
156 the superintendents at professional development meetings through their words and their actions, specifically their presence and engagement at all professional development meetings; and, theories of action are pronounced in their choice of professional development based on their respective organizational cultures. Lastly, available data are procured and integrated through DataDirector, a data management system, and through human resource support in the district office. Data literacy is integrated into the professional development component at the district and school site levels. Aside from the anchors for Data-Informed Leadership, are the various elements that establish an ongoing process in creating a learning organization. Buck and D’Angelo have invested in data infrastructure and access as well as in professional development to develop the expertise and data literacy of leaders. Through this process, they continue to build cultures of inquiry, and lasting culture of inquiry allows for school sites to implement a plan of action and observe the effects on student, teacher, and organizational learning. There is awareness, by district and school site administrators, wherein the goal is to engage in a culture of inquiry, where the district continually role models and expects principals to emulate the cycle of inquiry at their schools. It is clear from the research at Buck and D’Angelo that district leaders are stridently aware of the benefits of data, and they have been constant and consistent in the last 5-7 years in escalating the process. There is increasing sustainability to efforts at both districts, albeit through different theories of action. Buck USD
157 maintains an approach of autonomy and innovation at the school sites with regard to research based instructional materials and professional development. D’Angelo USD’s theory of action also maintains autonomy; however, based on the size of the district and the desire to streamline the message, the entire improvement process is funneled through a district-wide framework, Focus on Results. The mainstay of accountability expectations by federal and state policy environments has not only changed the culture within the districts but that of education in general. District leaders are increasingly demanding a scientific approach of inquiry by their school site leaders, and expect that this approach is implemented at their schools. Both Buck and D’Angelo are determined to sustain system learning through a culture of inquiry district-wide at an optimal level. In the context of educational organizations being systems of learning, it is evident that both Buck and D’Angelo Unified School Districts utilize Data-Informed Leadership as an influential conduit between existing performance and desired outcomes. The practices at Buck and D’Angelo merge with Data-Informed Leadership, the framework utilized in this study. The concept of Data-Informed Leadership includes a wide array of issues and questions requiring reflection by leaders when crafting their approach to advancing practices and dealing with challenges in data use to increase student achievement. In essence, data driven decision making does not exist in a vacuum, and indeed Data-Informed Leadership is required to glean the knowledge and skills, which at times seem like nuances, in educational leadership.
158 Data-Informed Leadership broadens the scope of expertise, beyond DDDM, required for advancing student performance.
159 CHAPTER FIVE Summary and Implications of Findings
Introduction Data driven decision making (DDDM) has become part and parcel of an era of K-12 educational accountability based on mandates by the federal and state governments. Invariably, this has resulted in a heightened level of accountability at the school district level. There is an expectation for district leaders to have the knowledge and skills to recognize and implement all the modalities necessary to meet the pressures of performance-based accountability to increase the quality of public education for all students. A viable strategy to developing and sustaining performance-based accountability district-wide is to increase the capacity of school site leaders through systematic and comprehensive professional development. The purpose of this study was two-fold -- to add to the literature and to provide a roadmap for adopting practices which lead to a comprehensive and systematic professional development for school site administrators in data-driven decision making. With external and internal accountability measures, districts are compelled to invest in professional development for school site administrators in data driven decision making; thus, empowering each school leader with the knowledge and skills to engage in a process of improvement to raise student
160 achievement. This begs the question, “How do districts build the capacity of school leaders to engage in data-driven decision making?” This qualitative study was conducted in Buck and D’Angelo Unified School Districts, both in the state of California, in order to answer the above question as well as related sub-questions: How is the need for professional development assessed? What types of training and support are provided to school site leaders? What evidence is there that leaders put new data-driven practices into place? What knowledge gaps still exist? The method of data collection to answer these questions involved interviews of district and school site administrators, observations of principal professional development, and document analysis. Through the above methods it was discovered that both districts since the inception of accountability mandates have understood the importance of utilizing data to engage in the improvement process. Of note, however, it was not until these districts procured the necessary assessment tools to produce reports that they were able to delve deeply into retrieving and analyzing data and developing focused action plans. Furthermore, both districts have extremely high expectations of their school site administrators’ use of data to drive decisions about instruction; however, both districts are aware that the level of knowledge and skills vary from one school site administrator to another. Answers to the research questions were achieved through protocols which allowed professional development processes to surface. Evidence revealed these districts to engage in: established methods to assess professional
161 development need, training and support of their principals, developed methods to measure implementation of new learning in data driven decision making, and recognized methods to evaluate existing knowledge gaps. The findings in this study have strong and relevant connections to research delineated in the literature review of chapter two. It is also important to note that the entirety of the data gathered and analyzed was examined within the framework of Data-Informed Leadership (Knapp et al., 2006). The intent of viewing this research in the context of this particular framework was to allow for a wider scope in understanding the complexity of educational leadership. Hence, it is believed that effective leaders not only gather and analyze data, but draw from their experiences and from accumulated knowledge of their organizational culture. In addition, these leaders also exhibit good judgment and wisdom prior to the development and implementation of a plan. Ultimately, the data revealed that mastery of datainformed leadership skills, by the superintendents, was the channel by which district and school site administrators were empowered to gain and implement new knowledge and then to extend it by effectively empowering others through distributive leadership. With that in mind, below are the salient points in this research study and how they connect to prior research.
162 Connections to Prior Research The results of this study engage existing literature on professional development for school site administrators in data driven decision making, leadership and learning organizations, and leadership development models. Chapter two examined research on professional development for school site administrators in relation to three crucial areas: Accountability in K-12 Education, The Role of DataDriven Decision Making in K-12 Educational Reform, and Professional Development to Build Capacity of School Site Administrators in DDDM. In the following sections, I will delineate the connection between these areas of research and the findings from this study. Accountability in K-12 Education The sub-section for Accountability in K-12 Education, found in the literature review, described consistencies between NCLB and California’s PSAA, specifically: student performance is based on standardized test results, a focus is placed on performance of numerically significant subgroups, and interventions are mandated to help those schools that are low-performing or program improvement (PI) schools. A study by AIR indicated AYP linked to NCLB was more challenging since it requires schools to meet proficiency targets versus California’s API target which is focused on growth. It went on to purport that a hindrance to school improvement was the professional development component: “. . . the district role (can hamper) school improvement, the influence of instructional coherence, and internal school capacity”
163 (AIR, p. I-5). Another substantial finding was the difference that emerged between schools that utilized an extensive amount of data and schools that did not. Growth schools were described as using data frequently and extensively to guide instruction, while low growth schools used it to identify students not performing well (p. IV-13). Findings from Buck and D’Angelo USD noted that overall there was consistent and progressive growth in schools within their districts, attributing growth at some of the schools to the implementation of effective DDDM practices. Through interviews, district administrators noted that initiatives were focused on targeting areas of challenge, with a keen focus on subgroups and content areas, coupled with a convergence and sharing of best practices. School site administrators utilized staff development meetings and grade level meetings to not only identify students not performing, but more importantly focused on methodologies for intervention, i.e. carving out time for intervention during the course of the day, sharing and agreeing to utilize best practices in instruction, identifying particular standards not mastered and consistently analyzing data to strategically regroup students based on need. Other research in the literature review, under Accountability in K-12 Education, noted the importance of adherence to federal and state expectations, but at the core of advanced practices in a school system is the interdependence between district and school sites. Knapp et al. (2006) demonstrate this relationship in their Data-Informed Leadership diagram, which establishes that it is incumbent upon school districts to create a comprehensive school system by building the capacity of
164 those concerned with DDDM through a culture of inquiry. In this study, it was noted that Buck and D’Angelo superintendents and district administrators have eliminated extraneous agenda items at principal staff development meetings and have targeted content to student achievement via school improvement. Principals were clear about the tapered message of student achievement and their responsibility to disseminate that message to the teachers as well as implement best practices of DDDM at their school sites. Furthermore, accountability is fostered at district professional development through an exchange of learning which extends vertically and laterally. This exchange has translated to accountability that extends to the student level. Both districts have couched the proficiency level of students in the idea that success occurs when students are keenly aware of their performance and progress. Thus data are broadcast through public announcements and the presence of school-wide and classroom data walls, extending the line of accountability to include district, principals, teachers, students, parents, and community. To further corroborate research in accountability, RAND researchers propose that the majority of educators believe data are useful for informing improvements in teaching and learning. However, it is noted that DDDM does not lead to effective decision making, but school improvement happens with the following actions: promotion of various data, allocation of time, training on data analysis and formulating action plans, identifying and implementing solutions, partnering with institutions that support data use, assigning people to filter data, and user-friendly
165 technology and data system. All of these actions exist within the elements that effect and contribute to Data-Informed Leadership, and were found in the research conducted in Buck and D’Angelo. Further to this, the acknowledgement and implementation of the above is spearheaded through the mastery of leadership strategies used by the superintendent and cabinet. The pace and tone for sustainability are set by district leadership. Sustainability is a result of institutionalization, involving a layered process embedded in structural routines and rituals within an organization (Datnow, 2005). District staff, at Buck and D’Angelo, is readily available to offer support and training at the district office and on-site. Furthermore, allocation of time for practices that move student achievement forward exist in both districts and are: principal meetings geared toward school improvement through the use of data analysis and the formulation of action plans; partnerships formed at both districts with outside consultants have provided professional development or a framework for school improvement through data use; and, implementation of a user-friendly data system, DataDirector. The superintendents’ messages reflect a priority of student achievement and that the job of the people within the district is to make this happen. The implication of practices, based on accountability, indicate that data driven decision making plays a tremendous role in the process of improving performance.
166 The Role of Data Driven Decision Making in K-12 Educational Reform The sub-section for The Role of DDDM in K-12 Educational Reform in the literature review indicates that DDDM is defined as: “. . . teachers, principals, and administrators systematically collecting and analyzing various types of data . . . to guide a range of decisions to help improve the success of students and schools” (Marsh et al., 2006, p.1). Further to this, Fullan (2002) stated, “Only principals who are equipped to handle a complex, rapidly changing environment can implement the reforms that lead to sustained improvement in student achievement (p.1).” The implication is that a process must be put into place for DDDM to play a prominent role in educational reform, but with a caveat – that principals have the leadership ‘know how’ to put this process in place and to make it sustainable. Buck USD has integrated the use of data with the consideration of other important knowledge and skills through the new principal’s evaluation system – a system based on CPSELs. Furthermore, professional development offered by the district for its principals is based on student achievement which directly correlates with data use or the use of assessment tools. One means of sustainability will manifest through the integration of district wide math benchmark assessments and then will follow with language arts benchmark assessments the following year (corresponding with the adoption cycle). D’Angelo USD has integrated sustainability in DDDM by undergoing a three year roll out process for each school site utilizing the Focus on Results framework. D’Angelo considers this a
167 comprehensive and systematic district wide professional development for the purpose of achieving equilibrium in the knowledge and skills of principals across the board. Professional development for principals and their leadership team occurs on a monthly basis, and this year a second principals’ meeting, eliminating ‘nuts and bolts’ information, will incorporate the Focus on Results framework for continuity sake. There is awareness at both districts that the role of data driven decision making is integral to the work of the district and its school sites. In a study about leaders in the schools and system, a case is made that DDDM should be the norm “. . . stating explicitly that data use is non-negotiable” (Datnow et al., 2007, p. 71). At Buck and D’Angelo, professional development, site visits, coaching and mentoring sessions (by district administrators and peers) are based on the analysis of data to guide school improvement. Action plans, evaluation, and monitoring of the improvement process at each school are based on the story told by the data. Schmoker (2006) notes, that from a system perspective, the superintendent can promote a culture of data use through the implementation of professional learning communities. Moreover, data discussions at professional development and with district administrators build capacity for school site administrators to continue the process at their respective school sites. Buck USD’s superintendent and cabinet have placed a deliberate focus on the use of data in their message, conversations, and method by which they enquire about the state of a school. Based on Buck’s culture of innovation, the superintendent has
168 very clear and high expectations for principals but does not dictate the professional development they seek for their site. Based on this expectation, all school site administrators interviewed reported receiving in depth professional development for their teachers, i.e. Effective Schooling Strategies by Dennis Parker, Professional Learning Communities by DuFour, and training by Marzano in cultivating and sustaining effective instructional strategies in the classroom. These professional development opportunities for those particular school sites have garnered results in student success as measured by a gain in API score and progress in closing the achievement gap for specific subgroups. Moreover, principals with best practices that have acquired success at their school site provide staff development at the principal meetings; thus, building a culture which values learning system-wide. Schmoker (2006) notes that districts who have more knowledge and skills capacity in DDDM have deepened and widened the scope of those learning through distributive leadership. From the interviews and observations at Buck and D’Angelo USD, it is clear that distributive leadership is being implemented and it has provided a foundation for collaborative data discussions school wide and at grade levels, wherein various data are discussed, analyzed, and action plans formulated. Best practices in distributive leadership were prominent at D’Angelo USD through Focus on Results, a district wide professional development, wherein the principal along with the entire leadership team attend monthly Focus on Results professional development. The process in and outside of these meetings, similar to Knapp et al.
169 (2006) cycle of inquiry, creates discussions which allow for adjustments in instruction and sharing of best practices. Most schools in D’Angelo USD have demonstrated consistent and steady gains in API and some schools have shown progress in closing the achievement gap for specific subgroups. Administrators at both districts, however, admit that there is still much work to do and learning gaps still exist amongst students, teachers, and administrators. Mac Iver and Farley (2003) note that among some of the best practices to build capacity within a school district includes implementation and training in data tools, effective DDDM in curriculum and instructional practice, and supporting schools in analyzing data and developing action plans. Both Buck and D’Angelo USDs are aware that the use of DDDM is a tiered process that exists at the district, school, and classroom levels. Superintendents at both districts are consistently engaged with district and school site administrators and sustain an unwavering message of student achievement based on data. Discussions between district administrators and principals during regular school site visits as well as observations (walk-through) center around: best practices in data use, best practices in instruction, and progress in the implementation of action plans. Both districts have staff which provides district and on-site training in data tools. Literature on the role of DDDM in K-12 educational reform by Coburn and Talbert (2006) describes the barriers in establishing a comprehensive system of DDDM: beliefs related to evidence/research and previous reform policies, varied
170 responses to accountability policy at the various levels of a school system. The issues are in the coexistence of conflicting views at the various levels of the system. These differences could hamper the use of data to make decisions that will ultimately improve the quality of education for students. Both districts indicated that there were barriers that hampered the process of DDDM. Through interviews with school site administrators, a Buck principal noted that camaraderie is difficult in a district that values innovation, noting that it sometimes brings feelings of isolation and an inability to share best practices across the district based on similar efforts. On the other hand, there are opportunities within the district for principals to present their findings of best practices to their colleagues at principal meetings. Furthermore, camaraderie has been formed through collaboration during principal meetings and amongst some of the school site administrators who meet on their own time to share leadership practices and strategies for implementing DDDM effectively at their school sites. D’Angelo, however, has worked the sharing of practices through its Focus on Results framework, wherein triads are formed amongst the schools. Schools formally visit one another, conduct walk-through to find evidence of best practices as it relates to a school’s instructional focus. Like Buck, some of D’Angelo’s school site administrators meet with their colleagues on their own time to discuss leadership practices to advance their teachers in refining their instructional practices for the purpose of school improvement. At times a barrier at D’Angelo USD is the teacher’s union, which has pushed back on components of the DDDM
171 process, the implementation of DataDirector is an example. Whereas the relationship between the teacher’s union and Buck USD is a partnership, a significant aberration from the typical district-union relationship, wherein the union supports the district with raising the expectations for teachers and offers heavy financial support for teachers’ professional development. Supovitz and Klein (2003) concluded that the American system of education does not allow for the structures and avenues to engage in the in-depth inquiry necessary to improve student learning outcomes. On the contrary, there were examples at both Buck and D’Angelo that indicate structures can and have been put in place to allow for in-depth inquiry. A principal in Buck USD notes that she has a ‘between the bells’ approach. Teachers are given time to do in-depth inquiry, looking at data and restructuring small group instruction in a grade level meeting, while students engage in a block of time spent in computer, PE, and music. Another principal in D’Angelo USD has taken an unconventional route to provide time for a teacher within a content area to serve as a resource specialist teacher, where one period a day would focus on academic content, making sure instruction, blue prints, and state test match. This was done first in English and Math, and then in Social Studies and Science. Principals who have mastered the leadership strategies necessary to ‘move’ a school to higher levels are evident at both Buck and D’Angelo. This includes the school site administrator’s ability to create a school
172 culture and increase the capacity of teachers to use data in making decisions about learning and instruction. Professional Development to Build Capacity of School Site Leaders in DDDM The subsection of the literature review, entitled Professional Development to Build Capacity of School Site Leaders in DDDM, indicates that in a number of studies administrators have been noted to make most decisions without the use of data (Coburn, Honig, and Stein, in press). In another study done by Corcoran, Fuhrman, and Belcher (2001) it states none of the districts that were evaluated utilized data to make decisions about professional development. Datnow et al (2007) adds routine of data use is not simply to analyze, interpret, and utilize data to formulate an action plan for the SSP, but to create a culture of data use at the school site. There was evidence in interviews, observations, and document analysis of Buck and D’Angelo USD that both districts understand that the most crucial investment is in the development of human capital. Both districts provide principals with professional development that focuses on the development of data literacy and analysis skills, which is also a significant element in Data Informed Leadership. Also significant is the element of building cultures of inquiry, wherein both districts have widened the scope of professional development to extend outside of the district office’s professional development center.
173 Superintendents and other district administrators consider cognitive coaching and mentoring of their principals a significant element in professional development – a necessary component to the implementation and sustainability of DDDM at the school sites. Peer-mentoring is also encouraged and occurs within both districts. As Darling-Hammond & Orphanos (2006) indicate, it is necessary for school site administrators not to simply be managers but organizational leaders with highly developed skills to understand and transform schools in organizational change. However, according to Schmoker (2006), school site administrators’ professional development has for the most part taken the form of transmitting procedural knowledge which often times does not pertain to practical application. This type of professional development leaves recipients in a vulnerable position, one that insinuates dependence on external sources rather than empowerment and ownership. Buck and D’Angelo utilize external sources; however, Buck couples external sources with a strong relationship between the district and the schools to establish internal accountability. Also, principals who have acquired the knowledge, skills, and mastery of leadership strategies are sharing their best practices through professional development provided to their colleagues. D’Angelo USD’s external source is professional development that introduces a framework rather than procedural knowledge, allowing for a weaning of the process to leaders in the organization who are being trained to become expert in the framework. An emphasis
174 to integrate professional learning with practical application is evident in both districts. Researchers, as indicated in the literature review, describe professional development to have relevance when it is intertwined with activities that reflect instructional improvement and are in harmony with the mission and vision of the organizational system. Learning should take place in a venue that creates individual and organizational habits and structures, encouraging learning on a continuum (Fullan, 1990). This requires an intense relationship between the district office and its schools, which was evident at both Buck and D’Angelo. Professional development was noted as one of the most important roles of the district. This is prominent in the strategic plans of both districts. In addition, both superintendents value and adhere to existing research in professional development by establishing the following: interpretation of data is a shared process to allow for sound instructional decisions, instructional strategies are highlighted through discussion and observation, team leadership is engaged through distributive leadership at school sites, decision making is based on evidence, and alignment of curriculum is addressed at the district and site level. All of the aforementioned processes can be implemented through structures; however, structures and processes alone must fall under the umbrella of building organizational culture simultaneously with professional development to assure harmony amongst all stakeholders. Of significant note is both superintendents
175 have presence, high visibility, and involvement in most professional development meetings. With high involvement and focus on professional development that concentrates on DDDM, Darling-Hammond (2002) asks, Are school site administrators actions based on intrinsic reasons or because they must conform to external pressure? She states that pressure must be accompanied by external mandates that incite a mind-shift that DDDM is part of systemic or organizational improvement that empowers educational leaders to have a collection of tools to plan and make decisions. In correlation with this research, it was substantiated through interviews and observations that pressure by the superintendents are applied through high expectations, performance results, and candid conversations rather than through castigatory messages. As a matter of fact both superintendents and their cabinet circulated throughout the entirety of the professional development sessions, engaging in the process, as well as connecting with principals through light-hearted conversations and/or ‘shop talk.’ Assistant superintendents in educational services and their directors continue this thread of expectation through timely and frequent site visits throughout the year to monitor and evaluate process as well as to offer support, coaching, and mentoring. The implication is that a principal will rise to the occasion through insights uncovered by new knowledge gained, experiences, collegial sharing, and proactive measures to construct meaning, as well as relationships that foster interdependence between district and school sites. District
176 superintendents and their district administrators set the pace and cycle for data literacy, inquiry, and action. Creating this type of understanding between district and site leaders requires trust and respect through relationship building and the mastery of strategies in Data-Informed leadership. Nonetheless, to this thesis is an antithesis. Based on a number of researchers from the literature review in chapter two, district leaders do more than make decisions based on data. Data, therefore, serve as reinforcement for improvement and is one of many skills that district leaders must have to move their organization into one that is a ‘learning’ organization. This idea is aligned to the framework of Data Informed Leadership, wherein a culture of inquiry manifests from the implementation of system learning (Knapp et al., 2006). While moving a district culture to one of inquiry (via relationship building), district leaders present evidence and utilize strategies to set or establish the purpose for change. In Buck and D’Angelo the presentation of evidence and the utilization of strategies are demonstrated through the conversations, planning, and decisions that occur through the district leadership teams, a close sphere of individuals that the superintendent works with on an intimate basis. Once a concise plan is formulated this is conveyed to all stakeholders via the strategic plan. The mode of communication with the public is much different than that with the leaders who will assist in making sure this plan manifests throughout the district; however, certain messages are consistent internally and externally. Buck USD’s superintendent espouses the district’s sense of urgency through specific initiatives that it
177 communicates through the ‘Unite Show’ and other media. D’Angelo USD’s superintendent communicates a ‘Good News’ and an ‘Urgency’ statement that is visible throughout the district and through various media. The mode and methods utilized by the superintendents and their sphere of individuals is a calculated one, characterized by intentional transparency and founded on evidence (data). Superintendents at both districts along with their cabinet relay the same message, a message spearheaded by the superintendent. This also serves as a role model for district administrators, school site administrators, and fosters external and internal accountability for the work to be done. It should also be noted the message given by the superintendent is expected to resonate from each district administrator and school site administrator in order for improvement to be system-wide. This was evidenced in the interviews and observations at both districts in the espoused message at principal meetings as well as at the school sites. Internal accountability exists through “learning at work — learning in context occurs, for example, when principals and their leadership teams are members of a district's inter-visitation study team for which they examine real problems — and the solutions they have devised — in their own systems” (Fullan, 2002, p. 19). This type of learning, evidenced at D’Angelo USD, fosters shared knowledge and commitment to improve the culture of inquiry within the organization. It also allows for a social setting that is conducive for performance monitoring by district administrators who facilitate in a triad, reinforcing DuFour and Burnette’s (2002) approach of “What
178 gets monitored, gets done.” It also serves as a support system within the organization for principals, allowing for problem-solving and strategizing. This process strengthens the interdependency between district leaders and school site leaders (Fink & Resnick, 2001). Professional development for school site administrators, based on the above research, is fragmented by the combination of state and district efforts, fragmented in that the state withholds funding to offer its own professional development, namely AB 430. District efforts, however, lend to a professional development model which allows for breadth and depth as a result of sequence and continuity. Based on this research, efforts in professional development for school site administrators should be localized to the district level wherein learning is customized directly affecting classroom instruction and improvement in student outcome. According to Elmore and Burney (2000) a recommendation for effective professional development for school site administrators is based on: monthly day long meetings; consistent principal site visits by district personnel to monitor progress and provide guidance; principal study groups; new principal support groups; and, strong organizational leadership through a comprehensive professional development plan. Based on the data collected, Buck and D’Angelo have implemented a rendering of the above. Below is a chart comparing findings across both the Buck and D’Angelo Unified School Districts as it pertains to effective practices in DDDM professional development for school administrators.
179 EFFECTIVE PROFESSIONAL DEVELOPMENT PRACTICES IN DDDM FOR SCHOOL ADMINISTRATORS Buck Unified School District D’Angelo Unified School District Superintendent presents evidence and uses DIL strategies to establish purpose for change. Need for PD assessed through gathering and analyzing data. PD plan based on needs assessment integrated into Data Informed Leadership (drawn from experiences and knowledge of organizational culture). Based on knowledge of district’s innovative organizational culture, a strategic approach was used. A variety of research based PD practices in DDDM exist and one element of district-wide practice implemented with a plan to add a second element on the following year. Superintendent and close sphere of district administrators and principals work intimately to formulate a concise PD plan to be executed. Mode and method of communicating PD plan by Superintendent and close sphere of district administrators is consistent, based on evidence, and intentional transparency. Organizational learning and culture of inquiry exists through internal accountability and fostered through a commitment to improve practices. Distributive leadership is modeled by Superintendent and District Administrators. Process of training, support, monitoring, and evaluation is consistent and conducive to strengthening inter- dependency between district and schools.
Superintendent presents evidence and uses DIL strategies to establish purpose for change. Need for PD assessed through gathering and analyzing data. PD plan based on needs assessment integrated into Data Informed Leadership (drawn from experiences and knowledge of organizational culture). Based on knowledge of an effective research-based framework for DDDM, Focus on Results, was implemented districtwide.
Superintendent with close sphere of district administrators and outside consultants work intimately to formulate a concise PD plan to be executed. Mode and method of communicating PD plan by Superintendent and close sphere of district administrators is consistent, based on evidence, and intentional transparency. Organizational learning and culture of inquiry exists through internal accountability and fostered through shared common knowledge and commitment to improve practices. Distributive leadership is modeled by Superintendent and District Administrators. Process of training, support, monitoring, and evaluation is consistent and conducive to strengthening inter-dependency between district and schools.
Table 7. Effective Professional Development Practices in DDDM for School Administrators
180 Evidenced, however, was a commonality between the superintendents at both districts, which made the difference in the effectiveness of preparing school site administrators in data driven decision making practices. It was the mastery of leadership strategies to move their respective organizations into a cycle of inquiry based on data-driven decision making.
Implications for Future Research The effectiveness to conduct professional development that raises the capacity of school site administrators to lead via data driven decision making has been highlighted in the research conducted at Buck and D’Angelo Unified School Districts as well as the research noted in the literature review. Whereas this study offered valuable insight in the framework, processes, and strategies to implement systematic and comprehensive professional development in DDDM for principals, future research is needed. Research is needed to intensify the understanding of how superintendents and their district administrators coach school leaders in the mastery of leadership strategies. Below are suggested areas for future qualitative research studies on professional development for school site administrators in Data Informed Leadership, a professional development that integrates DDDM with a mastery of leadership strategies.
181
•
Analysis in the consistency of district administrators to coach and assist principals in deepening the process of data-informed leadership
•
Analysis of principals within a district to establish a culture of inquiry through distributive leadership
•
Evaluation of a district’s process to design and monitor mastery of leadership strategies used by principals to increase teachers’ use of best practices in instruction
•
Evaluate school models, operating within the school day, to allow for professional development that includes: team meetings focused on instructional improvement through a cycle of inquiry (accessing and analyzing data, formulating and discussing actions, and opportunities to re-frame) The above areas to be researched would be best conducted through intensity
sampling choosing one unit of analysis (a district) which deems to have a comprehensive and systematic approach to professional development. The study would be across the board, with a greater number, if not all, district and school site administrators studied within the district, for an extended length of time through interviews, observations, and document analysis. This would allow for a more in depth study (Patton, 2002).
182 Implications for Policy and Practice Based upon the findings of this study in a merger with prior research, a district aiming to establish a comprehensive and systematic professional development program to support site administrators in effective use of data should consider the following: Mastery of Leadership Strategies in Data-Informed Leadership DDDM as a routine within a district does not exist in a vacuum, and certainly it is the ability to utilize data to inform leadership at the district, school site, and class room levels, which makes all the difference. The superintendent who sets the vision, tone, and dictates the pace of organizational learning must possess, master, and exercise the appropriate leadership strategies to launch the organization into a cycle of inquiry and sustain momentum, so that learning is continuous. This Mastery of Leadership Strategies was evidenced and role modeled by Buck and D’Angelo Unified School District superintendents and their respective cabinet members through the context of each district’s established organizational culture. Professional development to build the capacity of school site leaders in DDDM should include the Mastery of Leadership Strategies for Data Informed Leadership, which are: • Utilize evidence-based practices to establish the need for professional development
183 • Implement an effective agenda for organizational change through distributive leadership, which is first modeled by the superintendent and district administrators • Identify crucial actions for principals to be successful leaders, role model these actions through district professional development, and then monitor implementation during site visits • Utilize effective and consistent communication through cognitive coaching, mentoring, and problem solving to build collaboration, respect, trust, and to manage conflicts •
Evaluate, diagnose, and engage in cycle of inquiry to create high performance school-wide through a pre-determined construct
•
Develop enduring communication (with all stakeholders) to attain a firm grasp of the school culture to effectively bring organizational vision to reality, again role modeled at the district level
•
Synthesize all elements of a school’s performance, not just “bottom line” data to impact the culture and improvement process for student achievement
•
Allow time to identify and integrate new learning to build capacity schoolwide
184 Utilize evidencebased practices to establish the need for professional development (PD) in data-driven decision making (DDDM).
Superintendent and district administrators are present and engaged in all principal PD meetings and school site visits. Principals, district-wide, receive and implement DDDM framework via distributive leadership and professional collaboration.
Execute organizational change through modeling distributive leadership, which functions within the confines of a framework for professional collaboration.
Identify crucial actions for principals to be successful leaders, and then offer training, support, monitoring, and evaluation of implementation.
Superintendent and district administrators build capacity of principals by serving as facilitators, cognitive coaches, mentors, and consultants in PD and monthly school site visits. Investment is made in data infrastructure, access, and literacy.
Interdependency between district and school site leaders is prominent. Communication strategies are evident between principals and teachers. Symbiotic relationship in system learning between district and schools are marked.
Restructure communication through cognitive coaching, mentoring, and problem solving to build collaboration, respect, trust, and to manage conflicts.
Cycle of inquiry is part of the professional learning framework, which takes place at district and school site PD. Schools instructional leadership team use cycle of inquiry in grade level meetings.
Create a culture of inquiry by modeling the cycle of making sense of learning, taking action, learning from action, and problem re(framing).
District and schools have a pronounced message reflecting achievements and urgent needs for improvement. Schools and classrooms use various means of communic-ation, i.e. data walls, charts, chats, newsletters, and school events.
Calculate communication to impart a streamlined message of student achievement and to garner wide support to bring about organizational vision.
District celebrates wide range of endeavors. District effects and makes considerations and adjustments based on state and local policy environment. Leadership focus, core values, theories of action, data availability and literacy are prominent.
Synthesize all elements of a district’s performance, not just “bottom line” data to impact the culture and improvement process for student achievement.
Table 8. Mastery of Leadership Strategies in Data Informed Leadership
District administrators base PD for principals in DDDM on student achievement data and research-based best practices. A cohesive and receptive group of principals, districtwide, responsive to the implementation of a DDDM framework at their school sites.
`MASTERY OF LEADERSHIP STRATEGIES IN DATA INFORMED LEADERSHIP
Table 8. Mastery of Leadership Strategies in Data Informed
185 General principles can be extracted from this study, as its intent was to offer rituals and routines in effective professional development and to provide districts with a road map to support their school site leaders to confidently acquire the knowledge and skills to implement an evidence-based culture at their schools. Anchors for data-informed leadership (Knapp et al., 2006), with a resounding consistency in the leadership focus by superintendent, district administrators, and principals is imperative. Each tier, district, school, and classrooms, should understand the theory of action by which system learning occurs, whether it is through a predetermined school-wide professional development or through a district wide framework. Data literacy should be a priority through the investment in a data infrastructure, such as DataDirector, for use by all administrators and teachers. Furthermore, support and tools should be provided for effective implementation. Mastery of leadership strategies for Data Informed Leadership is vital to the effectiveness of the professional development programs. The conduits are: the ability to communicate and build trust and respect to bring together the support and genius of all stakeholders; the investment in data tools and support; and, the investment in third party sources to develop expertise in establishing an infrastructure which engenders system learning for district administrators, principals, teachers, and students, which includes, but is not limited to distributive leadership.
186 Conclusion The implementation of a systematic and comprehensive professional development to increase the capacity of school site administrators in DDDM indicates the level of performance based accountability expected through state and federal mandates. While this study provides well-timed and relevant information into the process by which districts provides school leaders with professional development in DDDM, a greater extent of research is necessary to ensure that mastery of leadership strategies to ensure proficient utility of DDDM is prevalent district wide. Visibly, the benefits of professional development are far reaching and necessary to develop the expertise of school site leaders’ ability to establish a culture of data literacy and data use to increase student achievement. At the end of the day, the level of expertise amongst school leaders will be a reflection of district leaderships’ ability to conduct effective professional development, and this will be measured by the degree in which a school effectively increases students’ performance levels.
187 REFERENCES American Institutes for Research (AIR). Evaluation Study of the Immediate Intervention/Underperforming Schools Program of the Public Schools Accountability Act of 1999 (2005). Retrieved February 26, 2008 from http://www.air.org/publications/pubs_ehd_school_reform.aspx Blasé, J. & Blasé, J. (2004). Handbook of Instructional Leadership: How Successful Principals Promote Teaching and Learning. University of Georgia, Athens, GA: Corwin Press. Bloom, G., Castagna, C. & Warren, B. (2003). More than mentors: Principal coaching. Leadership, May/June 2003. Sacramento, CA: Association of California School Administrators. Buck Unified School District (2008). Strategic Plan (2006-2010). Retrieved on May 25, 2008 from http://myBUCKusd.org/pdf/strategicplan.pdf Burke, J. C. (2004). Achieving accountability in higher education: Balancing public, academic, and market demands. In J. C. Burke (Ed.). The Many Faces of Accountability. (pp. 1-24). San Francisco, CA: Jossey-Bass. Butler, J.A. (1992). Staff Development. School Improvement Research Series (SIRS). Retrieved December 6, 2007 from http://www.nwrel.org/scpd/sirs/6/cul2.html California Department of Education (2007). 2007 Adequately Yearly Progress Report Information Guide. Retrieved March 2, 2008, from http://www.cde.ca.gov/ta/ac/ay/documents/infoguide07.pdf California Department of Education, Accountability Progress Reporting (2008), retrieved from: http://www.cde.ca.gov/ta/ac/ar/index.asp Coburn, C.E., Honig, M.I., & Stein, M.K. (in press). What is evidence on districts’ use of evidence? In J. Bransford, L., Gomez, D., Lam, N., Vye (Eds.) Research and practice: Towards a reconciliation. Cambridge: Harvard Educational Press. Coburn, C.E. & Talbert, J.E. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education 112(4), 469-495.
188 Corcoran, T., Fuhrman, S. & Belcher, C. (2001). The district role in instructional improvement. Phi Delta Kappan, 83(1), 78-84. Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage. Darling-Hammond , L. (2002). The right to learn (pp. 148-176). San Francisco: Jossey-Bass. Darling-Hammond, L. & Orphanos, S. (2006). Leadership Development in California. Retrieved on December 20, 2007, from: http://irepp.stanford.edu/projects/cafinance.htm Datnow, A. (2005). The Sustainability of Comprehensive School Reform Models in Changing District and State Contexts. Educational Administration Quarterly, Vol. 41, No. 1, 121-153. Datnow, A., Park, V. & Wohlstetter, P. (2007). Achieving with Data. Los Angeles, CA: Center of Educational Governance, University of Southern California. Dembosky, J., Pane, J., Barney, H. & Christina, R. (2005). Data Driven Decision-making in Southwestern Pennsylvania School Districts. Retrieved on December 20, 2007, from: https://www.rand.org/pubs/working_papers/2006/RAND_WR326.pdf DuFour, R., DuFour, R., Eaker, R., Many, T. (2006 ). Learning by Doing – A Handbook for Professional Learning Communities at Work. Indiana: Solution Tree Earl, L. & Katz, S. (2002). Leading schools in a data rich world. In K. Leithwood, Pl. Hallinger, G. Furman, P. Gronn, J. MacBeath, B. Mulforld & K. Riley (Eds.) The second international handbook of educational leadership and administration. Dordrecht, Netherlands: Kluwer. EdSource, California School Finance, Accountability Overview. Retrieved on March 27, 2008 from http://www.edsource.org/edu_acc.cfm Elmore, R. F. (2002). Bridging the gap between standards and achievement, Washington, DC, Albert Shanker Institute. Retrieved July 12, 2003, from http://www.nsdc.org/library/results/res11-02elmore.html
189 Elmore, R.F. & Burney, D. (2000). Leadership and learning: Principal Recruitment, induction and instructional leadership in Community School District #2, New York City. Pittsburgh, PA: University of Pittsburgh, HPLC Project, Learning Research and Development Center. Elmore, R. & Rothman, R. (1999). Testing, teaching, and learning: A guide for states and school districts. Washington, DC: National Academy Press. Fink, E., & Resnick, L. (2001, April). Developing principals as instructional leaders. Phi Delta Kappan, 82, 598-606. Fullan, M. (1990). Staff Development, Innovation, and Institutional Development. Changing School Culture Through Staff Development, edited by Bruce Joyce. Alexandria, VA: Association for Supervision and Curriculum Development (p. 3-25). Fullan, M. (2000). Change Forces: The Sequel. Philadelphia, PA: Falmer. Fullan, M. (2002). The Change Leader. Educational Leadership. Retrieved March 28, 2008, from http://www.sacle.edu.au/files/ChangeLeader.pdf Hale, E.L. & Moorman, H.N. (2003). Preparing School Principals: A National Perspective on Policy and Program Innovations, Institute for Educational Leadership, Washington, DC and Illinois Education Research Council, Edwardsville, IL. Heritage, M. & Yeagley, R. (2005). Data use and school improvement: Challenges and prospects. In Herman, J. L. & Haertel, E. H. (Eds.), Uses and misuses of data for educational accountability and improvement (pp320-339). Malden, MA: Blackwell Publishing. Knapp, M., Swinnerton, J., Copland, M. & Monpas-Huber, J. (2006). DataInformed Leadership in Education. Seattle, WA: Center for the Study of Teaching and Policy, University of Washington. 56 pages, from http://depts.washington.edu/ctpmail/PDFs/DataInformed-Nov1.pdf Leithwood, K., Louis, K.S., Anderson, S. & Wahlstrom, K. (2004). How leadership influences student learning. Retrieved on January 16,2008, from http://www.wallacefoundation.org/NR/rdonlyres/52BC34B4-2CC3-43D09541-9EA37F6D2086/0/HowLeadershipInfluences.pdf
190 Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education. RAND Corporation. Retrieved December 14, 2006 from http://www.rand.org/pubs/occasional_papers/2006/RAND_OP170.pdf Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass Publishers. National Staff Development Council (n.d.) Learning to Lead, Leading to Learn: Improving school quality through principal professional development. Retrieved, April 30, 2008, from http://www.nsdc.org/library/leaders/leader_report.cfm Oberman, I., Hollis, J. & Dailey, D. (2007). Bringing the State and Locals Together: Developing Effective Data Systems in California School Districts, SpringBoard Schools. O'Day, J., Bitter, C., Kirst, M., Carnoy, M., Woody, E., & Buttles, M., et al. (2004). Assessing California's accountability system: Successes, challenges, and opportunities for improvement (Policy Brief 04-2). Berkeley, CA: Policy Analysis for California Education. Retrieved March 26, 2008, from http://pace.berkeley.edu/policy_brief.04-2.pdf Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc. Peterson, Kent (2002). The Professional Development of Principals: Innovations and Opportunities. Educational Administration Quarterly, Vol. 38, No. 2 (April 2002) 213-232. Schmoker, M. (2006). Results Now. Alexandria, VA: Association for Supervision and Curriculum Development. Senge, P.M. (1999). The Leaders New Work, Building Learning Organizations. In Hickman, G.L. (Ed.) Leading Organizations: Perspective for a New Era (pp. 439-449). Thousand Oaks, CA: Sage Publications, Inc. Supovitz, J. A. & Klein, V. (2003). Mapping a course for improved learning: How innovative schools systematically use student performance data to guide instruction. Philadelphia, PA: Consortium for Policy Research in Education. Retrieved February 5, 2005, from http://www.cpre.org/Publications/AC08.pdf
191 Togneri, W. & Anderson, S. E. (2003). Beyond islands of excellence: What districts can do to improve instruction and achievement in all schools. Washington, DC: Learning First Alliance. U.S. Department of Education. (n.d.). Executive summary of the No Child Left Behind Act of 2001. Washington, D. C.: U.S. Department of Education. Retrieved February 24, 2008, from http://www.ed.gov/rschstat/eval/disadv/tassie1/index.html U.S. Department of Education (1998). San Francisco Unified School District, San Francisco, California: Profile of Award-Winning Professional Development. Retrieved on May 15, 2008, from http://www.ed.gov/inits/teachers/96-97/sanfran.html Woody, E. L., Buttles, M., Kafka, J., Park, S., & Russell, J. (n.d.). Voices from the field: Educators respond to accountability. Berkeley, CA: Policy Analysis for California Education (PACE). Retrieved March 3, 2005, from http://pace.berkeley.edu
192 APPENDIX A District Administrator Interview Protocol8 Note: I will begin the interview by informing the interviewee about myself and my study. An explanation will be provided regarding the tape recording of the interview and that their responses will be strictly confidential. They will also be informed that if there is something they would like to say off tape, I will oblige by stopping the tape midstream for their commentary. 1. Please describe your role in the district (background). 2. Please describe the support the district provides to school site administrators in terms of data use and/or data driven decision making (research question 1, 2, and 3). Probes: professional development, specialized data, reports, etc. How often? What are school sites expected to do with these data? 3. Please describe the district’s professional development plan to assist school site administrators in the collection and analysis of data? Technology, data the district collects, whether from schools directly or from other sources, such as state assessment data (background and research question 1, 2 and 4, indirectly, and 3 directly). Probe: How often and what types of professional developments are offered in data collection and analysis? Please describe the types of professional development offered to school site administrators in developing a culture of data use at their respective schools sites? How often do these types of trainings occur? Does the professional development plan include coaching or mentoring? 4.
How does the district measure the success or effectiveness of its professional development plan for school site administrators? (research questions 2, 4 and 5)
5. Are the selected school site leaders to be studied typical of other school site leaders in the district in terms of their establishment of school culture in data use? (research questions 2 and 4, and question 3 indirectly) Probe: How? If not, what are the challenges?
8
District Administrator Protocol Interview Questions were adapted from Dr. Amanda Datnow’s Protocol
193 6.
What is your role, as a district administrator, in building the capacity of school site administrators in the use of data and in establishing a culture of data use at their school sites? (research questions 1, 2, 3 directly, 4 and 5 indirectly) Probe: • Is professional development comprehensive and sequential? • Is professional development tailor made based on the needs of individual school site administrators • Is there a cycle of inquiry in the district’s infrastructure to allow for ongoing adjustments in professional development
7.
At a minimum, what do you expect school site administrators to use from the professional development offered as a means of student outcome improvement? (research question 2 and 4) Probe: • Are these expectations system-wide? • How do you expect school site administrators to execute the plan of improvement at their school site? • How will you know that the expectations have been met?
8. What problems have you seen schools run into in trying to use data for decision making? (research question 5) Probe: • How do you measure the need for additional professional development (research question 4)? • How are the problems addressed? 9. What problems have you seen school site administrators run into in trying to establish a culture of data use at their school site? (research questions 2 and 5) 10. Is there anything else you think might assist in raising test scores in the districts’ schools? (research questions 2 and 5) 11. What do you see as the next steps for the district’s professional development program in supporting schools’ use of data? (research questions 5) 12. What advice would you have for other district administrators about how they might provide a comprehensive professional development program to support site administrators in effective use of data? (research questions 1-4) Note: I will conclude the interview by thanking the participant for his/her time and valuable input.
194 Research Questions Cross-references of interview questions with the five research questions: 1. How do districts build the capacity of school leaders to engage in data-driven decision making? 2. How is the need for professional development assessed? 3. What types of training and support is provided school site leaders? 4. What evidence is there that leaders put new data-driven practices into place? 5. What knowledge gaps still exist?
195 APPENDIX B Administrator Interview Protocol9 Note: I will begin the interview by informing the interviewee about myself and my study. An explanation will be provided regarding the tape recording of the interview and that their responses will be strictly confidential. They will also be informed that if there is something they would like to say off tape, I will oblige by stopping the tape midstream for their commentary. 1. Please describe briefly the history of the school (background). 2. Please describe the district and school’s journey into data-driven decision making (all research questions, indirectly). Probes: history of data use, process of becoming more data-driven 3. Please tell me about your school’s performance in terms of accountability measures. (research question 3 and 4, indirectly) Probes: • How has the school’s API progressed and have targets been met? • How has the school’s AYP progressed and have targets been met? • Are there other district measures used to assess schools? If so, what are they and how has the school done? 4. If targets have been met/progress has been made: To what do you attribute your school’s growth? (research question 3 and 4, indirectly) 5. If data is mentioned: How are data used at your school site? (research question 3, directly) If data is not mentioned: From reading the data about your school on the California Department of Education website, data seems to play a part in the school’s improvement in accountability measures. How are data used at your school site? (research question 3, directly, and 4 indirectly) 6. What types of data are analyzed at this school? (research questions 1, 2, 3, and 4 indirectly) Probes: state assessments, district assessments; school, classroom, standardized, teacher-created
9
Administrative Protocol Interview Questions were adapted from Dr. Amanda Datnow’s Protocol
196 7. How are data analyzed at this school site? (research question 3 and indirectly 2 and 4) Probes: staff development, grade level meetings, administrative efforts, teacher efforts, professional learning communities 8. What are the district’s expectations of data use? (research questions 1 and 2, indirectly) Probe: Why do you believe that school administrators use data? (rewards/sanctions) 9. How does your district support the use of data by administrators? (research question 1 and 2, directly) Probes: Does the district provide you with professional development? If so, does the district expect administrators to analyze data individually? Does the district provide administrators time to analyze data together? If yes to either question: • Does the district provide options about use of this time or is an agenda or some other guidance/structure provided? • Do administrators provide the district with minutes or other evidence of their data use implementation, analyses, and discussions at school sites? • How does the district determine what type of professional development in data-driven decision making is needed by school administrators? How is the training and support provided (i.e., staff meeting time, site visit, mentoring, coaching, etc.) 10. Does the district provide a comprehensive professional development plan to assist school site administrators in data use, data driven decision making, and building a culture of data use at the school sites? (research question 2) If yes: Please explain the types of training provided. 11. Are there individuals at the district who provide support to you in the use of data and/or building a culture of data use at your site? (research question 2) If yes: Please describe what that person does. 12. What problems have you run into in trying to use data for decision making? (research questions 2, 3 and 4) Probe: What types of training and supports does the district provide? 13. Is there anything else you think might be making a difference/helping to raise test scores at this school? (research question 2 and 1, indirectly) 14. What do you see as the next steps in your school’s use of data? (research questions 2 and 3, indirectly)
197 15. What advice would you give to me, as a school site administrator, about how I might begin to use data or building a culture of data use for decision making to increase student achievement at my school site? (research questions 1 and 3, and 4 indirectly) Note: I will conclude the interview by thanking the participant for his/her time and valuable input. Research Questions Cross-references of interview questions with the four research questions: 1. How do districts build the capacity of school leaders to engage in data-driven decision making? 2. How is the need for professional development assessed? 3. What types of training and support is provided school site leaders? 4. What evidence is there that leaders put new data-driven practices into place? 5. What knowledge gaps still exist?
198 APPENDIX C Observation Protocol – District Professional Development Data Discussion Date: Time: DDDM Professional Development Discussion Observation List During the discussion, I will tape record and take running notes of the meeting. I will formulate specific records of whether the following items are discussed and/or used: •
Informal school/student assessment data
•
Formal school/student assessment data State assessment District-created assessment School-created assessment Publisher-created assessment
•
School Site Professional Development
•
Professional Learning Communities
•
Grade Level Meetings
•
Coaching/Mentoring
Other questions or items I will look for: Was there a formal agenda? If so, who generated it? Did administrators bring actual records and/or assessments to the discussion? Did administrators present and/or collaborate based on actual school site data?
District: Administrators present: Notes/Comments
199 APPENDIX D Preliminary Codes
1.
Assess Need for Professional Development
2.
Building Capacity
3.
Facilitators of Data Driven Decision Making (DDDM)
4.
Resources – Training and Support/Plan
5.
Process of DDDM
6.
Process of Data Analysis
7.
Technology/Support/Tools
8.
Culture of Data Use
9.
Types of Data
10.
Evaluation Tools
11.
Evidence of DDDM
12.
District Expectations
13.
Challenges
14.
Gaps
200 APPENDIX E Salient Codes
1.
Needs Assessment for Professional Development
2.
Leadership Capacity Building
3.
Support in Process of Data Driven Decision Making
4.
Accessibility to Tools
5.
Culture of Data Use
6.
Effectiveness of Professional Development
7.
District Expectations
8.
Challenges
