JELO Issue # 2 Spring 2015 - Central Valley Afterschool Foundation

Vol 1 • Issue 2 • Spring 2015

Researcher and Practitioner Dialogue: Building Networks and Systems

Examining Levels of Alignment Between School and Afterschool and Associations with Student Academic Achievement Expanding Common Core Learning Opportunities Through Professional Learning Communities in Afterschool Program Networks Science in California’s Public Afterschool Program: Exploring Offerings and Opportunities

JELO Editorial Board Members Jeff Davis Interim Director California Afterschool Network UC Davis School of Education Michael Funk Director, After School Division California Department of Education Helen Janc Malone, Ed.D Director of Institutional Advancement Institute for Educational Leadership Carolyn Martin Chin-Bow School-Age Division Director of Education The Children’s Aid Society Milbrey McLaughlin, Ph.D David Jacks Professor of Education and Public Policy, Emerita Stanford University Gil Noam, Ed.D Program in Education, Afterschool and Resiliency (PEAR) Harvard University Pedro Noguera, Ph.D Peter L. Agnew Professor of Education Steinhardt School of Culture, Education and Development, Executive Director Metropolitan Center for Urban Education New York University

Maria del Pilar O’Cadiz, Ph.D Project Scientist University of California, Irvine Diane Oliver, Ph.D Lecturer California State University, Fresno Elizabeth Partoyan Senior Fellow Forum for Youth Investment Sam Piha, M.SW Founder and Principal Temescal Associates Tia Quinn Founder and Executive Director BOOST Collaborative Nikki Yamashiro Research Associate Afterschool Alliance Nicole Yohalem Director, Road Map Project Opportunity Youth Initiative Community Center for Education Results

Welcome to the second issue of The Journal of Expanded Learning Opportunities (JELO)! The JELO is a peer-reviewed, online, open access publication of the Central Valley Afterschool Foundation. The mission of this journal is to foster the discovery, collection, and dissemination of scholarly research and deeper learning from a variety of disciplines related to out-of-school-time or expanded learning time. By publishing original empirical, practical, and theoretical manuscripts, the JELO promotes scholarship and consciousness of the ways in which young people’s engagement in expanded learning activities contributes to their learning and development. Ultimately, the JELO seeks to connect research and promising practices throughout the nation, with a particular focus on California, fostering a dialogue that engages researchers and practitioners in the field. This second issue of the JELO features a dialogue between Michelle Perrenoud, of Los Angeles County Office of Education, and Dr. Deborah Vandell, of University of California, Irvine, on the topic of the networks and systems which support the expanded learning field. We are also proud to feature three articles that focus on the value of networks and systems. Two articles discuss the importance of on-going communication between school day and afterschool providers to maximize student impact. Our third article articulates the importance of staffing structure, staff knowledge, and external partners as key factors associated with effective inquiry-based science opportunities in expanded learning programs. Since the launch of the first issue in the Spring of 2014, we have engaged in both statewide and national conversations about expanded learning. Much of the discussion has focused on how to bridge the divide between research and practice, as well as raising awareness of the JELO as a resource. We thank you for your commitment to expanded learning and your ongoing support of this thriving field.

Kimberley Boyer, Ed.D Chief Editor Central Valley Afterschool Foundation

Logan Robertson, Ph.D Associate Editor Bard College

Matilda Soria, Ed.D Associate Editor Reading and Beyond

Contents 01

Researcher and Practitioner Dialogue with Deborah Vandell, Ph.D (University of California, Irvine) and Michelle Perrenoud (Los Angeles County Office of Education)

04

Examining Levels of Alignment Between School and Afterschool and Associations with Student Academic Achievement Tracy Bennett, Ph.D. (University of California, Irvine in partnership with THINK Together)

23

Expanding Common Core Learning Opportunities Through Professional Learning Communities in Afterschool Program Networks Jazmine L. Frias, Ed.D, Brandon G. Palmer, Ed.D, Dylan M. Capilla, M.A (California State University, Fresno)

31

Science in California’s Public Afterschool Program: Exploring Offerings and Opportunities Ann House, Ph.D, Carlin Llorente, M.A., Torie Gorges, M.A., Patrik Lundh, Ph.D, Willow Mata, Ph.D (SRI Education)

RESEARCHER AND PRACTITIONER DIALOGUE with Deborah Vandell, Ph.D – University of California, Irvine and Michelle Perrenoud – Los Angeles County Office of Education The expanded learning field is making great strides to spread new knowledge and promote program quality to the field’s leaders and practitioners. To achieve this goal, many researchers and practitioners in the field are looking closely at building networks and systems. In this issue of the JELO, Dr. Deborah Vandell from the University of California, Irvine (UCI), and Michelle Perrenoud from the Los Angeles County Office of Education (LACOE) share their ideas on networks and systems. Dr. Vandell is a Professor and founding Dean of the UCI Department of Education. Ms. Perrenoud is an afterschool Project Coordinator for LACOE. Dr. Vandell is representing the researcher perspective and Ms. Perrenound is representing the practitioner perspective. In the expanded learning field (which includes afterschool and summer), there are a lot of discussions around building “networks” and designing “systems”. What do you see as the key differences between networks and systems? Deborah: For me, “networks” refer to connections between various afterschool programs that can provide technical assistance and mutual support. The CA Afterschool Network is an example of such a group. For me, “Systems” refer to inter-locking relationships that are connected across multiple levels. Building an afterschool system would focus on connections among afterschool and school, afterschool and parents, and afterschool and the community. Another way to think systematically is to look at the connections between


afterschool and various local, state, and national policy initiatives. Thinking systematically also leads people to consider how these external forces influence what happens within programs as well as individuals sites. Michelle: In my opinion, networks are groups of people with unique perspectives, like interests and shared passion linked together in a unity of purpose. Often these groups work to fill a need currently not being sufficiently addressed, lacking adequate funding/ resources, or in need of a formal system to support efforts. A network (such as a community of practice, local learning community, scholarly society) provides a forum for like-minded individuals to come together, share knowledge and learn from one another while discussing issues and ideas, challenging assumptions

The Journal of Expanded Learning Opportunities 1

and roadblocks, envisioning a brighter future, bridging current realities and creating informal, but shared, supports to address immediate needs, next steps, and discover promising practices. Additionally, the work of a network (such as an advocacy group) further informs practice, policy, and programs. To me, a system is a well-defined methodology or program used consistently in order to reach a designated end result. Within a system, a group of elements, or people, interact and function together as a whole as specified by formal (or informal) institutional procedures and processes. A system can also exist for the purpose of providing people access to resources and disseminating information to a larger group of individuals. Backing at the highest level within the authorizing agency is necessary, along with adequate financial support, intentional resources, and a thorough infrastructure. As a researcher or practitioner, who/what are your key networks? How do those connections help your work? Deborah: As a researcher, I am part of several networks that share information and findings about afterschool and summer learning. We sometimes engage in collaborative studies, such as the one that I conducted with research colleagues last year as part of the Power of Discovery: STEM2 Initiative. I have been a part of an early childhood research network for more than 20 years and much of the research by that group has been done collaboratively. There is power in networks! Michelle: My key networks include: California AfterSchool Network, Learning In After School and Summer, BOOST Collaborative, and UCLA Educational Leadership Collaborative. Each of these networks is a trusted convener of leaders, scholars, policymakers, practitioners and educators who work within, or are connected to, expanded learning. These networks provide robust opportunities for discourse, creating opportunities for innovation by connecting multiple levels of stakeholders and perspectives. Their work helps all those involved to better understand the nature of environmental shifts and proactive, dynamic, collective action. Each group also informs the field with resources, strategies, professional growth opportunities, and connections between research, leadership, and practice. These networks are a place where I belong as a learner and contributing member.


As a researcher or practitioner, describe a current system in which you are involved? How does this system help your work? Deborah: Systems are also powerful, but in a different way. Systems are characterized by their horizontal and vertical connectivity to other areas. Another important aspect of systems is the recognition that, when you change in one part of a system, other aspects of the system (and other connections) also are impacted. When one aspect of the afterschool system is changed (by funding, by quality standards, by what’s happening the schools), those changes ripple across the afterschool system. Michelle: I am involved in several systems that help my work and inform the field, two include: the CDE Statewide System of Support and the Region 11 System of Support. The most recent system design in which I am involved is the California Department of Education After School Division Statewide System of Support as a member of the Policy Guide Committee focusing on SB1221 (http:// partnerforchildren.org/wp-content/uploads/2014/07/ SB-1221-Hancock-Expanded-Learning-Enhances-Student-Success-7-24.pdf) and the implications to our work both statewide and locally. This committee was formed to develop, implement and maintain clear policies that support quality programs. Expanded Learning program policies will include the regulatory source and corresponding levels of program accountability. The Committee will also develop and implement a timely and collaborative process for reviewing, revising, and notifying the field of new and existing policies. These elements will interact and function together to provide a system of support to the field within a larger network of committees additionally focused on designing supports. I am also involved in coordinating and managing Region 11 Technical Assistance Coaching Site Visits in Los Angeles County on an annual basis to assess technical assistance needs of Expanded Learning programs. This system is comprised of two key elements to establish a comprehensive and coordinated coaching technical assistance structure to raise the quality of Expanded Learning programs in Region 11. The elements include: (1) a web-based tool and set of coordinated resources for site coordinators, program directors, and reviewers to utilize while entering specific data for discussion during the visit and (2) a cadre of trained reviewers who provide real-time coaching and support


that addresses needs and challenges of the program site at the time of the visit. The long-term benefits of this system include providing a framework and data the program site can use to inform their continuous quality improvement planning process.

Michelle: More than ever before, as the California AfterSchool Network states, we need to Lead Strategically, Work Collaboratively and Act Intentionally.

In general, what are the strengths and limitations of networks for expanded learning? What are the strengths and limitations of systems for the field?

• Operational definitions of networks and systems as they apply to our work.

Deborah: Both networks and systems have strength in their collaborative work. As research continues to grow in these areas and the more collaborative work that takes place among networks and systems, the less the limitations we will have. Michelle: While networks and systems are composed of differing elements, they can co-exist in a symbiotic relationship. Networks often mobilize faster than systems and are more adaptable due to infrastructure that is more fluid and adjustable. Systems can cause delays as a result of policies and protocol due to additional steps beyond what is done at the network level. In light of the transition taking place within our field, from afterschool to expanded learning both networks and systems are necessary to support the progression of our discipline and to achieve desired outcomes. Without the networking piece informing the interplay of systems development, we are not going to see quality improved just because systems are in place. The heart of a network (its people) work together to inspire a shared vision and to design systems that will enable others to act purposefully.

Items needing to be addressed:

• Understanding the nexus between networks and systems and how they work together. • What should a network be defined as in the field of Expanded Learning? • Starting to differentiate what networks look like and what purposes they serve and how and when they are useful in practice. • Systems supporting the advancement of Expanded Learning and the implementation of policy initiative like SB 1221 in California. • Creation of networks and systems supporting professional growth of site-based staff (site coordinators and frontline staff).

What more do we need to learn about networks and systems for expanded learning? What are the top three research/practice questions that the field needs to address at this time? Deborah: Much of what we need to learn about networks and systems connects with the research questions I’d like to see addressed in our field. One timely research question that I would like to see pursued is how we can better link early childhood, afterschool, and summer programs. Relatedly, I would like to see more systematic research identifying what are effective models for conceptualizing relations between afterschool and school. Finally, there remains a need to research ways in which afterschool networks can support professional development in afterschool programs that vary widely in their maturity and capacity.



EXAMINING LEVELS OF ALIGNMENT BETWEEN SCHOOL AND AFTERSCHOOL AND ASSOCIATIONS WITH STUDENT ACADEMIC ACHIEVEMENT Research-Based Article Tracy Bennett, Ph.D – University of California, Irvine (in partnership with THINK Together) Abstract In recent years, attention has been given to the academic impact of afterschool programs. Some schools collaborate with afterschool programs in an attempt to align the learning that occurs during the school day with the learning that occurs during afterschool hours, and thus maximize the potential to positively impact student academic achievement. However, very little research has sought to estimate the associations of alignment practices with academic achievement. This article proposes a conceptual framework of alignment between school and afterschool programs that incorporates measuring academic resources, communication, and a sense of partnership. It reviews the research on such practices, and synthesizes the work within the proposed framework. In statistical analyses, survey data were collected from principals and afterschool staff at 78 schools across 11 school districts in Southern California. Sites in the study were designated as highly aligned or misaligned. Results indicate a positive association between high alignment and academic achievement of students in both English Language Arts and Math, when compared with lower aligned sites. Significant negative associations were detected in Math when sites were misaligned. Results indicate the need for more research on the effectiveness of collaboration between school and afterschool. Keywords: alignment, misalignment, afterschool, academic achievement



Introduction Afterschool programming is playing an increasingly active role within the education system (Bodilly & Beckett, 2005). While the initial goal of afterschool programs was to provide a safe place for children to be after the school bell rings, accountability measures in education transformed the purpose of some afterschool programs from basic supervision to being responsible for contributing to the academic achievement of students (Mahoney, Parente, & Zigler, 2010. Specifically in the past decade, societal interest in afterschool programming has increased and efforts to expand the field have grown (Halpern, 2006). In an effort to promote development and learning during afterschool hours, schools may collaborate with afterschool programs. The overall goal of such efforts is to provide a complementary learning environment that provides students an opportunity to reinforce and practice skills (Afterschool Alliance, 2011). Complementary learning refers to staff at afterschool programs collaborating with staff at schools to align and maximize learning for students (Weiss, Little, Bouffard, Deschenes, & Malone, 2009). As the responsibility of afterschool programs grows, so does the need to examine which levels and types of alignment make programs effective at contributing to the academic success of students.

(2008) assert that afterschool partnerships can provide and support diverse, quality services for students that the school may not have the capacity to sustain during the regular school hours such as tutoring, academic enrichment or physical fitness. Collectively, a school and afterschool program can develop a set of common capacities that will enable afterschool programs to function as high-performing and adaptable entities that create new opportunities for students, and a plan for learning that extends beyond the hours of the traditional school day. Even though the afterschool hours can be used to promote learning, it does not necessarily mean that the afterschool program mimics the regular school day. Instead, afterschool programs may support academic achievement not by mimicking schools, but instead by supplementing the schools’ academic focus with a more integrative approach, targeting positive youth outcomes across multiple domains (Adger, 2001). Some research affirms this approach, indicating that these programs can benefit students by decreasing their risk-taking behaviors and supporting the development of a range of non-academic competencies that in turn support academic learning and achievement (Hall, Yohalem, Tolman, & Wilson, 2003; Honig, Kahne, & McLaughlin, 2001).

For the purposes of this research, the term “alignment” is used to describe specific collaboration practices between staff at afterschool programs and schools that attempt to coordinate learning as students transition from the regular school day to the afterschool program. These intentional alignment efforts have several dimensions including the degree of effort by either or both entities, resources available, and time dedicated to collaboration. The study presented here conceptualizes alignment across such varied levels as academic resources, communication and partnership, and then empirically examines the relationships between varied levels of alignment and student achievement.

An aligned afterschool program that is connected to the school would maximize learning for students (Weiss, et. al, 2009). To date, the hypothesis that high alignment between schools and afterschool programs will yield benefits for student academic achievement has not been tested. In such an effort, the main purpose of this research is to:

Context of Alignment Between School and Afterschool Staff

3. Provide recommendations for future research in afterschool

School staff collaborate with their afterschool programs and other community-based organizations to create programs that can reach student achievement goals by capitalizing on each entity’s assets, resources, and perspectives (Ashcraft, 2002). There are multiple potential benefits to partnerships with schools that focus on afterschool time. Little, Wimer, and Weiss


1. Develop a conceptual framework of alignment between school and afterschool that accounts for various levels of alignment practices 2. Empirically test a measure of alignment as a predictor of student academic achievement

Conceptual Framework for Measuring Alignment A growing body of research documents positive outcomes associated with afterschool programming. When compared with non-participants, afterschool program participants in several recent studies demon-


strated significant positive changes in academic achievement (Lauer et al., 2006; Reisner, White, Birmingham, & Welsh, 2001; Klein & Bolus, 2002; Vandell, Reisner & Pierce, 2007). Academic outcomes include improvements in reading and math scores, higher rates of homework completion, and higher grades. Positive socio-emotional and developmental outcomes have also been associated with afterschool programming. Recent studies have reported significant findings associated with lower levels of behavioral problems, gains in social competencies including confidence and leadership skills, and increased levels of civic engagement (Durlack & Weissberg 2007; Lauer et al. 2006; Harvard Family Research Project, 2006 Mahoney et al., 2010 Riggs & Greenberg 2004; Vandell et al., 2007. The positive results reported in these studies indicate that attendance in an afterschool program was able to significantly predict positive changes in student-level outcomes. A more limited number of studies have examined the relationships that exist within programs that can also affect student outcomes. For example, adult-child relationships are documented in some studies as a key element of a high-quality afterschool program, and one that can impact student achievement (Smith et al., 2010). While adult-child relationships may be an important component of a high-quality afterschool program, most research tends to overlook the adultto-adult relationships; specifically the relationship between afterschool program staff and school administrators. There are no experimental studies on the impact of alignment practices between schools and afterschool programs affecting student academic achievement, yet evidence of such positive impacts will likely become an essential element of program sustainability (Stonehill et. al, 2011). There are a few studies that correlate alignment practices with academic outcomes, yet there are no conclusive links between alignment practices and academic achievement. In their book Afterschool Education: Approaches to an Emerging Field, Noam, Biancarosa, and Dechausay (2003) present a framework that delineates levels of relationships that can occur between afterschool programs and schools. Noam et al. view afterschool programs as a bridge between different worlds within a student’s life; connecting the learning that occurs during school with the learning that occurs afterschool. These different levels of alignment are indicative of the


intensity with which schools and afterschool programs implement certain practices or not. The strength of these practices places a program on a continuum of alignment. When a program is completely separate from a school, Noam consider it a self-contained program. In this case, both the program and the school make no effort to connect with each other. On the other end of the spectrum is a unified program that is indistinguishable from the regular school program. It would be a true extension of the school day, and contains no individualized characteristics distinct from the school. Based on the framework Noam et al. (2003) provide for alignment, an integrated afterschool program would be considered highly aligned for the purposes of the current study. An integrated afterschool program makes it an organizational priority to allow time for collaboration between staff and school administrators. Furthermore, the school administrators invite the afterschool program staff to attend various meetings such as leadership, staff, and parent meetings. Also, school administrators are involved in afterschool program planning. While this framework is foundational in understanding alignment as a relationship between school and afterschool programs, it does not offer specific practices of alignment that can be implemented by practitioners or studied by researchers. Expanding the Framework Building on the framework provided by Noam et al. (2003), this research focuses on three particular alignment practices: academic resources, communication, and sense of partnership, which can be measured and studied. These areas of practice can vary depending on the degree of effort and intention that both the school and the afterschool program put into the relationship and alignment practices. The framework demonstrates that as the intensity and intentionality of alignment practices increase, an afterschool program would progress from self-contained (not aligned) to integrated (highly aligned). A unified program is not included in the framework because it implies no distinction between a school and an afterschool program and is therefore inapplicable to this research. The types of practices within each level of alignment are academic resources, communication, and partnership. Next, these three proposed areas of alignment are discussed.


Academic resources This framework proposes that a key alignment practice is to coordinate the use of academic resources between the school and the afterschool program to ensure there is alignment of academic goals and strategies when students transition from school to afterschool. Curriculum design should be intentionally aligned to program goals (Huang & Dietel, 2011). Most often, schools have access to materials, facilities, and resources that could be made available to the afterschool program. A measure of alignment on academic resources would include an examination of which resources the afterschool program has access, and which resources are used in programming such as curriculum materials, computer lab access, curriculum pacing guides, student level data and teacher guides.

programs into a measure that is applicable across varying programs and contexts, researchers can establish a foundation for future studies on alignment. This can provide valuable insight into the varying degrees of alignment that occur and the potential impact on student achievement. Table 1 provides examples of survey items under each component that a study could potentially use. Researchers can examine the components individually, or as a comprehensive measure. Furthermore, researchers can use the measure as a statistical predictor to student academic outcomes. Results would be comparable across multiple programs and contexts, which is a feature that is currently lacking in afterschool alignment research.

Communication The framework discussed here suggests that the frequency and purpose with which schools communicate with afterschool programs is an important component of alignment. Regular and intentional communication provides school and afterschool staff with access to common information that will improve how each supports the development of students (Bosland, Rucker, Cohen, Fischer, & Rogers, 2012). A measure of alignment would need to be able to examine the frequency and content of such communication efforts. Based on the alignment practices discussed here, a measure of communication should address specific types and frequency of communication topics including academic concepts, student needs, facility use, program and school policies, and goal setting.

Alignment is conceptualized as a relationship between the school and afterschool program, and therefore, both entities must be involved in the research. In its truest form, alignment entails a sense of understanding and dedication from both the school and afterschool program. Understanding the differences and varied approaches from both sides of the relationship will produce the most useful research for the field. Researchers must implement a measure of alignment with the intent to measure the efforts put forth by both the school and afterschool program.

Sense of partnership The framework proposed here notes a sense of partnership as a component of alignment between school and afterschool. This is especially important since many school-based administrators consider afterschool programs as an afterthought and may not connect it to the traditional school day (Norris-Holmes, 2008). Alignment includes the need for a strong relationship between the school and afterschool program that incorporates trust, shared vision, and common goals for students. Therefore, a measure of alignment should include an assessment of the relationship between the school and afterschool program, how that relationship builds alignment, and the process by which efforts are translated into programmatic efforts afterschool.

Measuring Alignment

Alignment is conceptualized as principals and afterschool site coordinators being in agreement about academic resources, communication, and partnership with the afterschool program at their school. Alignment is measured through surveys administered to principals and site coordinators of an afterschool program. A school is considered highly aligned if both the principal and the site coordinator report that they work together closely on all three types of alignment practices. Misalignment is conceptualized as those schools in which principals and site coordinators are not in agreement on all three of those categories. This study tests this measure of alignment as a predictor of student academic achievement. In this study, a survey measure of alignment was administered in 11 school districts to determine if alignment is associated with student achievement. All of the afterschool program sites in this study are located in southern California. The study addresses the following questions:

By incorporating these three proposed elements of alignment practices between school and afterschool



1. Is alignment / misalignment between principals and site coordinators associated with student academic achievement scores in English Language Arts and Mathematics? 2. Is the perception of alignment by a single respondent (principal or site coordinator) associated with student academic achievement scores in English Language Arts and Mathematics? 3. Is the interaction of program attendance and alignment / misalignment associated with student academic achievement scores in English Language Arts and Mathematics? Methods Participants Survey sample. Surveys were administered to principals and site coordinators at 116 schools across 11 districts. Eighty four principals and 91 site coordinators completed the survey. In 78 schools, surveys were completed for both sets of respondents (68% overall response rate). Student sample. The sample consisted of afterschool program participants within Grades 3 through 8 at 78 schools (67 elementary and 11 intermediate) across 11 districts. Table 2 displays the demographic descriptive statistics of the student sample. After accounting for missing data, the student outcome sample consisted of 8,129 students. A majority of the student sample was Hispanic (54%), and there were more females than males (56%). Measure Alignment was assessed with items from a survey developed by Vandell and colleagues (2004). The survey contained three subscales: Academic resources, Communication, and Partnership. The reliability analysis conducted on each scale yielded similar alpha levels to the first study. Reliability analysis yielded lower alpha levels for Academic Resources (Principal α=.76, Site Coordinator α= .71), high alphas for Communication (Principal α= .89, Site Coordinator α= .84) and also high alpha levels for Partnership (Principal α= .91, Site Coordinator α= .87). 2011 California Standards Test (CST) scaled scores These variables come in the form of a scaled score, ranging from 150 - 600. The scaled score was the unit of analysis. Scaled scores for both English Language


Arts (ELA) and Math were used as outcomes. Scores for ELA range from 327.13 to 357.18, and scores in Math range from 332.29 to 381.48. The mean scaled score for the entire sample in ELA was 343.39 (SD=52.72) and 359.78 (SD=74.59) for Math. Covariates Student-level Students’ CST scaled scores from the prior school year were included in the analysis in order to control for prior achievement level. In addition, the grade level of each student from the 2011 school year was obtained from district records. The data set includes students in Grades 3 through 8, as those are the students who would have two years of CST data to provide a control for prior achievement. Each student was dummy-coded as male or female, based on district records from the 2011 school year (Male=1, Female=0). Student records from the districts indicate whether the student is of Hispanic descent or not. The variable was dummy-coded (Hispanic =1, Not Hispanic = 0). Lastly, afterschool program attendance rates for students was added as a covariate, in the form of a continuous variable (ranging from 1 to 180 days). Site-level Three school-level covariates are used in the analyses. First, as a potential indication of economic status, the percent of students who receive free/reduced lunch (FRL) at the school was included as a covariate. Eightytwo percent of the entire sample receives free/reduced lunch. Secondly, the percentage of students at each school who are designated English Language Learners (ELL) was used as a covariate. Thirty-three percent of the sample is ELL. Both the FRL and ELL covariates were entered into the models as continuous variables for each school (representing the proportion of each student population). Lastly, as an indicator of school-level academic achievement, each school was coded as “in” Federal Program Improvement status or not. Records from the California Department of Education indicate whether the school is or is not, and the variable is dummy-coded (In Program Improvement = 1, Not in Program Improvement = 0). Within the entire sample, 74% of schools were in program improvement. Program Attendance This study includes afterschool program attendance as a moderator variable. Table 3 displays the mean and


ranges of afterschool program attendance for each district. The average afterschool program attendance for the entire sample is 87 days (SD=66.03). By creating an interaction variable (program attendance x alignment), analyses can explore whether program dosage moderates the association of alignment or misalignment on student academic achievement. An interaction variable will assess whether the association of alignment on student academic achievement is different for a student who attends a highly aligned or misaligned afterschool program at a higher rate. The interaction variable was added to each regression model in this study. Analysis Multiple regression analyses were used to assess the relationship between alignment and student academic achievement scores. For greater detail of preliminary analyses that created the alignment variables, please refer to Appendix A. Since the intended audience of this research is geared toward practitioners, unstandardized coefficients were reported because the outcome variables were already standardized in terms of interpreting the results. For example, a significant coefficient of .25 can be interpreted as a .25 change in score on the CST test. It is believed that this is a more interpretable version of the results for the intended audience. The following sections detail the rationale of each approach and how the data for each analysis were prepared. The subsequent section will document the results of the analyses.

regression models for the misalignment analyses are as follows: Student Achievement = a + b12010 Student Achievement + b2Male + b3Grade + b4Hispanic + b5FRL + b6ELL + b7PI + b8ProgAttn + b9Misalignment + b10AcadRes + b11Comm + b12Partner + b13Misalignment x ProgAttn Individual Reports of Alignment In addition to the high and low alignment predictor variables mentioned earlier, this study also examined the association of alignment at those sites in which only one respondent reported alignment efforts. For example, a site can have an afterschool program site coordinator that reports alignment efforts, yet the principal at the same site does not. This is a potentially important distinction because there may still be an association of alignment on student academic achievement, even if alignment efforts were one-sided as when there are alignment efforts made by one side, but with varied response by the other. Therefore, in addition to the alignment and misalignment analyses, additional regression models will be used to account for these types of sites. Those models are as follows: Student Achievement = a + b12010 Student Achievement + b2Male +b3Grade + b4Hispanic + b5FRL + b6ELL + b7PI + b8ProgAttn + b9PrincipalAlign + b10ASPAlign + b13PrincipalAlign x ProgAttn + b13ASPAlign x ProgAttn

High Alignment Alignment scores were computed for each site (overall and by each survey scale). The alignment predictor variables indicate whether a site is designated as having “high alignment” between principal and afterschool staff reports. With the addition of school-level controls and the program attendance interaction, the regression models for the alignment are as follows:

To account for variability within schools, analyses were clustered at the school level. For each analysis, an individual report of alignment variable is entered as a predictor, along with the interaction variables of individual reports and program attendance. The next section reviews the results.

Student Achievement = a + b12010 Student Achievement + b2Male + b3Grade + b4Hispanic + b5FRL + b6ELL + b7PI + b8ProgAttn + b9Alignment + b10AcadRes + b11Comm + b12Partner + b13Alignment x ProgAttn

Table 4 displays the bivariate associations between each covariate and the student outcome variables in both ELA and Math. As Table 4 shows, there are significant correlations for a majority of the covariates with both outcome variables. Notably, afterschool program attendance was positively correlated with both outcome variables. In terms of the alignment predictors, the correlations were mixed in significance, depending on the subject. In terms of high alignment, only the partnership scale was correlated with ELA student achievement. For Math achievement however, all scales

Misalignment The misalignment predictor variables are a difference score between principals and afterschool staff, and a higher difference score indicates a higher rate of misalignment between the two respondent groups. The


Result


(academic resource, communication and partnership), in addition to high alignment were correlated with the outcome variable. For misalignment, every scale and overall score were negatively correlated with both outcome variables of ELA and Math. In order to examine the associations of alignment and misalignment on student achievement, five regression models were run within each subject area (ELA and Math). The first model within each analysis represents the associations between only the covariates and student academic achievement. The second model within each examines the associations between afterschool program attendance and student academic achievement. The third and fourth models examine the associations between alignment and student academic achievement; with the third model including the overall alignment score, and the fourth model only including the scales of the survey (academic resources, communication and partnership). As mentioned earlier, separate regressions were run in order to account for the linear dependence between the scales and the overall score. The fifth (and last model) in these analyses examines the interaction of program attendance and overall alignment. High alignment The first analyses of this study examined the association of high alignment between principals and afterschool staff to student academic achievement outcomes in English Language Arts and Mathematics. Table 5 displays the findings for Alignment and English Language Arts. As with the first study, it was predicted that high alignment sites would be linked to higher scores for students, when compared to sites with lower alignment. As can be seen in Model 2, there was a significant positive association between afterschool program attendance and student academic achievement in ELA (b= 0.016, p< .05). Furthermore, results from the regression analysis (Model 4) indicate a significant positive association of the subscale of Academic Resources on English Language Arts (b= 0.036, p