The British Educational Research Association (BERA) Annual Conference Institute of Education, University of London 5th – 8th September 2007 Conference Session Number 6.12 Symposium 8719 Friday 7th September 2007 15.00 – 16.30 Logan Hall The Impact of Research on Policy
BERA SIG: Educational Research and Educational Policy making
For further information about this session contact: Professor Edward Melhuish Brenda Taggart Professor Iram Siraj-Blatchford Professor Peter Tymms September 2007
[email protected] [email protected] [email protected] [email protected]
Contact information for the presenting authors Professor Edward Melhuish Executive Director National Evaluation of Sure Start Birkbeck, University of London Institute for the Study of Children, Families and Social Issues 7 Bedford Square London WC1B 3RA 00 44 (0) 207 079 0834 / email
[email protected] Brenda Taggart Principal Investigator and Research Co-ordinator EPPSE Project Room G2 Institute of Education, University of London 15 Woburn Square London WC1H OMS 00 44 (0)207 612 6219 / email
[email protected] Professor Iram Siraj-Blatchford School of Early Childhood and Primary Education Room 101 Institute of Education, University of London 15 Woburn Square London WC1H OMS 00 44 (0)207 612 6218 / email
[email protected] Professor Peter Tymms Director Curriculum, Evaluation and Management Centre University of Durham Mountjoy Research Centre 4, Durham University, Stockton Road, Durham, United Kingdom, DH1 3UZ. 00 44 (0) 191 33 48413 / email
[email protected])
Contents
Page Number
Symposium Overview
1
Presentation One: The role of research in the development of Sure Start Abstract Paper
2 3
Presentation Two: Influencing policy through research in Early Childhood Education Abstract Paper
12 13
Presentation Three: An Evaluation of the Foundation Phase in Wales Abstract Paper
21 22
Presentation Four: The First Six Years at School Abstract Paper
30 31
Symposium Overview The Impact of Research on Policy The election of the New Labour Party’s (1997) heralded a commitment to policy reform through a focus on education. The PM suggested that reforms should be informed by research evidence resulting in a number of high profile evaluations/reviews (NESS 2003, Tomlinson 2002). The intervening years have seen the development of policies (Working Family Tax Credits etc.) that have purported to be steered by research evidence. The relationship between research and policy-making has never been more important. In her Professorial lecture (2004), Lesley Saunders (policy adviser, GTC) gave arguments for research influencing policy-making in the public sector. This symposium brings together well established researchers who have managed high profile research that has impacted on policy development. The first paper reports on the Sure Start initiative, a cornerstone of policy in early years. It explores the impact of local programmes on services to 600-800 children under four, through the National Evaluation of Sure Start (NESS). The evidence led to a fundamental shift in the development of Children’s Centres and answers research questions about service level provision for young children. The second paper uses an early years longitudinal study (EPPE 3-11) to explore the ‘evidence base’ for UK policy in pre-school. EPPE studied project the effects of pre-school on 2,800 children from a wide range of social backgrounds. The paper identifies policy questions that have been answered by research and show how this has ‘shaped’ policies and legislation. The third paper reports on national reform in Wales and the evaluation of the Foundation Stage (MEEIFP project set in 41 early years settings in the 22 Welsh LAs). The evaluation was funded to support evidence based practice and answer policy questions about one country’s approach to educating its young children. The paper summarises recommendations across a range of domains (curriculum, pedagogy etc.) The fourth paper uses the PIPS data, collected since 1991 to examines the cumulative longterm impact of successive years of high quality provision on children’s attainment. The results have important consequences for policy, suggesting that teachers are at the heart of educational success, not policies or leaders and classroom and teachers should be at the forefront of initiatives and reform. These four studies combine a range of methodologies and statistical analyses and are important given the current policy agenda They are relevant to all BERA members who are concerned with the impact research has on policy at a range of levels.
1
The Role of Research in the Development of Sure Start Paper 1 Abstract The ‘Cross-Departmental Review of Services for Young Children’ (1998) reported that, “The provision of a comprehensive community based programme of early intervention and family support which built on existing services could have positive and persistent effects, not only on child and family development but also help break the cycle of social exclusion and could lead to significant long term gain to the Exchequer.” This Review led to the Sure Start Initiative, which became a cornerstone of the government’s endeavours to reduce child poverty and social exclusion. The initiative had set up 260 Sure Start Local Programmes (SSLPs) by 2001, which were to be expanded to 524 programmes within another 2 years. SSLPs aimed to enhance the health and development of children under four and their families living in deprived communities. SSLPs are charged with improving services and creating new ones in typically small geographic areas averaging just under 13,000 people, including on average 600-800 children under four years of age. SSLPs represent an almost unique intervention in being area-based, with all children under four and their families living in a prescribed area being automatic members of the Sure Start community, with the right to a say in the services provided. This area-based strategy allowed the relatively efficient delivery of services to those in need (i.e., living in deprived areas) without introducing stigmatisation of those receiving services. Disadvantaged areas are selected but within the area the service is universal. As a consequence of their local autonomy SSLPs did not have a prescribed “protocol” of services that promotes adherence to a prescribed model. Instead, each SSLP had autonomy to improve and create services, with general goals and some specific targets but without specification of how services are to be delivered. This contrasts markedly with interventions with clear models of provision and demonstrable effectiveness that formed the basis of the research evidence justifying the creation of Sure Start. It appeared that while the research evidence was critical in winning the argument for increased Early Years expenditure it was largely overlooked in the detailed planning of programmes. Treasury insisted upon a rigorous evaluation of Sure Start. After a competitive tender the National Evaluation of Sure Start (NESS) was commissioned in early 2001 to undertake a multifaceted evaluation of SSLPs, addressing the nature of the communities in which SSLPs were situated, the ways in which SSLPs were implemented, the impact of SSLPs on children, families and communities and the cost-effectiveness of SSLPs. By 2004/05 research evidence from NESS and the EPPE project was available. This evidence led to a fundamental shift in SSLPs towards all Sure Start programmes becoming Children’s Centres. The story of how this happened will be discussed in this paper.
2
The Role of Research in the Development of Sure Start Paper 1 Author: Professor Edward Melhuish On May Day 1997 Labour won a landslide election victory, returning to power for the first time since 1979 with a 177-seat majority. The end of 18 years of successive Conservative governments represented an opportunity to change to policies that would be seen to place the improvement of people's lives at the centre of government strategy. On May 6th, Gordon Brown, the Chancellor of the Exchequer, enacted a policy at the forefront of macroeconomic thinking by giving the Bank of England operational independence from the government, including freedom to set interest rates. One consequence of this change was that a number of high-level economists at Treasury had time to pursue other tasks. As well as being the Government’s foremost economic thinker, Gordon Brown also embodied its social conscience. He wanted the Government to break the cycle whereby disadvantaged children relived their parents’ experiences of poor education, physical ill-health and poverty. Hence, when this surfeit of economists became available he gave one of them, Norman Glass, the job of exploring what policies might fulfil this aim. A fresh look at children’s services At this time the notion of “joined-up” services was also in vogue within the government. Regarding services for children, Prime Minister Tony Blair commented in the Foreword to the Comprehensive Spending Review (Blair, 1998) that “We have looked at key problems across government. The old departmental boundaries often do not work. Provision for young children-health, childcare, support--will be co-ordinated across departments so that when children start school they are ready to learn.” In preparation for this review which set out government spending plans for the following three years, six cross-cutting reviews were established. The need for cross-cutting reviews arose from concerns that some policy areas either might be ignored if left to a sole department or might be tackled inadequately, and such reviews were meant to facilitate “joined-up” government. It was agreed by Ministers that there should be such a review of services for young children. Provision of services appeared, in many cases, to be failing those in greatest need but there was evidence from programmes in the Unites States like Head Start and the Perry Pre-School programme (Barnett, 1995), as well as from experimental programmes in the UK, that comprehensive early years' interventions could make a difference to children's lives. The “Cross-Departmental Review of Services for Young Children” was established, charged with considering all available evidence and producing policy recommendations for counteracting the cycle of disadvantage. In particular, the review was to assess whether greater emphasis on preventative action and a more integrated child-centred approach to service delivery could help cut the costs of crime and unemployment, and reduce the need for extra help for individuals at school and in later life, by helping parents, carers and communities provide the best possible start for children. The review was designed to: (1) look at the policies and resources for children aged seven and under, in order to ensure effectiveness in providing preventative action and the necessary support to ensure the development of their full potential throughout their lives; (2) consider whether the multiple causes of social exclusion affecting young children could be more effectively tackled at the family and community levels using an integrated approach to service provision; and (3) to take account of policy developments in initiatives elsewhere (HM Treasury, 1998b). The review involved 11government departments together with the Social Exclusion Unit and the Number 10 Policy Unit, with a Steering Group comprised of ministers from all the departments. It gathered information through meetings with pressure groups, representatives of service providers and service users, written submissions, papers commissioned from experts, and a series of open seminars. Visits were made to various projects offering services to young children and their families and the final report to the 1998 Comprehensive Spending Review was presented by Tessa Jowell, then Minister of Public Health. It contained a wide ranging analysis of the state of services and made a number of recommendations and conclusions including the following (HM Treasury, 1998b):
3
The earliest years in life were the most important for child development, and very early development was much more vulnerable to adverse environmental influences than had previously been realised. Multiple disadvantage for young children was a severe and growing problem, with such disadvantage greatly enhancing the chances of social exclusion later in life. The quality of service provision for young children and their families varied enormously across localities and districts, with uncoordinated and patchy services being the norm in many areas. Services were particularly dislocated for the under fours-- an age group that tended to get missed by other Government programmes. The provision of a comprehensive community based programme of early intervention and family support which built on existing services could have positive and persistent effects, not only on child and family development but also in helping break the intergenerational cycle of social exclusion, which could lead to significant long term gain to the Exchequer. With regard to the nature of the programme, the review argued that while there was no single blueprint for the ideal set of effective early interventions, they should be: (1) two-generational, involving parents as well as children; (2) non-stigmatising, avoiding labeling "problem families"; (3) multifaceted, targeting a number of factors, not just, for example, education or health or parenting; (4) persistent, lasting long enough to make a real difference; (5) locally driven, based on consultation with and involvement of parents and local communities; and (6) culturally appropriate and sensitive to the needs of children and parents. It was argued also that a range of services should ideally be integrated to support the complex and varied physical, developmental and emotional needs of young children and families. Such services should be easily accessible--within "pram pushing distance"--and backed up by outreach to offer support in the home. The programme was to be area-based, with all children under four and their families living in a prescribed area being clients of the local programme, with the right to a say in the services provided. This area-based characteristic was congruent with other area-based initiatives that were a feature of much government policy. Background to early intervention work Considerations relevant to government policy for early interventions such as Sure Start Local Programmes (SSLPs) derive from research evidence and also from concerns that had arisen over a number of years in child health. These considerations are considered in turn. The use of research evidence on early child development In an innovation for the development of policy, empirical findings of research studies was taken into account in the review, as described by Norman Glass in oral evidence to House of Commons Science and Technology Committee (2006): We were influenced very heavily by a series of experimental studies in the United States, many of them different but relating to early years programmes, which appeared to show significant improvements on a number of measures.……… We were influenced by issues of evidence from our own birth cohort studies which showed that many of the influences in people's later lives were present in the first seven years of their lives and that those were the most significant influences affecting people's lives, in so far as you could see what affected people's lives. There was a lot of evidence on the importance of things like parental attachment and so on. There was a lot of stuff around of that kind which did not point to particular programmes but nevertheless pointed in the direction of saying that early years mattered and probably mattered more than interventions you could make later on in people's lives and that there were things that appeared to be effective which were being carried out elsewhere. The evidence from the USA that particularly influenced the review included randomised control trials of early years’ interventions, demonstrating clear benefits for disadvantaged children of high quality childcare provision, whether started in infancy (Abecedarian Project, Ramey & Campbell, 1991) or at three years of age (Perry Pre-school Project, Schweinhart et al., 1993).
4
Where quasi-experimental studies had rigorous methodology, they produced similar results. Small-scale tightly controlled interventions had produced larger effects than the more extensive large-scale interventions, such as the Chicago Child-Parent Centers (Reynolds, Temple, Robertson & Mann, 2001) and Head Start (Karoly et al., 1998). Nevertheless, the impact of large-scale interventions was still substantial, producing worthwhile benefits for children, families and communities. Such early childhood interventions often used home visiting and parental support as a supplement to childcare, which was said to have additional benefits. There was other evidence for the benefits of home visiting provided by nurse-qualified staff, rather than by paraprofessionals, particularly if a highly structured approach was implemented (Olds et al., 1997; 1999). Research on early years interventions was presented to the Cross-Departmental Review (e.g. Oliver, Smith & Barker, 1998; Pugh, 1998), since synthesised by Melhuish (2004), with conclusions similar to the earlier government review. The mounting evidence of the importance of the early years and the potential of early intervention (e.g., Shonkoff & Phillips, 2000) encouraged the development of several intervention projects in the USA, Australia, Canada as well as SSLPs in the UK, all setting out to address factors such as parent attitudes and mental health, childrearing, and high-quality early stimulation and education programmes. However, early results from these intervention efforts highlighted the need for caution as only sustained high-quality interventions proved to be effective (Melhuish, 2004; Olds, 2002). Community Development In addition to the lessons from research on early intervention and child health, Norman Glass came to believe in the importance of community involvement and thus advocated a community development approach to early-years intervention. This was somewhat surprising given the nature of the evidence that was used to justify Sure Start spending, which had little, if anything, to say about community development. The emphasis placed on community development in the case of SSLPs would be an interesting tale, only part of which is known. A Treasury minister, Alastair Darling, asked Norman Glass “How can you assure me that this programme will not lead in 10 years' time to a lot of boarded-up, fly-blown family centres such as I have seen in my own constituency and elsewhere?” to which Glass (2005) responded as follows: This programme would be "owned" by local parents, local communities and those who worked in the programme. Because those who benefited would be able to shape it to do what they wanted, rather than it being done to, or for, them, it would not be seen as just another initiative by Whitehall to do something about the feckless proles. Glass (2005) subsequently recalled: What I learned from visits to successful early years programmes and local communities was that it was necessary, in the case of early years at any rate, to involve local people fully in the development and management of the programme if it was to take root and not simply be seen as another quick fix by middle-class social engineers. The community development approach of Sure Start was also consistent with broader principles central to New Labour’s interest in modernizing government, including the view that public services should be user not provider driven, evidence based, joined up and innovative. The birth of Sure Start The findings of the Cross-Departmental review were incorporated into the 1998 Comprehensive Spending Review that delineated future government expenditure. Announcing its details to the House of Commons on 14th July 1998, the Chancellor of the Exchequer introduced the plan for what would be know as Sure Start aiming to bring together quality services for children under four and their parents — nursery, childcare and playgroup provision, and pre-natal and other health services. One new feature involved extending to parents the offer of counseling and help to prepare their children for learning and for school (HM Treasury, 1998a). The review’s final report noted that disadvantage among young children was increasing, that this could result in difficulties in later life and that the earlier intervention was undertaken, the
5
more likely poor outcomes were to be prevented. Further, the report noted, current services were uncoordinated and patchy, that young children often missed out on services which concentrated on older children, and that the quality of services varied. Nevertheless, there existed good practice that could inform the enhancement of programmes for young children. It was recommended, therefore, that there should be a change of approach to the design and delivery of services. They should be jointly planned by all relevant bodies, both within the local authority and outside it (HM Treasury, 1998b). A total of £542 million became available to be spent over three years, with £452 million designated for England. England, Wales, Scotland and Northern Ireland would each develop their own Sure Start plans. In England it was decided that there would be 250 programmes up and running by 2001-02, supporting about 187,000 children, or 18% of all poor children under four. On average, a local programme was to include 800 under fours. The intention was to provide each programme with a ring-fenced budget--roughly equivalent to £1,250 per annum per child—at the peak of funding. Programmes were to be funded for 10 years to ensure money was not diverted to other services and to signal a long-term commitment. Each SSLP was to run for at least seven to 10 years with Government funding peaking at year three of operation and declining from year six through to zero at year 10. It was implicit that some funding would be picked up by local authorities and that some funding would no longer be needed because of the successful “reshaping” of mainstreaming services to more appropriately meet the needs of local families. This commitment and investment utterly transformed early years services in the UK, while still representing a relatively small contribution from the perspective of Treasury--just 0.05% of public expenditure. The programmes were to be targeted on the 20% most deprived areas. There are no published figures on how many poor children live in such areas. However using statistics from the Indices of Multiple Deprivation (Office of the Deputy Prime Minister, 2004) we have calculated that for children in families with an income 60% or less than the national median (official poverty line), 51% of all such poor children live in the 20% most deprived areas and 65% live within the 30% most deprived areas. Joined-up government and Sure Start The Sure Start Unit (SSU) responsible for administering the new initiative was crossdepartmental, run by a ministerial steering group representing the Departments of Education and Employment; Health; Social Security; Environment, Transport and the Regions; Culture, Media and Sport; Trade and Industry; and the Home Office; Lord Chancellor's Department and HM Treasury. The group was chaired by the Minister for Public Health (first Tessa Jowell, later Yvette Cooper), but the then Secretary of State for Education and Employment, David Blunkett, would speak for Sure Start in cabinet. In order to underscore the cross-cutting nature of the initiative, there was an interesting constitutional innovation: In the House of Commons questions about the programme would be tabled during Education question time and answers would be provided by the Minister of Public Health. This made clear that the principal departments involved were Health and Education. Such inter-departmental cooperation could only have been achieved with Treasury’s influence. The embryonic SSU advertised for a Head and Naomi Eisenstadt’s appointment was announced in December 1998 (see Foreword). Work began on making the Review’s ideas a reality. Eisenstadt had a background in the voluntary sector and community development as well as early years services, and hence the aforementioned emphasis on local community autonomy was congruent with her previous experience. Also around this time government announcements became more specific about the contents of SSLPs: home visits for all families with newborn children to inform about available services and support; health, education and childcare services; toy libraries; toddler groups and family support. SSLPs were meant to bring "joined-up" services of health, childcare and play, early education and parental support to families with a child under four years of age. SSLPs were to be a
6
completely new way of working for central and local governments. It was to be the glue that would bind together a range of local services for families. It was to be based on the best evidence and on experience of what works to give children and families the very best chance to thrive. Programmes were directed to provide outreach for difficult-to-reach families and could add extra services to suit local needs, such as debt counselling, employment and benefits’ advice. Community control was to be exercised through local partnerships. Initially service providers in a deprived area were invited to submit a bid for Sure Start funding. The invitation indicated that a partnership of local stakeholders had to be constituted and that this partnership needed to draw up a plan for a Sure Start programme, nominating a lead agency. These partnerships were to be at the heart of the initiative and bring together everyone concerned with children in the local community, including health, social services, education, the private sector, the voluntary sector and parents. Thus, partnerships were to provide local community influence for the design of each SSLP and, as a consequence, even though a set of core services were required, no specification was provided of how they would be delivered, only what they should aim to achieve. Funding was to flow from central government—the SSU—directly to programmes. Programmes could act largely independently of local government, although local government departments of education, social services and the like would typically be part of the partnership. This bypassing of local government was welcomed by a number of government ministers who viewed local government as largely failing to deal with the needs of disadvantaged communities; this belief bolstered the emphasis on community control. Central government guidance for SSLPs The Sure Start Unit (1998) prepared guidance for local programmes, drawing upon and elaborating the Cross-Departmental Review. This stipulated the key principles of the programme. Emergent SSLPs were told that services must co-ordinate, streamline and add value to existing services in the SSLP area, including signposting to existing services; involve parents; avoid stigma; ensure lasting support by linking effectively with services for older children; be culturally appropriate and sensitive to particular needs; be designed to achieve specific objectives relating to Sure Start’s overall objectives; and promote accessibility for all local families, later changed to “promote the participation of all local families in the design and working of the programme.” The first guidance also outlined the core services that all SSLPs were expected to provide: (1) outreach and home visiting; (2) support for families and parents; (3) support for good quality play, learning and childcare experiences for children; (4) primary and community health care and advice about child health and development and family health; and (5) support for people with special needs, including help getting access to specialised services. Getting Sure Start started The speed with which funding was made available for SSLPs was to some extent overwhelming, resulting in a somewhat slow start to the establishment and development to operational status. Only 6% of the millions of pounds allocated to the Sure Start scheme in 1999 was spent in that year. Of the 60 local area groups invited to form a partnership and submit programmes in January 1999, only 15 were granted full approval and allocated funds in that year. Another 44 were not approved until June, 2000, after they had refined their programme plans. The 60th was delayed until the second round of programmes was announced. Despite this slower-than-expected start, and without any information pertaining to the success of the initiative, the Treasury, in its 2000 Spending Review, expanded Sure Start-doubling the planned number of programmes from 250 by 2002 to over 500 by 2004, thereby more than doubling expenditure to almost £500 million by 2003-04. The expanded Sure Start initiative was to reach one third of poor children under four years of age. In return for this investment, each SSLP was expected to deliver quantified improvements (i.e., specified targets) in local children's social and emotional development, health and ability to learn, as well as strengthening families and communities.
7
This rapid and largely unexpected rush to expand the number of SSLPs so soon after the initial setting up of the initiative was not universally welcomed, and had implications for the design of the evaluation. Some people advised that it was too early to double the number of SSLPs, but the advice was not taken For example, Norman Glass (2006) argued against it: I certainly believe and I argued strongly at the time that there was sufficient evidence to have a small Sure Start programme. I think we started off with a programme of 200 which is not that small but was still small. My view--and I argued it at the time when I was in the Civil Service and I have argued it subsequently-- was that we should have learned much more about the experience from those 200 before we rolled it out on any scale. I do feel that we have rolled it out on the basis of inadequate evidence about how best it should be done as much as whether it has an effect, and being clear about the kinds of impact we wanted this programme to have. Thus it was that SSLPs became a cornerstone of the UK Government’s campaign to reduce child poverty and social exclusion. SSLPs were to serve all children under four and their families in a prescribed area. This area-based strategy allowed the relatively efficient delivery of services to those living in deprived areas without stigmatising those receiving services: Disadvantaged areas were targeted, but within the area the service was universal. The autonomy of SSLPs As a consequence of the local autonomy central to the original conceptualisation of community control of SSLPs, and as already noted, they did not have a prescribed “protocol” of services to promote adherence to a prescribed model even though they had a set of core services to deliver. Thus, each SSLP had freedom to improve and create services in the manner they saw fit to do so, with general goals and some specified targets (e.g., reduce number of low birthweight babies, improve language development of young children), but without specification of exactly how services were to be delivered. This contrasted markedly with interventions with clear models of provision and demonstrable effectiveness that formed the basis of the research evidence justifying the creation of Sure Start (e.g. Abecedarian project, Ramey & Campbell, 1991; Perry Pre-school project, Schweinhart et al., 1993; Incredible Years, Webster-Stratton, 1993). It appeared that while the research evidence was critical to winning the argument for increased Early Years’ expenditure, it was largely overlooked in the detailed planning for and actual operation of programmes, despite entreaties to local programmes that their services be “evidence based”. Indeed, the SSU published a guidance handbook offering a menu of ‘evidence-based’ interventions from which to choose, but there is little evidence that it was much used. Evaluation The Treasury’s involvement with Sure Start was central to the creation of the programme and the changes to it that have occurred. One of the conditions insisted upon by Treasury was that there be a rigorous evaluation of the initiative. Following competitive tender, the National Evaluation of Sure Start (NESS) was commissioned in early 2001 to undertake a multifaceted evaluation of SSLPs, addressing (1) the nature of the communities in which SSLPs were situated, (2) the ways in which SSLPs were implemented, (3) the impact of SSLPs on children, families and communities and (4) the cost-effectiveness of SSLPs. In addition, NESS was charged with (5) providing technical support to local programmes so that each could undertake its own local evaluation to inform the further development of the services being offered. The great diversity amongst SSLPs posed a particular set of challenges for the national evaluation in that there were not several hundred programmes delivering one well-defined intervention, but several hundred unique and multifaceted interventions operating in different places. In order to undertake the evaluation, NESS used a variety of strategies to study the first 260 SSLPs that were rolled out. These included the gathering of administrative data already available on the small geographic areas that defined SSLP communities (e.g. census data, police records, work and pension records); developing geographical information systems that allowed the collating of information in non-standard geographic units (SSLP areas); conducting
8
surveys of SSLPs dealing with many aspects of SSLPs; carrying out face-to-face and telephone interviews with programme managers, programme employees and parents about the operation of their local programme; and conducting a large-scale survey of child and family functioning in thousands of households in SSLP areas, and in SSLP-to-be areas. In the following chapters, a summary of insights garnered across the period 2001 through 2006 from these diverse datacollection strategies is presented. Subsequent Policy Developments The policy-making process did not stand still while the evaluation work presented in this volume was being carried out. The recognition of the need for even greater government coordination of children’s services led, in 2003, to the creation of a Minister for Children, Young People and Families within the Department for Education and Skills. The first Minister was Margaret Hodge. As NESS progressed its evidence was monitored within DfES and has influenced guidance given to SSLPs. When evidence of impact upon children and families became available, discussed in Chapters eight and nine, this contributed to a fundamental change in the structure of SSLPs. The NESS findings indicated that SSLPs were not having the impact that had been hoped for. Also, evidence from another ongoing research project, the Effective Provision of Preschool Education (EPPE) (see Sylva, Melhuish, Sammons, Siraj-Blatchford & Taggart, 2004), indicated that a particular type of early years provision, integrated Children’s Centres, was particularly beneficial to children’s development. Margaret Hodge, as Minister for Children Young People and Families, was responsible for Sure Start and she decided that this combination of evidence from these two research projects justified changing SSLPs into Children’s Centres. This was announced in 2005 alongside a move to transfer the new Sure Start Children’s Centres into Local Authority control. This transfer of control from central to local government was politically inspired to ensure that Sure Start Children’s Centres became embedded within the welfare state by government statute and would thus be difficult to eradicate by any future government, but the transfer of control to local authorities has proved unpopular with many Sure Start advocates. Also concern about child protection that had been mounting in the 1990’s reached a crisis with the horrific case of Victoria Climbié, who, despite being supposedly monitored by several agencies (health, social services, police etc.), was tortured and murdered in 2000. This triggered a major governmental review (Laming, 2003), which emphasised the importance of high-quality work and research by all relevant professionals, so interagency collaboration and training was once again stressed (Shardlow, Davis, Johnson, Long, Murphy, & Race, 2004). This resulted in a series of government reports, including “Every Child Matters” (HM Government, 2003, 2004a, b) which set out plans to reform and improve children’s services. These plans were incorporated into the HM Government Children Act (2004c), which set out a new framework for children’s services, ensuring accountability and partnership at local level. The Act placed a duty on health agencies to co-operate with local authorities to improve the well-being of children, and a duty on all agencies to promote the welfare of children. It also introduced measures to support the sharing of information about children between different agencies and professionals. The term “safeguarding” replaced “child protection” to emphasise the responsibility of all professionals working with children to consider how their best interests, health and development could be promoted. Within a similar time-frame, various reports on children’s health care criticized the long-term neglect of children’s services and led, in 2001, to a National Service Framework (NSF). In 2003 responsibility for policy on social services for children and young people transferred from the Department of Health to DfES and came under the new Minister for Children, Young People and Families. The finalised NSF (DoH & DfES, 2004) thus became the joint responsibility of both departments. It was probably the most comprehensive exposition of child health policy anywhere, reflecting a very broad view of what is meant by health. It endorsed previous policy developments in the fields of early detection, child mental health and child protection, reinforcing guidance on inter-disciplinary collaboration. The concepts underpinning Sure Start were strongly supported for future policy.
9
These changes meant that from April, 2006, local authorities became the accountable body for the whole Children’s Centre programme in their areas, and health agencies were legally obliged to cooperate in the provision of services within Children’s Centres. The spend on Children's Centres and the associated programmes was £1.3 billion in 2005-06. For 2006-07 £1.7 billion was provided to local authorities for Children's Centres. For 2007-08, £1.8 billion was set aside. This represented almost four times the amount spent on equivalent services in 2001-02. Sure Start thus became a significant part of the Welfare State. As Prime Minister Blair (2006) recently stated: Sure Start is one of the government's greatest achievements. It is a programme that gives antenatal advice, and early-years help for children who need it. It is a vital source of learning to parents who often find work on the back of it; and a community facility that becomes a focal point for local health, childcare and educational networks. It has become a new frontier of a changing welfare state. References Barnett, W. S. (1995). Long term effects of early childhood programs on cognitive and school outcomes. The Future of Children: Long Term Outcomes of Early Childhood Programs, 5, 94 -114. Bishop, D. V. M. (2000). How does the brain learn language? Insights from the study of children with and without language impairment. Developmental Medicine & Child Neurology, 42, 133–142 Blair, T. (1998). Foreword. In H.M. Treasury, Modern public services for Britain: Investing in reform. Comprehensive Spending Review: New Public Spending Plans 1999-2002. London: HM Treasury. Accessed at http://www.archive.official-documents.co.uk/document/cm40/4011/foreword.htm January 15, 2007. Blair, T. (2006). A failed test of leadership. The Guardian, October 5th, 2006. Farrington, D. P. (2003). Developmental and life-course criminology: Key theoretical and empirical issues. Criminology, 41, 201-235 Glass, N. (2005). Surely some mistake? The Guardian, January 5th, 2005. Glass, N. (2006). Oral evidence to House of Commons Science and Technology Committee, on Scientific Advice, Risk and Evidence: How the government handles them. 24th May, 2006. http://www.publications.parliament.uk/pa/cm200506/cmselect/cmsctech/c900-ix/c90002.htm HM Government. (2003). Every Child Matters. Green Paper. Nottingham, Department for Education and Skills. HM Government. (2004a). Every Child Matters: Change for Children Programme. Nottingham, Department for Education and Skills. HM Government. (2004b). Every Child Matters: Next Steps. Nottingham, Department for Education and Skills. HM Government. (2004c). Children Act 2004. London, HMSO. HM Treasury. (1998a). Modern Public Services for Britain: Investing in Reform. Comprehensive Spending Review: New Public Spending Plans 1999-2002. London: HM Stationery Office. http://www.archive.official-documents.co.uk/document/cm40/4011/4011.htm Accessed on January 15, 2007. HM Treasury. (1998b). Comprehensive Spending Review: Cross Departmental Review of Provision for Young Children. London: HM Stationery Office. (http://www.archive.officialdocuments.co.uk/document/cm40/4011/401122.htm) Accessed on January 15, 2007. Karoly, L.A., Greenwood, P.W., Everingham, S.S., Hoube, J., Kilburn, M.R., Rydell, C.P., Sanders, M., & Chiesa, J. (1998). Investing in our Children: What we know and don’t know about the costs and benefits of early childhood interventions. Santa Monica, CA: RAND. Lord Laming (2003). Inquiry into the death of Victoria Climbié. London: HMSO. Melhuish, E. C. (2004). A literature review of the impact of early years provision upon young children, with emphasis given to children from disadvantaged backgrounds: Report to the Comptroller and Auditor General. London: National Audit Office. Available at http://www.nao.org.uk/publications/nao_reports/0304/268_literaturereview.pdf Office of the Deputy Prime Minister, 2004. The English Indices of Deprivation, 2004: Summary (Revised), London, ODPM. Olds, D. L., Kitzman, H., Henderson, C. R., Eckenrode, J., & Cole, R. (1997.) It worked in Elmira, but will it work in Memphis? The long-term effects of nurse home visiting on mothers' lives and children's wellbeing. Focus 19(1). Olds, D. L., Henderson, C. R., Kitzman, H., Eckenrode, J. J., Cole, R. E., & Tatelbaum, R. C. (1999). Prenatal and Infancy Home visitation by Nurses: recent Findings. Future of Children, 9, 44-66.
10
Olds, D.L. (2002). Prenatal and Infancy Home Visiting by Nurses: From Randomized Trials to Community Replication. Behavioural Science, 3, 153-172. Oliver, C., Smith, M., & Barker, S. (1998). Effectiveness of Early Interventions. In HM Treasury, Comprehensive Spending Review, 1998. Supporting Papers Vol. 1. (Copies available from the Public Enquiry Unit, HM Treasury, Parliament Street, London SWlP 3AG) Pugh, G. (1998). Children at risk of becoming socially excluded: An introduction to the problem. In HM Treasury, Comprehensive Spending Review, 1998. Supporting Papers Vol. 1. (Copies available from the Public Enquiry Unit, HM Treasury, Parliament Street, London SWlP 3AG) Ramey, C. T., & Campbell, F. A. (1991). Poverty, Early Childhood Education, and Academic Competence: The Abecedarian Experiment. In A.C. Huston (Ed.), Children in Poverty: Child Development and Public Policy. Cambridge, MA: Cambridge University Press, 190-221. Reynolds, A. J., Temple, J. A., Robertson, D. L., & Mann, E. A. (2001). “Long-Term Effects of an Early Childhood Intervention on Educational Achievement and Juvenile Arrest: A 15-Year Follow-Up of LowIncome Children in Public Schools”. Journal of American Medical Association 285, 2339-2346. Rutter, M., Tizard, J., Yule, W., Graham, P., & Whitmore, K. (1976). Research report: Isle of Wight Studies, 1964-1974. Psychological Medicine, 62, 313-32. Rutter, M. and the English and Romanian Adoptees (ERA) study team (1998). Developmental Catch-up, and Deficit, Following Adoption after Severe Global Early Privation. Journal of Child Psychology and Psychiatry and Allied Disciplines, 39, 465-476. Schweinhart, L. J., Barnes, H., & Weikhart, D. (Eds). (1993). Significant benefits: the High/Scope Perry Pre-school Study through age 27. Ypsilanti, Michigan: High/Scope Press. Shardlow, S., Davis, C., Johnson, M., Long, A., Murphy, M., Race, D. (2004). Education and training for inter-agency working: New standards. Salford Centre for Social Work Research. http://www.chssc.salford.ac.uk/scswr/ Shonkoff, J. P., & Phillips, D.A. (2000) From neurons to neighbourhoods: The science of early child development. Washington DC: National Academy Press. Sure Start Unit (1998). Sure Start: Guide for trailblazer programmes. Department for Education and Employment, London. Sylva, K., Melhuish, E., Sammons, P., Siraj-Blatchford, I., & Taggart, B. (2004). Effective Pre-school Provision. London: Institute of Education. Webster-Stratton, C. (1993). Strategies for helping families with young oppositional defiant or conductdisordered children: The importance of home and school collaboration. School Psychology Review, 22, 437-457.
11
Influencing policy through research on Early Childhood Education Paper 2 Abstract The last 20 years has seen a profound transformation of services available for England’s youngest children (aged 0-5). Accompanying this transformation has been a plethora of policy initiatives and legislation that have sought to move the ‘early years’ sector from a ‘patchwork quilt’ to a ‘seamless cover’ of joined-up services. The Labour Party swept to power in 1997 on a manifesto commitment to ‘concentrate resources on the essential tasks of combating unemployment and poverty’ (Manifesto 1997). The key to delivering this promise was to lower the number of workless households. This was to be brought about by a raft of social policies including the introduction of Working Families Tax Credits and expanding child care in order to encourage parents’ return to the work force. The Labour Party also made clear that its social reforms would be informed by research evidence. This paper will explore the implementation of early years’ educational policy in England since the early 1990s. It will focus on the evidence concerning the expansion of services and the benefits of early years’ education to children’s development. Having given the background to reform it will examine the Effective Pre-School and Primary Education 3-11 (EPPE 3-11 Project, 1997 - 2008) as an example of a study that has provided research evidence to inform policy development. The EPPE project (1997-2008) is a longitudinal study of the effects of pre- and primary school experiences on 2,800 children who were initially recruited from 141 pre-schools (plus 300 ‘home’ children who had no pre-school experience) who then went on to attend over 800 primary schools throughout England. The study has established the positive effects of early childhood education on children’s development after taking account of child (gender etc.) and family (parental qualifications etc.) background factors. It has also highlighted the importance of the quality of the home learning environment for ensuring children’s long term success. These findings have been used as the ‘evidence base’ for UK policy on universal pre-school provision as well as targeted services in disadvantaged communities. The paper will identify key policy questions that have been successfully answered by the EPPE research and show how this has helped ‘shape’ developing policies and legislation. It could be argued that the EPPE research was ‘pushing on an open door’ in policy development in that it’s finding happen to have co-incidentally coincided with the need to develop early years education and care. Whilst the time was ripe for reform to this non-statutory sector the counter arguement is that without this research evidence base the extent and speed of the reform (made possible by the Treasury) would have been more precarious. EPPE has been successful in influencing policy because its findings are large scale and broadly representative, longitudinal and based on ‘value added’ analyses that established the measurable contribution of a range of influences on children’s development.
12
Influencing policy through research on Early Childhood Education Paper 2 Authors: Brenda Taggart, Institute of Education, University of London Kathy Sylva, University of Oxford Edward Melhuish, Birkbeck, University of London Iram Siraj-Blatchford, Institute of Education, University of London Pam Sammons, University of Nottingham This paper presents the evidence from one study that has informed Early Years policy in the U.K. It begins with the policy context in the early 1990s and shows how research has supported policy innovation. There have been many studies of ways that research has informed practice but fewer on instances of research shaping policy. This paper is one step in that direction, as advocated by Whitty (2007) and Huw et., al. (2001). Families’ need for childcare: The Socio-economic Agenda Prior to the Labour Party’s 1997 electoral victory, policy in early years had a discernable split between ‘care’ and ‘education’ and was far from universally uniform embracing both a growing voluntary and private sector alongside the maintained (public) sector. There were considerable geographical and socio-economic differences in pre-school availability. The new Government was committed to ‘concentrate resources on the essential tasks of combating unemployment and poverty’ (Labour Party Manifesto 1997). The key to delivering this was to lower the number of workless households by encouraging the female workforce, which required reliable provision for young children. This bold commitment was summed up as follows: ‘While the nineteenth century was distinguished by the introduction of primary education for all and the twentieth century by the introduction of secondary education for all, so the early part of the twenty first century should be marked by the introduction of pre-school provision for the under fives and childcare available to all’ Rt Hon Gordon Brown, MP, Chancellor of the Exchequer, (Her Majesty’s Treasury, 2004a). Raising educational achievement: The Standards Agenda The end of the twentieth century saw an increased focus on educational ‘standards’ in the UK. International comparisons of standards in literacy, mathematics and science 1 was a wake-up call to those concerned with a ‘standards’ agenda. The relative position of different countries on international ‘league tables’ added to the debate about effective teaching, particularly in reading and mathematics, through-out each age phase. This increased interest on standards during statutory schooling led to a focus on children’s skills, attitudes and dispositions to learn at school entry. This turned the policy searchlight on early years education as one means of raising standards in primary schools. The effects of early education: The Research Agenda Internationally a number of key early years research studies had demonstrated the positive benefits of pre-schooling. The Perry Pre-school study (Schweinhart et., al. 1993) and the Abercedarian study (Ramey & Ramey, 1998) both demonstrated positive effects of early education/care, especially for children from disadvantaged backgrounds. The Perry study provided evidence about the long term benefits that pre-school can bestow and showed that it improved high school grades, decreased delinquency and improved employment status and earnings. There were also savings in social, health and justice systems costs later on from
1 PISA (Programme for International Student Assessment), TIMSS (Trends in International Mathematics and Science Study and PIRLS (Progress in International Reading Literacy Study) see Mullis et. al., 2003
13
early economic investments. In the UK there was little research evidence to suggest that our pre-schooling could replicate these outcomes. The early 1990 saw change in research interest in the early years. The Rumbold Report (DES, 1990) highlighted the potential for pre-school education to give children a better start at school. The Start Right Report (Ball, 1994) called for investment in universal pre-school education based on Sylva’s (Sylva 1994) review of the research evidence that made a convincing case for the positive effects of early education on motivational and academic outcomes. Evidence based policy making The Labour Party, on election, made clear that its social reforms would be informed by research evidence (Humes & Bryce, 2001). The policy challenge was to transform services available for England’s youngest children through policy initiatives and legislation that sought to move the ‘early years’ sector from a ‘patchwork quilt’ to a ‘seamless cover’ of joined-up services combining education and care. This paper illustrates how research has helped to inform early years policy by focusing on the Effective Pre-School and Primary Education 3-11 (EPPE 3-11), an example of a study that provided evidence to inform policy development. The EPPE 3-11 Project (1996-2008) The EPPE study was funded by the UK’s Department for Education and Skills (DfES) to provide answers to policy questions about early childhood education and care. The project was the first major European longitudinal study of a national sample of young children’s development between the ages of 3 and 7 years. Research aims EPPE investigated what kinds of pre-schooling were most effective for young children. It researched three issues with important implications for national policy and educational practices: • the effects on children’s cognitive and social/behavioural development of different types of pre-schooling, • the structural and process characteristics of more effective pre-school centres, and • the interaction between pre-school and child/family characteristics. Research design EPPE collected extensive data (Sammons et al., 2002a & 2003) on 3,000 children’s cognitive and social/behavioural outcomes and family background characteristics (this included a sample of ‘home’ children with no/minimal pre-school experience) as well as extensive information on the pre-school setting (141) they attended (local authority day nursery, integrated centres, playgroups, private day nurseries, nursery schools and classes). The study used observational rating scales 2 and interviews with the Centre Managers to produce a profile for each centre, which included information on staffing profiles and organisational practices (see Sylva et. al., 1999, Taggart et. al., 2000). In addition to investigating the effects of pre-school, EPPE explored the characteristics of effective practice (see Siraj-Blatchford et. al., 2003) through a mixed methods design (Sammons et. al., 2005, Siraj-Blatchford et. al., 2006). The growing field of school effectiveness research has developed an appropriate methodology for the separation of intake and school influences on children's progress using 'value added' multilevel models (Goldstein, 1987, 1995). EPPE was the first major early years study to adopt an educational effectiveness design applied to pre-schooling. This methodology enabled the study of the progress of individual children as well as the effects of individual pre-school centres on children's outcomes (at aged 5, 7 and 11 yrs of age) after controlling for background factors.
2
The early Childhood Environmental Rating scales: Revised (Harms et al., 1988) and Extension Sylva et al., 2003), Caregiver Interaction Scale (Arnett, 1989).
14
Findings EPPE identified significant positive effects of early childhood education on children’s development. It showed that better quality, longer duration and the effectiveness of the preschool setting attended made an important contribution to children’s progress and attainment which could have lasting effects (Sammons et. al., 2004) It also highlighted the relative strengths (expressed in Effect Sizes) of background characteristics on specific cognitive and social/behavioural outcomes. A key finding was the importance of the quality of the early home learning environment (HLE) for ensuring children’s long term success (Melhuish et.al., 2007, Sylva et. al., 2004). The impact of EPPE on policy These findings contributed to the ‘evidence base’ for UK policy on universal pre-school provision and targeted services in disadvantaged communities. The paper will identify important policy questions that have been addressed by the EPPE research and show how this helped ‘shape’ developing policies and legislation through references to key documents. In 2006 the DfES’s Analytical Strategy (DfES, 2006a) stated: “Effective Provision of Pre-school Education (EPPE) at Key Stage 2 This project is helping to unpack the impact that pre-school education has on children’s learning when they reach Key Stage 2. The results of the EPPE study have already had a significant impact on policy development in early years, especially through clearly demonstrating the vital influence of the quality of provision on successful outcomes. Continuation of the cohort will enable us to understand the lasting impact of early years experience and the factors which either enhance or negate this” (DfES, 2006a, p20). EPPE’s ‘significant impact on policy development’ is best considered through the published documents that refer to its influence and the importance of the research findings. In ‘Meeting the Childcare Challenge, The National Childcare Strategy’ (DfEE, 1998) the Prime Minster stated that: “good quality childcare ….. is vital to them (children) growing up happy and secure in themselves, socially confident and able to benefit from education” adding that “childcare has been neglected for too long” (DFEE, 1998, p3). This Green Paper paved the way for a major policy initiative outlined in ‘Choice for parents, the best start for children: a ten year strategy for childcare’ (HMT, 2004). This document sets out a long term vision for services which would change the experiences of young children. The Ten Year Strategy is clear that ‘the main source of analyses of the impact of pre-school provision on child development in the UK is the Effective Provision of Pre-School Education Project (HMT, 2004, p65). This strategy provides a rationale for the benefits of early education and repeatedly cites EPPE as providing wide ranging evidence on policy issues. The paper quotes the research evidence cited in the Ten Year Strategy illustrative of policy questions EPPE responded to. It covers only a limited number of the policy implications of EPPE, namely those concerning centre-based provision. EPPE also reported on the importance of the contribution of the family to children’s development through the quality of the HLE. This has influenced the development of policies to support families and communities (Sure Start Local Programmes see Melhuish & Hall, 2007). Policy Question 1 - Who benefits from pre-school education? One of the key policy questions facing the Government concerned the benefits of universal provision versus the strategic use of pre-school for disadvantaged children to ‘catch-up’. EPPE showed that ‘any pre-school experience can have clear positive effects on children’s social, emotional and cognitive development’ (Her Majesty’s Treasury, 2004b, p8). The finding that pre-school has benefits for ALL children led directly to policy development. This is not to say that the benefits for disadvantaged children are not of policy interest (see below), but it did provide the research evidence that suggests universal provision would benefit society as a whole. The big policy shift announced in the Strategy was twofold:
15
1) “legislation for a new duty on local authorities in place by 2008 so that over time they will secure sufficient supply to meet the needs of families” and 2) “a goal of 20 hours a week of free high quality care for 38 weeks for all 3 and 4 year olds with this Pre-Budget Report announcing a first step of 15 hours a week for 38 weeks a year reaching all children by 2010” (HMT, 2004, p1). For the first time, pre-school in England was made universally available at the State’s expense. Policy Question 2 – At what age should children begin pre-school? This has long been a contentious issue. The Strategy stated: “Evidence from the Effective Provision of Pre-school education (EPPE) project shows an early start to pre-school can have significant positive effects on children’s cognitive and social development. For example, every additional month of quality pre-school from the age of two improves cognitive performance at the start of school, a gain that remains to at least age seven. Those who started in a good quality pre-school at two or younger were up to 10 months ahead of those without pre-school. The EPPE evidence also shows that an early start in pre-school improves children’s social skills at entry to school.” (HMT, 2004, p8). This notion of an ‘early start’ remains controversial as EPPE reported positive and negative findings on institutional care, as the Strategy states: “However, the studies indicate that high levels of group care of poor quality below the age of three can have a small negative effect on behaviour for some children’ (p8). This finding resonates with other research (NICHD Early Child Care Research Network, 2004) that suggest that early institutional (group) care, especially for long hours may not be in the best interest of some very young children. Policy Question 3 – For how long should children attend pre-school? On the question of duration, again the paper refers to EPPE findings: ‘Evidence from EPPE shows that the benefits are gained from regular part-time attendance through the week. Fulltime attendance gives no better gains than part-time although EPPE suggests that pre-school experiences at all levels of quality and duration have positive effects on children’s development compared with children who had no pre-school experience’ (HMT, 2004, p8). The findings have been used extensively across policy documents, most influentially in A Code of Practice on the Provision of Free Nursery Education Places for Three- and Four-Year-Olds (DfES, 2006b) and in the House of Commons’ enquiry into Early Years (House of Commons. Education SubCommittee, June 2000). Policy question 4 – Can early childcare help children ‘at risk’ of development special educational needs (SEN) and reduce social disadvantage? Early identification for SEN is crucial, especially for children from disadvantaged backgrounds. An EPPE sub-study the Early Transitions and Special Educational Needs (EYTSEN, see Sammons et. al., 2002b, 2004c Taggart et. al., 2006) explores this issue. The EPPE index of multiple disadvantage (Sammons et. al., 2002b) showed children with multiple disadvantage were more likely to be ‘at risk’ of developing SEN. The Strategy states: ‘EPPE data suggest that while one in three children were ‘at risk’ of having special educational needs at the start of pre-school, that proportion fell to one in five by the time they started primary school, suggesting that pre-school can be an effective intervention for the reduction of special needs’ needs’ (HMT, 2004, p8). Whilst not eliminating the impact of disadvantage, quality pre-school education can provide children from lower income households with a better start at school. This led to a pilot project studying the impact of free pre-schooling for under 2 year olds from disadvantaged backgrounds. Policy question 5 – What kinds of pre-school provide the best outcomes for children? The impact of different types of provision is an important question. Multilevel statistical modeling enable individual ‘centre’ effects to be studied so that ‘effective’ centres could be studied. Findings associated with quality, type and the characteristics of the work force had a considerable impact on policy. The Strategy sets out the EPPE evidence on quality:
16
“EPPE suggests that in comparison with children having no pre-school, all levels of quality and duration show a significant positive effect compared to none. The quality of preschool experience is directly related to the intellectual, social and behavioural development of children. After taking into account the impact of child, family and home environment characteristics, evidence shows significant links between higher quality and better children outcomes, with children from high quality pre-schools possessing higher reading attainment and showing fewer conduct problems. It goes on to say: “evidence suggests that the effects can be substantial, EPPE analysis indicates that the difference in child development between having pre-school and not having pre-school is 4-6 months of development. For the highest quality integrated centres, the difference can be as much as nine months. This is a substantial difference given that it occurs over just two years. The combination of high quality provision and high duration shows a particularly strong effect” (HMT, 2004, p66). EPPE’s finding on the importance of quality has been persuasive and accepted across the party political divide (Liberal Democrats, 2003). The Strategy also cites EPPE evidence on where quality can be found and factors that contribute to enhancing quality: “EPPE conclude that good quality provision can be found across all types of pre-school settings but is higher overall in integrated learning and childcare centres, nursery school and nursery classes. Better quality pre-school centres are associated with better outcomes, with key explanatory factors being: staff with higher qualifications, staff with leadership skills and longserving staff; trained teachers working alongside and supporting less qualified staff; staff with a good understanding of child development and learning and strong parental involvement” (HMT, 2004, p66). The EPPE qualitative case studies (Siraj-Blatchford et. al., 2003) gives information for practitioners on aspects of effective practice. These findings have been incorporated wholesale into Primary National Strategy (DfES, 2006c, p13). Given the highest overall quality was in integrated centres, these settings, now known as Children’s Centres have been pivotal to the new vision of early child care services. They provide childcare and support parents through job seeking, parenting and study support etc. The policy expansion in this area has been profound, “525,000 additional childcare places have been created, benefiting 1.1 million children” (HMT, 2004, p22), 1,279 Neighbourhood Nurseries have been opened (HMT, 2004, p26), and 2,599 Children’s Centres will be in place by 2008 (p1) with the promise of a “Children’s Centre in every community by 2010” (HMT, 2004, p33). This is educational expansion in unprecedented terms. Policy Question 6 – What kind of workforce? In order to ensure quality the Strategy is clear that the Government has “a commitment to radical reform of the early years childcare workforce through a new qualification and career structure” (HMT, 2004, p43) achieved through a £125 million workforce Transformation Fund that seeks out to “ensure that all full daycare settings are led by a graduate qualified early years professional…. and there will be single qualification framework and greater opportunities for existing workers to increase their skills.” (HMT, 2004, p45). Many Local Authorities have reconfigured their workforce to ensure the strategic deployment of qualified teachers across settings. This is based on the EPPE finding that qualified teachers were associated with better quality practices (Sylva et. al., 1999). In conclusion The examples above show that the EPPE project has contributed to policy development by providing sound and persuasive research evidence. It could be argued that the EPPE research was ‘pushing on an open door’ in policy development in that its finding coincided with the need to develop early years education and care. Whilst the time was ripe for reform to this nonstatutory sector the counter argument is that without this research evidence base the extent and speed of the reform (made possible by the Treasury) would have been more precarious. EPPE
17
has provided information across Government departments and has informed successive Treasury Spending Reviews. The last decade has seen an irreversible change to the three agenda’s identified at the beginning of this paper and EPPE has contributed to all three, however challenges remain: 1 ‘socio-economic agenda' The current challenge is to focus on strategies that will improve the lives of children overrepresented in disadvantaged groups. EPPE has sought to contribute to this by providing evidence to the Cabinet Office’s Equalities Review (see http://www.theequalitiesreview.org.uk/). 2 ‘standards agenda’ The development of the workforce is crucial in delivering the ‘standards agenda’ and it will be interest to see if the implementation of the Early Years Professional (CWDC, 2006)) qualification for staff will live up to its promise of improving quality. 3 ‘research agenda’ The availability of more extensive demographics datasets and more sophisticated analysis techniques enables more detailed investigations of the impact of a range of factors on a range of outcomes as well as how factors interact with one another. EPPE continues to work at the cutting edge of these developments (Melhuish et. al., 2006). The research challenge is to see how emerging results from a number of longitudinal studies (LYPSE, ALSPAC etc.) can be synthesised for meaningful patterns and trends that have new policy relevance (e.g., personalised learning, the development of ‘soft’ non-cognitive skills etc.). EPPE has influenced policy because its findings are large scale and broadly representative, longitudinal and based on ‘value added’ analyses that established the measurable contribution of a range of influences on children’s development. This has proved important to policies which identify the relative costs and benefits that might be expected to accrue from investments of public money to enhance public services. References Arnett, J. (1989), 'Caregivers in Day-Care Centres: Does training matter?' Journal of Applied Developmental Psychology, 10 541-552. Ball, C. (1994), Start Right: The Importance of Early Learning. London: RSA. Barnett, W. S. (1996), Livers in the balance: Age 27 Benefit-cost analyses of High Scope/Perry Preschool Programme, Monographs of the High Scope Educational Research Foundation, 11. Ypsilanti: High/Scope Press. Brandon, M., Howe, A., Dagley, V., Salter, C., Warren, C. and Black, J. (2006), Evaluating the Common Assessment Framework and Lead Professional Guidance and Implementation. Research Report RR740. Nottingham: DfES Publications. Children’s Workforce Development Council. (2006), Early Years Professional Prospectus. Part of the Sector Skills Council, Skills for Care and Development. Leeds. Great Britain. (1988), Education Reform Act 1988. London: HMSO. Great Britain (2006) Childcare Act 2006, Explanatory Notes. Chapter 21. London. HMSO. DES. (1988), The National Curriculum. London: HMSO. DES and Welsh Office. (1988), Task Group on Assessment and Testing. London DES. DES. (1990), Starting with Quality: report of the committee of inquiry into the quality of educational experience offered to three and four year olds (Rumbold report): HMSO DfEE. (1998), Green Paper: Meeting the Childcare Challenge. Norwich: HMSO. DfES. (2002), Delivering for Children and Families: Inter-Departmental Childcare Review. London: DfES/DWP/HM Treasury/Women and Equality Unit. HMSO. DfES. (2006a), Analytical Strategy. Nottingham: DfES Publications. http://www.dfes.gov.uk/research/data/general/AS2006.pdf (Accessed 06/02/07). DfES. (2006), A Code of Practice on the Provision of Free Nursery Education Places for Three- and FourYear-Olds. Nottingham: DfES Publications. http://www.surestart.gov.uk/_doc/P0002205.pdf (Accessed 06/02/07).
18
DfES. (2006), Primary National Strategy. Guidance on Curriculum and Standards. Improving outcomes for children in the Foundation stage in maintained schools. Process-based targets in the Foundation Stage London: DfES Publications. Effective Pre-school and Primary Education 3-11 (EPPE 3-11) Team. (2007), Promoting Equality in the Early Years: Report to The Equalities Review. London: http://www.equalitiesreview.org.uk. Goldstein, H. (1987), Multilevel Models in Educational and Social Research. London: Charles Griffin Goldstein, H. (1995), Multilevel Statistical Models. (2nd ed.). London: Arnold. Harms, T., Clifford, R. M. and Cryer, D. (1998), Early Childhood Environmental Rating Scale, Revised Edition (ECERS-R). New York: Teachers College Press. Hew Davies, T.O., Nutley, S., & Smith. P.C. (2001) What Works? Evidence Based Policy and Practice in Public Services. Eds. Bristol UK Policy Press HM Government. (2004), Every Child Matters: Next Steps. Nottingham: DfES Publications. Her Majesty's Treasury. (2004a), 2004 Spending Review: Stability, security and opportunity for all: investing for Britain's long-term future. London: HM Treasury. Her Majesty’s Treasury. (2004b), Choice for parents, the best start for children: a ten year strategy for childcare. Norwich: HMSO. http://www.hm-treasury.gov.uk/media/8F5/35/pbr04childcare_480.pdf (Accessed 06/02/07). The House of Commons Session Education and Employment Committee (Education Sub-Committee) Early Years. (2000), Minutes of Evidence. Wednesday 21st June 2000. Evidence given by Professors Sylva, Siraj-Blatchford and Sammons. Stationary Office Ltd. Humes, W. & Bryce, T. (2001). Scholarship, research and the evidential basis of policy development in education. In British Journal of Educational Studies, 49 (3). Labour Party Manifesto. (1997), http://www.labourparty.org.uk/manifestos/1997/1997-labourmanifesto.shtml. Liberal Democrats. (2003), Early Years. Consultation Paper No. 65. London: www.libdems.org.uk. Melhuish, E., Romaniuk, H., Sammons, P., Sylva, K., Siraj-Blatchford, I. and Taggart, B. (2006), Effective Pre-school and Primary Education 3-11 Project (EPPE 3-11): The Effectiveness of Primary Schools in England in Key Stage 2 for 2002, 2003 and 2004. Full Report. London: Institute of Education, University of London. http://www.ioe.ac.uk/schools/ecpe/eppe/eppe3-11/eppe3-11pubs.htm Melhuish, E.C., Sylva, K., Sammons, P., Siraj-Blatchford, I., Taggart, B., & Phan, M. (2007) Effects of the Home Learning Environment and pre-school center experience upon literacy and numeracy development in early primary school. Journal of Social Studies Issues (forthcoming). Melhuish, E. and Hall, D. (2007), 'The policy background to Sure Start'. In J. Belsky, J. Barnes and E. Melhuish (eds), Evaluating Sure Start: Does area-based early intervention work? Bristol: Policy Press. Mullis, I. V. S., Martin, M. O., Gonzalez, E. J. and Kennedy, A. M. (2003), PIRLS 2001 International Report: IEA’s Study of Reading Literacy Achievement in Primary Schools. Chestnut Hill, MA: Boston College. NICHD Early Child Care Research Network. (2004), 'Type of child care and children's development at 54 months'. Early Childhood Research Quarterly, 19 (2), 203-230. QCA/DfES/SureStart. (2003), Foundation Stage Profile. Sudbury: QCA Publications. Ramey, C. T. and Ramey, S. L. (1998), 'Early intervention and early experience'. American Psychologist, 53 (2), 109-120. Sammons, P., Sylva, K., Melhuish, E. C., Siraj-Blatchford, I., Taggart, B. and Elliot, K. (2002a), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 8a - Measuring the Impact of Pre-School on Children's Cognitive Progress over the Pre-School Period. London: DfES / Institute of Education, University of London. Sammons, P., Smees, R., Taggart, B., Sylva, K., Melhuish, E. C., Siraj-Blatchford, I. and Elliot, K. (2002b), The Early Years Transition and Special Educational Needs (EYTSEN) Project: Technical Paper 1 - Special Needs Across the Pre-School Period. London: DfES / Institute of Education, University of London. Sammons, P., Sylva, K., Melhuish, E. C., Siraj-Blatchford, I., Taggart, B. and Elliot, K. (2003), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 8b - Measuring the Impact of Pre-School on Children's Social/Behavioural Development over the Pre-School Period. London: DfES / Institute of Education, University of London. Sammons, P., Sylva, K., Melhuish, E. C., Siraj-Blatchford, I., Taggart, B., Elliot, K. and Marsh, A. (2004), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 11 - Report on the continuing effects of pre-school education at age 7. London: DfES / Institute of Education, University of London. Sammons, P., Smees, R., Taggart, B., Sylva, K., Melhuish, E. C., Siraj-Blatchford, I. and Elliot, K. (2004c), The Early Years Transitions and Special Educational Needs (EYTSEN) Project: Technical Paper 2 - Special Educational Needs in the Early Primary Years: Primary school entry up to the end of Year 1. London: DfES / Institute of Education, University of London.
19
Schweinhart, L. J., Barnes, H. V. and Weikart, D. P. (1993), Significant Benefits: The High Scope Perry pre-school study through age 27. Monograph of the High Scope Educational Research Foundation, No. 19: High Scope Press. Siraj-Blatchford, I., Sylva, K., Taggart, B., Sammons, P., Melhuish, E. C. and Elliot, K. (2003), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 10 - Intensive Case Studies of Practice across the Foundation Stage. London: DfES / Institute of Education, University of London. Sylva, K. (1994), 'The impact of early learning on children’s later development '. In C. Ball (ed.), Start Right: the importance of early learning. London RSA. Sylva, K., Siraj-Blatchford, I., Melhuish, E. C., Sammons, P., Taggart, B., Evans, E., Dobson, A., Jeavons, M., Lewis, K., Morahan, M. and Sadler, S. (1999), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 6 - Characteristics of the Centres in the EPPE Sample: Observation Profiles. London: DfEE / Institute of Education, University of London. Sylva, K., Siraj-Blatchford, I. and Taggart, B. (2003), Assessing Quality in the Early Years: Early Childhood Environment Rating Scale Extension (ECERS-E): Four Curricular Subscales. Stoke on Trent, UK and Stirling, USA: Trentham Books. Sylva, K., Melhuish, E., Sammons, P., Siraj-Blatchford, I. and Taggart, B. (2004), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 12 - The final report. London: DfES / Institute of Education, University of London. Taggart, B., Sylva, K., Siraj-Blatchford, I., Melhuish, E. C., Sammons, P. and Walker-Hall, J. (2000), The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 5 - Characteristics of the Centres in the EPPE Sample: Interviews. London: DfEE / Institute of Education, University of London. Taggart, B., Sammons, P., Smees, R., Sylva, K., Melhuish, E., Siraj-Blatchford, I., Elliot, K. and Lunt, I. (2006), 'Early Identification of Special Needs and the Definition of 'At Risk': The Early Years Transition and Special Education Needs (EYTSEN) Project'. British Journal of Special Education, 33 (1), 40-45. Whitty, G. (2007) Making sense of educational policy. London. Paul Chapman Publishing.
20
An Evaluation of the Foundation Phase (FP) in Wales Paper 3 Abstract The Foundation Phase (FP) is a Welsh Assembly Government, national reform covering the combined ages of 3-5 Early Years and 5-7 Key Stage 1 provision. From 2004-2006, the pilot for this major policy initiative commenced in 41 early years settings across the 22 local authorities in Wales. This paper reports on The Monitoring and Evaluation of the Effective Implementation of the Foundation Phase (MEEIFP) research project, funded by the Welsh Assembly Government as a two-year evaluation. The FP advocates children learning through first hand, experiential activities and play, and places a child’s personal and social development and well-being at the heart of the curriculum; it contains seven areas of learning (AOLs). The project research questions: 1. What does research tell us about effective pedagogy and curriculum for children of ages 3-7, from recent research studies? 2. What is the quality of provision provided by the pilot settings, and is it the kind of quality that has been shown to promote children’s learning and attainment and meet the needs of the individual? 3. How do the 7 areas of learning in the Foundation Phase curriculum contribute to children’s all-round development and transition issues? 4. How does implementation differ for the Foundation Phase in the maintained and nonmaintained sectors? 5. What are the perceptions of local authority partnerships, staff, parents and governors on the impact of the Foundation Phase in the pilot settings and the issues surrounding their implementation e.g. space, adult: child ratios, ‘play’? 6. What are the main strengths of the Foundation Phase, where are the gaps which would make the implementation more successful e.g. in management or identifying training needs? Methodology and Sample The evaluation has both qualitative and quantitative components, adopting a mixed method approach, consisting of a literature review, systematic observations, field notes, semi-structured interviews, questionnaires and informal conversations involving all major stakeholders. Each setting was visited four times for observations and interviews with practitioners and parents. In addition interviews (with quantitative and qualitative components) were conducted to obtain the perceptions and opinions of all those involved in the pilot, including: head teachers/managers of early years setting, governors, Directors of Education, Early Years and Primary Advisors, representatives from the non-maintained sector and other interested Associations/groups. The paper and presentation will summarise the main findings of the evaluation and report on recommendations for: Curriculum, Pedagogy, Assessment and Transition- Best practice in the FP- Current Quality and Standards- Qualifications, Training and Ratios- Funding and Resources- Organisation and Management- Setting Level- LEA/Association Level- National Level- Parents, and Key Recommendations. This evaluation was funded to support evidence based practice and answer policy relevant questions about one country in the UK’s approach to educating its young children. It is relevant given the current debate and reforms in early years across the UK as a whole. It provides new evidence about what are the successes and short comings of implementing a new policy initiative, reference will be made to the Action Plan devised by the Department for Education, Lifelong Learning and Skills (DELLS) as a direct response to the evaluation and its findings.
21
An Evaluation of the Foundation Phase (FP) in Wales Paper 3 Authors: Iram Siraj-Blatchford, Institute of Education, University of London Emmajane Milton, University of Wales Institute, Cardiff Kathy Sylva, Department of Educational Studies, University of Oxford Janet Laugharne, University of Wales Institute, Cardiff Frances Charles, University of Wales Institute, Cardiff Introduction The Foundation Phase is a national reform programme of the Welsh Assembly Government that has been developed to improve the quality and continuity of provision for children aged 3-7 years. Two policy documents published by the National Assembly for Wales (NAfW) have been particularly influential in the development of this new initiative for children: The Learning Country (NAFW, 2001a) sets out the agenda for education in Wales up to 2010, and Iaith Pawb, (Everyone’s Language) (Welsh Assembly Government, 2002a) aims for Wales to become a more fully bilingual country by 2011. The curriculum for 3 to 5 year olds in Wales is currently provided under the six areas of learning outlined in the Desirable Outcomes for Children’s Learning before Compulsory School Age (ACCAC, 1996). The national curriculum for Wales currently applies to children between 5 and 7 years of age. The main feature of the current provision for 3-5 year olds is the variety and range that exists: from Nursery Schools and Integrated Centres to early years classes in schools (mainly nursery and reception classes), through private day nursery provision to pre-school playgroups and child minders. Equally varied and complex is the take-up of the provision by parents and carers. Indeed, for one child several different providers may be used. For example, a three year old child might begin and end the day with a child minder who would take him/ her to nursery school or class for part of the week and perhaps to a playgroup on one or two other occasions. According to the National Assembly for Wales (NAfW) statistics for 2003/4, 75% of three year olds and 80.6% of three and four year olds attend some form of nursery education (NAfW, 2005). In that year there were 34 LEA maintained nursery schools and 1,588 maintained Primary schools in Wales (NAfW, 2005). Provision for 3 – 5 year-olds is offered in both Welsh and English. The Welsh nursery association, Mudiad Ysgolion Meithrin (MYM), offers playgroups and nurseries through the medium of Welsh; 82.6% of those children transfer from the cylch to Welsh medium or bilingual Primary school (Wyn Siencyn, 2004). In some areas, usually cities, other bilingual language provision is offered, especially in school settings where bilingual assistants speak the children’s community language, through the Ethnic Minority Achievement Service (EMAS). The Foundation Phase advocates children learning through first hand, experiential activities and play, and places a child’s personal and social development and well-being at the heart of the curriculum. It contains seven areas of learning (AOLs): • Personal and Social Development and Well-being; • Language, Literacy and Communication Skills; • Mathematical Development; • Bilingualism and Multicultural Understanding; • Knowledge and Understanding of the World; • Physical Development; • Creative Development. The 2007 revised documents now replace multicultural understanding with cultural diversity in the personal and social development domain. Bilingualism remains but as a separate AOL. The implementation of the Foundation Phase in Wales has been monitored by the Foundation Phase Pilot: Final Evaluation Report (Siraj-Blatchford et al 2006) which provides the primary source for this paper, it was funded by the Welsh Assembly Government for both of these two years.
22
The Evaluation of the Foundation Phase in the first two years The following research questions were central to the evaluation: 1. What do recent research studies tell us about effective pedagogy and curriculum for children of ages 3-7? 2. What is the quality of provision provided by the pilot settings, and is it the kind of quality that has been shown to promote children’s learning and attainment and meet the needs of the individual? 3. How does implementation differ for the Foundation Phase in the maintained and nonmaintained sectors? 4. What are the perceptions of local authority partnerships, staff, parents and governors on the impact of the Foundation Phase in the pilot settings and the issues surrounding their implementation? 5. What are the main strengths of the Foundation Phase and where are the gaps the filling of which would make the implementation more successful e.g. in management or identifying training needs? Methodology and Sample The Welsh Assembly Government selected 41 pilot settings from a shortlist compiled by each local education authority (LEA), based on Welsh Assembly Government prescribed criteria. Two settings were chosen from each of the 22 LEAs in Wales with the exception of three local authorities where there were no non-maintained sector settings funded to provide education. The settings covered a wide range of early education and childcare provision in Wales. Although the sample included all types of childcare and education providers the settings were not randomly selected by LEAs and therefore cannot be considered fully representative of settings across Wales. The evaluation had both qualitative and quantitative components. A mixed method approach was adopted, consisting of a literature review, systematic observations, the analysis of field notes, semi-structured interviews, questionnaires and both formal and informal conversations involving all of the major stakeholders. In the course of the evaluation process one field researcher visited each of the pilot settings four times, twice during each academic year 2004-2006; observations were conducted, practitioners and parents were interviewed. In addition, both in the first and the second year of the pilot, the research team devised, piloted and conducted semi-structured interviews and questionnaires with quantitative and qualitative components to obtain the perceptions and opinions of all those involved in the pilot, including: head teachers, school governing bodies, owners/ managers/ management committees of non-maintained settings, LEA Directors of Education, LEA Early Years / Primary Advisors, Early Years Development and Childcare Partnerships (EYDCP), representatives from the non-maintained sector Associations, the Early Years Education and Safeguards Team (EYEST) at the Department for Education, Lifelong Learning and Skills (DELLS) and representatives from institutions delivering Initial Teacher Education and Training (ITET) and courses in Early Education and Childcare (Further Education). High response rates were achieved during the study from every stakeholder group (mostly 100%, 75% for some groups) which ensured strong triangulation and more reliable data and findings being provided for evidence based policy development. Four observational instruments were used to assess quality in the pre-school, Nursery, Reception and Year 1 or mixed Year 1/2 settings: 1. The Early Childhood Environment Rating Scale (ECERS-R) (Harms, Cryer and Clifford 1998) has been used extensively as a means of measuring quality of provision for 3-5 year olds in Early Childhood Settings since its publication in the U.S. The scale has been adapted for use in different countries and translated into a number of languages. The basic scale has remained the same, with assessment of the quality of 7 domains of provision: Space and Furnishings; Personal Care Routines; Language-Reasoning; Activities; Interaction; Program Structure; Parents and Staff.
23
2. In the U.K, the Early Childhood Environment Rating Scale Extension (ECERS-E) (Sylva, Siraj-Blatchford and Taggart, 2003 second edition 2006) was developed to supplement the ECERS-R, for use in the Effective Provision of Pre-school Education (EPPE) project (Sylva et al, 2004), because the ECERS-R did not examine in depth provision for developing emerging literacy, numeracy and scientific thinking. It also included a more detailed subscale to look at diversity issues such as gender, racial equality and meeting the needs of individual children. 3. The Caregiver Interaction Scale (CIS) consists of 26 items focusing on the interactions between caregivers and children. The items are grouped to produce 4 subscales: positive relationships, punitiveness, permissiveness and detachment. The CIS was developed by Arnett (1989). • ‘Positive relationships’ is a subscale made up of 10 items indicating warmth and enthusiasm interaction with children by the caregiver. • ‘Punitiveness’ is a subscale made up of 8 items indicating harsh or over-controlling behaviour in interaction with children by the caregiver. • ‘Permissiveness’ is a subscale made up of 4 items indicating avoidance of discipline and control of children by the caregiver. • ‘Detachment’ is a subscale made up of 4 items indicating lack of involvement in interaction with children by the caregiver. 4. It was also decided that an adapted version of the Assessment of Practices in Early Elementary Classrooms (APEEC) Hemmeter, Maxwell, Ault, Schuster (2001) would be appropriate as its basic format was the same as the ECERS, items which were not appropriate for use in a British/Welsh classroom were removed and observations were made on a range of items covering the physical environment, instructional context, and social context. In order to maintain consistency throughout the evaluation it was necessary in addition to make observations on provision for Diversity within Year 1 (and mixed Year 1/2) classrooms. The decision was therefore made to include the items on this area from the ECERS-E instrument as, although aimed at 3-5 year olds, they were not age specific and therefore were considered appropriate in this context. Both elements were therefore combined to create an entirely new scale ECERS-EP which was used in conjunction with the CIS (Caregiver Interaction Scale) and revised versions of the new Welsh subscales amended again to be appropriate for use in Year 1 (and mixed Year 1/2) classrooms (See Siraj-Blatchford et al, 2006). It was also considered essential that we create two entirely new subscales to cater for the assessment of quality of provision for Welsh as a Second Language in English medium schools and for Welsh as a first language in Welsh medium schools (See Siraj-Blatchford et al 2006). All of these observational instruments were checked for reliability and validity (see Siraj-Blatchford et al 2006, Appendix D). The observers were trained in the use of systematic observations of quality for this age range and they achieved good inter-observer reliability with other teams in England and Northern Ireland. Summary of Findings from the research The Quality of Provision in the Pilot settings Observations were made in 2005 and 2006 applying the pre-school rating scales for assessing quality (ECERS-R, ECERS-E and CIS). The observations were made in 51 Foundation Phase Pilot settings (since some of the 41 FP pilots had more than one FP setting e.g. a nursery class as well as a reception class). In 2006 quality observations (using the early primary rating scale, ECERS-EP) were also made in the 20 Year 1 or mixed Year 1/2 settings which had joined the pilots.The proportion of trained teachers was found to be directly related to quality in all of the pilot settings and classes. The higher the proportion of trained teachers within a setting or classroom, the higher the quality. Results on the total score on the ECERS-R (which assesses daily routines, social interactions and facilities) showed no significant differences between the two years in settings with 3-5 year old children. However, when looking at subscales individually, settings showed significantly lower quality on the ‘Interaction’ scale in 2006 than they had in 2005. Graph 1 below shows the
24
comparison of ECERS-R quality scores from 2005 and 2006. The vertical axis shows the range of scores on the ECERS-R (the lowest score possible is 1 and the highest is 7). Note that the asterix above the bars for interaction denote a statistically significant difference. Interaction was the only subscale to show significant decline in quality but this does suggest that the quality of staff-child and child-child interaction had lessened over the course of one year. Graph 1: Comparison of quality in 2005 and 2006 in settings with 3-5 year olds (as measured on ECERS-R)
Graph 2 below shows the comparison of ECERS-E quality scores in 2005 and 2006. Note that the star above the literacy subscale, and the total score bars denote reductions in quality which are statistically significant. The literacy subscale measures quality in settings in relation to environmental print, book and literacy areas, whether adults read with children, emergent mark making (early writing), talking and listening and whether adults draw attention to sounds in words. Graph 2: Comparison of quality in 2005 and 2006 in settings with 3-5 year olds (as measured on ECERS-E)
Quality scores on the Caregiver Interaction Scale (CIS, which assesses how responsive and sensitive the staff are as they interact with children) showed no significant differences between 2005 and 2006. The results from the ECERS-EP (only used in Year 1/2 classes) showed the highest scores on the ‘use of materials’ item, with the lowest scores on the three items of the ECERS-E diversity subscale. These three items relate to the tailoring of the curriculum and pedagogy to individual interests and needs.
25
Implementation in the Maintained and Non-maintained sectors To explore the differences in quality scores across maintained and non-maintained settings we compared the mean ECERS-R, ECERS-E and CIS scores of the two groups (maintained or non-maintained). In both 2005 and 2006 overall quality, measured by the total score on the ECERS-R and ECERS-E, was significantly higher in the maintained settings. In particular, maintained settings scored higher on the ‘language-reasoning’, ‘activities’ and ‘parents and staff’ subscales of the ECERS-R and all four subscales of the ECERS-E. However, non-maintained settings scored higher on the ‘personal care routines’ subscale of the ECERS-R for children 3-5 years. For the fine grained detail on these observations see the main body of the report (SirajBlatchford et al 2006) and the appendices which give ample examples of ECERS subscales and rating procedures. Stakeholder Perceptions Curriculum, Pedagogy, Assessment and Transition Generally, most stakeholders have received the Foundation Phase Curriculum with enthusiasm, at least 85% of respondents agreeing that the seven AOLs would provide a broad and balanced basis for young children’s learning and development during the Foundation Phase. Similarly, high numbers from all stakeholder groups agreed with the emphasis on ‘active learning and play’, although the importance of maintaining balance and matching learning to meet all children’s needs (which is apparent in the documentation) was raised as a challenging issue. Substantial numbers of all respondents also continued to report that Bilingualism and Multicultural Understanding had been particularly difficult to introduce. The non-maintained sector perceived Bilingualism and Multicultural Understanding to be less difficult to introduce than the maintained sector as they felt it was already part of their existing practice, although scores on ‘diversity’ in the ECERS-E observational ratings for 2006 show that both sectors need to improve significantly in this area. The ACCAC (now Curriculum and Assessment Division 3-14 - CAD 3-14) 2004 Draft Framework, was considered to be appropriate for planning for Foundation Phase, particularly in terms of style, approach and emphasis on skills development and active learning but most stakeholders felt that the layout and usability needed to be improved. The materials are now being revised in light of these findings. In terms of planning, a majority of practitioners from both sectors reported they felt greater flexibility and freedom. Consequently there was less attention paid to adult pedagogy, and in some settings staff was not always gainfully and appropriately deployed. Higher ECERS observed scores, which measure quality, were noted particularly in the small minority of settings where planning was reported to be more detailed and specific. Current Quality and Standards Stakeholders showed throughout the evaluation overwhelming support for the philosophy and aims of the Foundation Phase. All stakeholders were asked whether they felt there had been any changes in the standards of children’s learning and achievement and well-being, confidence, independence and maturity. Interestingly perceptions were mixed but very similar across all stakeholder groups. Although a majority felt that there had been improvements, a notable percentage of all stakeholder groups felt that it was too early in the pilot to make judgements on standards and at this stage it was impossible to draw firm conclusions. Qualifications, Training and Ratios Although there was overwhelming support from the maintained sector for the proposed ratios of one adult to eight children for Nursery and Reception classes, with respondents feeling that it had made a difference to the experience of the children they teach, many respondents found it difficult to give specific examples of these benefits and invariably gave examples about how classroom management and organisation had improved. This was reflected by the research observations where the biggest change identified was that some aspects of teacher workload
26
(e.g. planning, assessment and teaching for a set number of children) were being redistributed to the additional adults rather than having those adults provide higher quality learning experiences and interactions with the children. As previously noted, the quality observations showed a decline in the quality of provisions for literacy and interactions. While an increased number of settings were using observations as an assessment tool and were using all staff in assessment procedures settings in both sectors identified the need for further guidance and support with assessment for the FP. A stronger view among Practitioners, EY/Primary Advisors and Directors of Education was that the provision of higher trained staff would have more impact on children’s learning. Typical arguments centred on the ability of trained staff to be able to support and extend children more appropriately. These responses corroborate the findings from our quality observations which demonstrate that settings with a higher proportion of trained teachers have higher quality of provision. Funding and Resources Generally there are high levels of satisfaction with the funding allocation for staffing in the maintained sector. However stakeholders in both sectors feel that the allocations for settings £3000 (maintained sector) and £800 (non-maintained sector) to cover resources, management and training are inadequate especially because of the inequity between the two sectors and as the allocation is not linked to setting size, number of children or classes or number of staff. In the majority of LEAs there have been some improvements in the second year in providing clear channels for accessing funding although there is still considerable improvement required to facilitate this for the non-maintained sector. In respect of space and accommodation it is clear that further work needs to be done, as changes that schools are making are being self financed because there is no ring-fenced funding stream available for developments to existing accommodation (both indoors and outdoors). Organisation and Management Around 50% of the practitioners felt that their role continued to change in the second year of the pilot. In the non-maintained sector this was reported to be due to increased levels of paperwork. However, in schools practitioners felt it was as a result of needing to lead and manage additional numbers of support staff. Practitioners also reported that the roles of support staff had changed and they were now more responsible for teaching and assessment of small groups of children. Almost all head teachers agreed that to implement the Foundation Phase changes in roles were necessary, with limited training staff they would struggle to fulfil these roles. Stakeholders from the non-maintained sector reported the difficulties which were persistent in the past and that they continue to face, in the following areas: lack of funding and inequity in its allocation;, low staff wages; inadequate and inappropriately pitched training; unavailable and inaccessible supply cover; poor communication with, and a lack of recognition from, other professionals; the nature of existing premises and outdoor facilities. All stakeholder groups were overwhelmingly supportive of the postponement of the roll-out to 2008 as they felt it would allow time to prepare more fully in terms of workforce development, training needs, support structures and in raising the Foundation Phase profile with parents and practitioners. Parents Overall parents’ responses were very positive to the key components of the proposed Foundation Phase but despite their overwhelming support for active learning many parents took the opportunity to express their concerns that ‘teachers’ should respond to individual needs and offer a balanced approach, as specified in the guidance documents. Although a large majority of parents reported that they were satisfied with the way in which their child was being taught it was clear from their comments that many parents do remain concerned about the teaching of
27
basic skills (especially reading and writing) and the apparent lack of structure in the provisions for older children. Major Strengths and Recommendations Some settings have made very good advances in their implementation of the Foundation Phase within the short period of the evaluation. The best settings in terms of implementing the Phase were found to have the following common characteristics: • Detailed and focused planning. • Lead practitioners with good leadership and management skills and the ability to allocate effective roles for other adults whilst planning together for children’s learning • Guided and supported play activities with higher levels of adult child interaction that support children’s thinking. • Clear and dynamic vision and leadership from setting heads who have a good grasp of effective early years practice and are able to communicate this effectively to staff. • The best settings did not slavishly adhere to the Foundation Phase guidance but took it seriously and built on existing good practice. • A move away from over-formal practice in the basics towards a more experiential, child centred and adult guided, play based practice. • The leadership of the setting has a culture of investing in staff development. • Some well trained and qualified staff who have a good understanding of child development and pedagogy and who actively support other staff in working with children. The evaluation found that further support was needed to develop and maintain play based and experiential pedagogies giving sufficient emphasis to activities that involve adult guided play and learning and interaction with appropriate challenge. There was also a need to assure an appropriate balance between academic and social emotional aims (as currently emphasised in the documentation) in the curriculum and in day-to-day and long term planning. Although many areas of quality as measured by the ECERS scales remained the same across the two years, the perception amongst many practitioners and some advisors of improvements in language development and oracy skills contrasted significantly with the lower quality observed in 2006, especially with respect to the ECERS-E Literacy subscale. The evaluation showed a decline in children’s opportunities for learning in the areas of literacy and in interaction with adults. A greater emphasis on prioritising these areas of learning was therefore recommended along with caution not to return to the over formal pedagogy that characterised some previous practice. Weaknesses were identified in terms of assessment and in the provision for individual needs. The transition arrangements between the maintained and non-maintained sectors were also found to be inconsistent. Considerable levels of concern over the transition of children from the Foundation Phase to Key Stage 2, due to the perceived differences in content and approach were also noted. While some of these issues are not as a result of the implementation of Foundation Phase, the success of the Phase will be dependent on such improvements. It was felt that a common assessment system would be useful across all sectors and would also facilitate children’s transitions and family moves. Given the difference in the quality of provisions identified by the ECERS-E, the evaluation recommended that progress needed to be made to achieve a situation in which the non-maintained sector settings received similar levels of training, resources and support to those available in the maintained sector, to ensure future comparability in standards and quality. The evaluation confirmed findings of other studies that emphasised the importance of providing trained teachers and it identified a tension between lower ratios and highly qualified, better paid staff. The evidence suggesting some reductions in the use of planning was considered particularly worrying. It was therefore felt that the training requirements for staff, especially on the pedagogy of the Foundation Phase need to be recognized. Guidance (or, in some cases,
28
statutory orders) is required for training providers (LEA, ITET and FE) to develop plans for meeting the needs of the Phase within a set period of time. This could be led by the CAD 3-14 team at DELLS, in collaboration with the workforce review. The evaluation found that communication to all stakeholders was perceived as very arbitrary and poor. The evaluation recommended the appointment of regional advisors and Foundation Phase advisors at LEA level could liaise with and support non-maintained Associations in collaboratively implement the Phase in all their member pre-school settings. The evaluation also recommended the need for a visible action plan which outlined the strategies, training and resource allocation for effective roll-out by 2008. Concluding Discussion The Foundation Phase Pilot Final Evaluation Report (Siraj-Blatchford et al, 2006) was published in December 2006 and the clearest initial response may be found in ‘Building the Foundation Phase: Action Plan published by DELLS in the same month (DELLS, 2006). In her foreword to the document the Minister for Education, Jane Davidson re-emphasises the importance of promoting pedagogic practices that involve ‘responsive interactions between children and adults’. The foreword also emphasises the commitment of the Welsh Assembly Government to evidence-based policy development. In making explicit reference to the Foundation Pilot Evaluation she argues that: “Its recommendations, many of which are already being implemented, are incorporated into this plan”. The Action Plan does indeed provide a framework for optimism regarding the future roll-out of the Foundation Stage. The provisions for reduced adult-child ratios are not compromised, not surprisingly given the popularity of low ratios. The Action Plan provides details of a national development and training pack that is to be developed with an emphasis on “the role of play, including adult guided play”, curriculum planning and assessment (the new Foundation Phase profile), balance between emotional and social, and academic aims, transition and literacy. More importantly, the challenge of better and more successful leadership of the implementation is met head-on with the promised recruitment of a Foundation Phase advisor for every local authority and four regional, consortia advisors for the Phase. In addition the Plan sets out a new and more proactive structure for ensuring delivery through the development of key Foundation Phase management committees with a specific remit to oversee the strategic development of the Phase across Wales. However, some areas of policy remain less clear. Although the training packs and appointment of staff to support new pedagogies are welcome it is unclear how courses in initial training will directly support the development of new and existing staff in the early years. The workforce review (Dallimore, 2007) is currently grappling with some of the key training issues and the role of existing courses such as the new foundation degrees, childcare courses in further education and initial teacher education in higher education. The key question of supplying sufficient teacher input, particularly for the 3-5 age group, is not fully resolved and although more funding has been made available it is not yet clear whether this large scale national reform, with the challenges facing those implementing it, is sufficiently funded to provide adequate support. What is clear is that the Assembly Government is implementing a popular educational reform which most stakeholders believe will improve educational opportunities for most children. The challenge that lies ahead is to demonstrate that Wales can deliver this effectively and in a way that improves children’s well-being as well as educational outcomes, especially for those children and families from more disadvantaged backgrounds.
29
The First Six Years at School Paper 4 Abstract This paper addresses the cumulative long-term impact of successive years of high quality provision in schools. This will be achieved by looking at the long-term consequences, up to the age of 11 (Year 6), for children who has been in a school where there was high or low valueadded in the Reception Year, Year 1, Year 2 and so on. The relative contribution of each will be assessed independently, as will the combination of two and more years of good, or bad, valueadded results in a row in a school. The motivation for this work comes from tantalisingly important findings from Tennessee (Sanders and Rivers 1996). They report that: “Differences in student achievement of 50 percentile points were observed as a result of teacher sequence after only three years.” And claim that: “The effects of teachers on student achievement are both additive and cumulative with little evidence of compensatory effects.” We will look for evidence of similar trends in English data. The PIPS project has been collecting data on pupils’ attainment and attitudes in schools in England since 1991 and a growing data base has been established. It is therefore possible to look at progress of individual children suing assessments at the end of every year as well as on entry to school at the age of 4. By interrogating this unique dataset the progress of children can be tracked and related to the progress of a representative sample of children in England. We have shown in previous papers (Tymms et al, 1997, Tymms et al 2000) that attendance at a school with good value-added scores during the Reception Year has an important impact within one year and as far ahead as the end of Key Stage 1. This paper will go a much further and look for the impact of a good reception year on performance at the end of the primary period and then extend the analysis to look for cumulative and compensatory effects. The final outcome in Year 6 which will include two sets of measures. One is the Key Stage 2 results and the other is the PIPS parallel assessments which include attitude measures. The results of this research should have important consequences for policy since, if the US findings are confirmed they suggest that teachers are at the heart of educational success, not policies nor leaders. It would suggest that that the classroom and the teacher should be at the forefront of initiatives and reform. It will quantify the importance that we should place on such a perspective and thereby address an issue of vital importance. References Sanders, W.L. and Rivers, J.C (1996) Cumulative and Residual Effects of Teachers on Future Student Academic Achievement, University of Tennessee Value-Added Research and Assessment Center,. Tymms, P., C. Merrell and B. Henderson (1997). "The First Year at School: A quantitative investigation of the attainment and progress of pupils." Educational Research and Evaluation 3(2): 101-118. Tymms, P. B., C. Merrell and B. Henderson (2000). "Baseline assessment and progress during the first three years at school." Educational Research and Evaluation 6(2): 105-129.
30
The First Six Years at School Paper 4 Authors: Peter Tymms University of Durham Paul Jones University of Durham Stephen Albone University of Durham Brian Henderson University of Durham Please do not quote without permission
Introduction The motivation for this work comes from tantalisingly important findings from Tennessee (Sanders and Rivers 1996). They report that: “Differences in student achievement of 50 percentile points were observed as a result of teacher sequence after only three years.” And claim that: “The effects of teachers on student achievement are both additive and cumulative with little evidence of compensatory effects.” This paper looks for evidence of similar trends in English data. In previous work (Tymms et al, 1997, Tymms et al 2000) it was shown that attendance at a school with good value-added scores during the Reception Year has an important impact within the year and as far ahead as the end of Key Stage 1. This paper aims to go much further and look for the impact of a good reception year on performance at the end of the primary period and then extend the analysis to look for cumulative effects throughout primary schooling.
Data The Performance Indicators in Primary Schools (PIPS) project has been collecting data on pupils’ attainment and attitudes in schools in England since 1993 (www.pipsproject.org; Tymms 1999) and a growing data base has been established. Information was available for assessments carried out at the start of Reception (SR), end of Reception (ER) and every year from Year 1 (Y1) to Year 6 (Y6) in England. It is therefore possible to look at the progress of individual children from the start of school at the age of 4 and thereafter at the end of every year. Data were available on 73347 pupils who started school in England in 1999 and who were tracked to the end of Year 6 in 2005 and for whom there were assessments on at least two occasions. Schools buy into the PIPS project and they choose which years to buy into. As a result just a few schools did every assessment and data were available for 3507 pupils who had been assessed every year. More details on the dataset are given below. No attempt was made to identify classes but it should be noted that there were on average 243 pupils per school (SD 18.7) in a year group. This amounts to about one class per school. Some times a class will be made up of mixed years and sometimes there will be several classes in a year group. The most usual arrangement is one class per year group. So, although the analysis will refer to schools it should be understood that the “school” in one year is often the class. The analyses concentrate on attainment levels which were collected through a series of assessments, the reliabilities of which are given in Table 1. Table 1 Reliabilities of the cognitive measures
Variable Mathematics Reading Vocabulary
SR 0.89 0.87 0.74
ER 0.9 0.97 0.74
Y1 0.89 0.95 0.77
Y2 0.90 0.98 0.86
Y3 0.92 0.95 0.88
Y4 0.90 0.97 0.87
Y5 0.92 0.96 0.89
Y6 0.93 0.94 0.90
3 The estimate is based on the matched data and it will be a slight underestimate as not all pupils could be matched across the years.
31
All of the continuous variables were normalised and given a mean of zero and a standard deviation of 1. Other data such as sex and deprivation were available but the analyses concentrated simply on each area of attainment separately, and this in itself produced very complex models which explained a considerable proportion of the available variance. Further more complex analyses of the data are possible. Analyses Multi-level models were constructed in which attainment at each time point was nested within pupils and pupils were nested within schools. This procedure allowed us to estimate the correlations between pupils’ scores and the schools’ average scores year on year. It also allowed us to introduce measures from previous years to “explain” measures in later years. Models for vocabulary The vocabulary analyses were restricted to pupils on whom there were at least 5 separate data points. This meant that information from 1140 schools was analysed. The number of pupils used each year is set out in Table 2. Table 2 Vocabulary Data
Data collection point Start of reception End of reception Year 1 Year 2 Year 3 Year 4 Year 5 Year 6
Abbreviation
Time of year
N of pupils
SR ER Y1 Y2 Y3 Y4 Y5 Y6
September
16630 19900 17822 19980 16505 20042 15469 14750
June
Summer January
Summer Summer Summer January
The basic model for vocabulary, the null model, is shown in Table 3.
32
Table 3 Vocabulary: null model Fixed Cons Random School ER Y1 Y2 Y3 Y4 Y5 Y6 Pupil ER Y1 Y2 Y3 Y4 Y5 Y6 %@ school
SR
ER
Y1
Y2
Y3
Y4
Y5
Y6
0.030 (.013)
0.017 (.012)
-0.047 (.012)
-0.047 (0.012)
-0.109 (0.012)
-0.091 (0.011)
-0.127 (0.012)
-0.127 (0.012)
0.298 (0.013) 0.215 0.212 0.197 0.215 0.187 0.195 0.178
0.248 (0.011)
0.240 (0.011) 0.701
0.246 (0.11) 0.673 0.891
0.257 (0.011) 0.668 0.866 0.896
0.219 (0.010) 0.657 0.851 0.891 0.923
0.229 (0.011) 0.661 0.879 0.884 0.937 0.922
0.222 (0.011) 0.624 0.847 0.873 0.901 0.920 0.918
0.718 (0.008) 0.482 0.381 0.388 0.381 0.379 0.365 0.364 29.6
0.694 (0.007)
0.171 0.166 0.168 0.153 0.157 0.147
0.368 0.375 0.365 0.369 0.351 0.353 26.3
0.216 0.215 0.195 0.206 0.195 0.710 (0.008) 0.524 0.455 0.439 0.442 0.410 0.421 25.3
0.225 0.207 0.210 0.204 0.687 (007) 0.543 0.652 0.483 0.502 0.469 0.471 26.4
0.219 0227 0.215 0.698 (0.008) 0.525 0.624 0.697 0.527 0.514 0.503 26.9
0.206 0.203 0.735 (0.008) 0.517 0.613 0.707 0.736 0.532 0.562 23.0
0.207 0.696 (0.008) 0.504 0.584 0.678 0.738 0.743 0.528 24.8
0.727 (0.008) 0.497 0.586 0.666 0.706 0.769 0.742 23.4
a) Standard Errors (SE) are shown in parentheses b) In the School section the variances, covariances and correlations are shown. For the sake of clarity SEs are not shown for the covariances – they are very similar to errors on the variances. Numbers in bold to the top right are correlations. c) Similar results are shown in the Pupil section The null model is based on normalised vocabulary scores and as expected the mean for each year was close to zero. About a quarter of the variance each year was associated with the school and the correlations between the pupil scores were around the 0.7 level. For the younger children the correlations were lower presumably because the reliabilities of the vocabulary measures were lower. At the school level the correlations got up to 0.94 showing considerable stability for the average score of a cohort year on year. The next stage was to create a model in which the vocabulary scores from previous years were used to predict the vocabulary score of that year. This is shown in Table 4.
33
Table 4 Vocabulary Full Model Fixed Cons
SR
ER
Y1
Y2
Y3
Y4
Y5
Y6
-0.033 (.014)
-0.023 (.009) 0.686 (0.006)
-0.094 (.010) 0.355 (0.009) 0.300 (0.010)
-0.047 (0.009) 0.198 (0.009) 0.158 (0.009) 0.496 (0.008)
-0.099 (0.009) 0.129 (0.010) 0.076 (0.010) 0.239 (0.010) 0.443 (0.010)
-0.053 (0.009) 0.030 (0.010) 0.041 (0.010) 0.135 (0.010) 0.297 (0.011) 0.413 (0.011)
-0.048 (0.009) 0.089 (0.010) 0.012 (0.010) 0.057 (0.011) 0.133 (0.012) 0.347 (0.012) 0.337 (0.012)
-0.040 (0.010) -0.013 (0.011) 0.020 (0.011) 0.066 (0.012) 0.076 (0.014) 0.129 (0.014) 0.387 (0.014) 0.286 (0.014)
0.279 (0.013) 0.009 0.012 0.002 0.017 -0.010 -0.002 -0.003
0.095 (0.005)
0.090 (0.006) -0.083
0.047 (0.004) -0.055 0.329
0.041 (0.004) -0.189 0.089 0.122
0.028 (0.003) -0.036 0.004 0.044 0.094
0.021 (0.003) -0.080 0.335 0.112 0.201 0.072
0.027 (0.004) -0.051 0.153 0.170 0.063 0.083 0.133
0.690 (0.008) -0.028 -0.025 -0.018 -0.017 0.017 0.002 0.002
0.357 (0.004)
28.8
21.0
16.3
12.1
11.8
9.3
9.1%
10.4%
6.4%
44.2%
62.5%
80.9%
84.0%
87.2%
90.8%
87.8%
3.9%
48.6%
35.1%
50.5%
56.2%
63.0%
65.9%
68.1%
SR ER Y1 Y2 Y3 Y4 Y5 Random School ER Y1 Y2 Y3 Y4 Y5 Y6 Pupil ER Y1 Y2 Y3 Y4 Y5 Y6 % @ school Explained @ school Explained at Pupil
-0.008 -0.004 -0.012 -0.002 -0.004 -0.003
0.005 -0.002 0.008 0.002 0.003 -0.001
0.021 0.005 0.000 0.015 0.008 0.461 (0.006) 0.011 -0.022 -0.005 -0.001 -0.012 -0.008
0.005 0.002 0.004 0.006 0.340 (0.005) -0.006 -0.055 -0.006 0.000 -0.004 -0.005
0.003 0.006 0.002 0.306 (0.005) 0.023 -0.013 -0.017 -0.002 -0.006 0.000
0.002 0.002 0.272 ((0.005) 0.006 -0.003 -0.001 -0.007 -0.001 -0.003
0.003 0.237 (0.005) 0.012 -0.035 -0.013 -0.021 -0.005
0.232 (0.005) -0.004 -0.023 -0.019 -0.002 -0.013 -0.014
-0.003
In the Full Model the prediction is increasingly successful as the pupils become older and as more data on prior vocabulary scores become available. At the end of Reception nearly half of the variance at the pupil level and at the school level is “explained”. By Y6 about two thirds of the pupil level variance and nearly 90 per cent of the school level variance is “explained”. The correlations between the residuals over the years are close to zero at the pupil and the school level. An average pupil’s vocabulary at an average school might be tracked from Reception to Y6. Being an average pupil in an average school, the pupil would be expected to maintain an average performance throughout the time. Since the mean was given a value of zero the pupil is expected to score zero each year. But if that pupil experienced a particularly successful Reception year then his or her attainment level would be boosted and some advantage may be
34
maintained for the rest of primary school. This idea was tested by constructing Figure 1 as follows. Imagine a pupil who attended a school during the Reception year which made above average progress to the tune of one standard deviation of the schools’ value-added scores. This impetus will be referred to as a “boost”. The boost would be equivalent to about 0.3 standard deviation units. This can be calculated from Table 4 above. The variance at the school level for ER is 0.095 and its square root gives the SD of schools’ value-added scores at the end of reception. The model can then be used to indicate what the same pupil’s attainment would be at the end of Y1 in a school which made average progress. This can be calculated by multiplying the boost by the coefficient for the ER measure when predicting Y1 scores. The coefficient is 0.3 and so the attainment was nearly 0.1 and at the end of Y1. Similar but more complex calculations indicate that this level of attainment remains approximately the same until Y6 assuming that the school’s value-added was zero each year after Reception. Figure 1 shows this relationship as the lowest line. The chart confirms a major drop in the first year following the boost and a constant advantage thereafter. Figure 1 Vocabulary: levels after boosts in various years 0.80
Vocabulary level in SD units
0.70 0.60 ER Y1 Y2 Y3 Y4 Y5
0.50 0.40 0.30 0.20 0.10 0.00 ER
Y1
Y2
Y3
Y4
Y5
Y6
Year
If a boost were experienced in the second year (Y1) and if this directly followed the boost from the Reception year then the average pupil would be lifted to about 0.4 of a standard deviation above the average. This advantage would fade a little the next year but, as with the Reception boost, the fade would be mainly in the immediate next year. It would then remain at about a third of the standard deviation thereafter. Similar patterns are seen in Figure 1 if boosts are added in the next years. If a pupil received a boost each year until Year 6 the advantage would be about 0.7 SD units – an educationally important gain. This would be expected to fall the next year but still to be above about two thirds of a standard deviation unit. This analysis and the chart show a number of interesting features. a) The largest boosts come in the earlier years. The earlier the bigger. b) The boosts give some long lasting advantages throughout primary schooling. The long term gain is similar for boosts in the first few years but smaller later. c) The boosts can be added to year on year.
35
d) A boost in Reception followed by boosts in the next three years results in a modest educational advantage of about a third of a SD for years to come. It should be noted that whilst a boost is an advantage to some there is an equal and opposite group of pupils who will have experienced setbacks. Boosts have mirror images. According to the model boosts and setbacks within one school in different years for the same cohort are unrelated to one another (the correlations at the school level are close to zero). In order to emphasise this point the chart below plots the fortunes of six hypothetically pupils from different schools who were given a random boosts/setbacks each year. Figure 2 Cummulative results for 6 pupils with random value-added scores each year 1.00
0.80
0.60
Attainment
0.40
0.20
0.00 ER
Y1
Y2
Y3
Y4
Y5
Y6
-0.20
-0.40
-0.60
-0.80
-1.00
All six pupils varied across the years but one had a very stable experience hugging the zero line for the seven years. Aother had a dramatically poor Year 1 from which he or she never quite recovered finishing primary school about two fifths of a SD below average despite being average on entry to school at the age of 4. How often will a pupil experience boosts? The boost was defined as a school having a valueadded score of one standard deviation above the average. This indicates that 16% of schools will give a boost to their pupils in any one year. The chance of a pupil getting a boost in Reception followed by Y1 is therefore 2.5%4. The chance of a pupil getting a boost every year is two in a million. It is far more likely that a pupil will experience a mixture of boosts, setbacks and average years as in Figure 2. Sometimes pupils will get a greater boost than that defined for this paper and sometimes a setback. Using a series of simulations we estimate that the SD of scores at the end of primary school which result from a random normal mixture of school advantageous and disadvantageous value-added scores is 0.3. This suggests that a small proportion of schools (2.5%) will have pupils going to secondary schools with vocabulary scores 0.6 standard deviation units better than expected although this is likely to fade a little to about half a standard deviation. Could it be that the pattern shown in the data are the result of the information that happen to be available? This is similar to asking if the same result would be seen if the models were not perfect reflections of what had happened. To some extent the answer must be yes. The greater variance “explained” for younger pupils must surely be at least partly because less information was available than for older pupils. On the other hand the model does “ring true” in that it matches other analyses (Sanders & Rivers 1996; Tymms et al 2000) and general perceptions of schools, classrooms (Campbell et al 2004) and the development of ability (Dickens 2007). 4 This is simply 0.16*0.16 since the correlation between school year on year is zero.
36
A difficulty In Years 2 and 6 pupils in state schools must take the statutory assessments and so many state schools do a basic version of PIPS in which only attitudes, vocabulary and non-verbal ability are measured. At the time of writing statutory test data were not available for some schools and the multi-level models would not converge. So for the maths and reading models which follow data from Years 2 and 6 were omitted. Models for mathematics The initial model used the same structure as the vocabulary analysis in which the maths attainments were nested within pupils and pupils were nested within schools. This basic model, the null model, is shown in Table 5. Table 5 Mathematics: null model Fixed Cons Random School ER Y1 Y3 Y4 Y5 Pupil ER Y1 Y3 Y4 Y5 %@ school
SR
ER
Y1
Y3
Y4
Y5
0.097 (.010)
0.022 (.010)
0.016 (.010)
-0.028 (0.011)
-0.021 (0.011)
0.040 (0.012)
0.186 (0.007) 0.089 0.094 0.099 0.109 0.115 0.837 (0.005) 0.837 0.597 0.537 0.506 0.477 18.2
0.200 (0.007)
0.179 (0.007) 0.608
0.157 (0.008) 0.621 0.716
0.166 (0.007) 0.604 0.714 0.842
0.170 (0.008) 0.201 0.236 0.265 0.277
0.115 0.110 0.110 0.117 0.817 (0.005) 0.817 0.594 0.562 0.537 19.7
0.120 0.123 0.130 0.830 (0.005) 0.721 0.604 0.581 0.581 17.7
0.136 0.137 0.835 (0.007) 0.680 0.726 0.566 0.657 15.9
0.147 0.835 (0.006) 0.650 0.698 0.786 0.657 16.6
0.820 (0.007) 0.655 0.704 0.794 0.794 17.2
In the null model the average maths score in each year was very close to zero as expected. The correlations between the pupils’ scores one year and the next were high - between 0.65 and 0.84.The correlations between the schools average scores were modest to high varying from 0.20 to 0.84. The variance was partitioned each year between the school and the pupil. Between 15 and 20 per cent resided with the school. The full model is shown in Table 6.
37
Table 6 Mathematics Full Model Fixed Cons
SR
ER
Y1
Y3
Y4
Y5
0.097 (.010)
.0022 (.010) 0.655 (0.003)
0.016 (.010) 0.28 (0.005) 0.506 (0.010)
-0.028 (0.011) 0.137 (0.009) 0.259 (0.010) 0.432 (0.009)
-0.021 (0.011) 0.050 (0.009) 0.089 (0.011) 0.202 (0.010) 0.570 (0.010)
0.040 (0.012) 0.041 (0.009) 0.047 (0.010) 0.132 (0.010) 0.343 (0.012) 0.381 (0.011)
SR ER Y1 Y3 Y4 Random School ER Y1 Y3 Y4 Y5 Pupil ER Y1 Y3 Y4 Y5 % @ school Explained @ school Explained at Pupil
0.186 (0.007) -0.035 -0.004 0.001 0.005 0.008 0.810 (0.005) 0.031 0.001 0.004 -0.003 -0.011 18.7
0.158 (0.006)
0.012 0.009 0.009 0 29.3
0.009 0.002 -0.005 21.7
0.001 0.003 15.2
-0.003 13.6
12.7
0%
21%
42.5%
62.4%
74.1%
81.2%
0%
53.3%
55.3%
60.5%
67.3%
73.3%
-0.011 -0.014 -0.012 0.002 0.381 (0.003)
0.103 (0.005) -0.086 -0.008 -0.003 0.000 0.371 (0.003) 0.032
0.059 (0.005) -0.145 -0.103 -0.002 -0.005 0.330 (0.005) 0.025 0.026
0.043 (0.004) -0.146 -0.045 -0.040 0.004 0.273 ((0.004) 0.028 0.006 0.003
0.032 (0.003) 0.028 0.000 -0.115 0.108 0.219 (0.004) 0.000 -0.018 0.011 -0.012
In the Full Model the prior achievement scores account for a considerable proportion of the variance at the pupil and school levels. As the number of years of available prior data increases, the proportion of variance accounted for increases. By the final year more than 80% of the school and 70% of the pupil level variances have been “explained”. Still, there were significant differences between the schools. The correlations between the residual pupils’ scores and the schools’ scores are now close to zero. Patterns of achievement derived from the models As with vocabulary, the models were used to quantify what would happen to pupils who had experienced various levels of value-added in different years. The chart below shows the patterns and they are very similar to those for vocabulary.
38
Figure 3 Mathematics: Attainment after value-added boosts 0.80 0.70
Attainment
0.60 0.50
Reception Y1 Y3 Y4 Y5
0.40 0.30 0.20 0.10 0.00 ER
Y1
Y3
Y4
Y5
Year
The lowest line shows a value-added boost in the first year at school which is followed by the general fading away in Years one to five. As with vocabulary, the chart indicates that the major drop is in the first year but thereafter an advantage for the average pupil of about a fifth of a standard deviation units remains for the rest of primary schooling. This is twice the size of the advantage for vocabulary. If a boost is experienced in the second year (Y1) and if this directly follows the boost from the reception year then the average pupil is lifted to about half a standard deviation above the average. This advantage gradually fades but, as with the Reception boost, the fade is mainly in the immediate following year. It remains constant at about a third of the standard deviation thereafter. Similar patterns are seen if boosts are added in the next years. If a pupil received a boost each year until year 5 the advantage would be about three quarters of a standard deviation unit. This would be expected to fall the next year but only to about 2/3 of a standard deviation unit. The same conclusions are reached as were found with vocabulary but the effects were generally greater for maths in the earlier years and about the same in later years.
39
Reading For the sake of brevity the models are not set out for reading but the chart below (Figure 4) summarises the impact of the analyses. The chart parallels the maths chart (Figure 3) shown above. Figure 4 Reading: Attainment after value added boosts 0.80 0.70 0.60
Attainment
0.50
Reception Y1 Y3 Y4 Y5
0.40 0.30 0.20 0.10 0.00 ER
Y1
Y3
Y4
Y5
Year
The results for reading were generally very similar to those found for maths with the following small differences: a) In the null model the correlations between pupils’ scores year on year were higher as were the school level correlations. b) More of the variance was explained in the model at both the pupil and school levels.
Summary and conclusions This study examined the relative performance and progress of children in some schools compared with others. In that sense it is typical of many school effectiveness studies but it differs in that the results for pupils each year were available throughout primary school. This gave a more detailed view than has previously been available. It suggests that the progress made in every year in primary school is relevant to attainment at the end of primary schooling and that progress early on is of greatest value for its long term consequences. The impact on maths and reading were greater than for vocabulary. There is an interesting parallel here to the analysis of Ramey and Ramey (1998) concerning interventions in the early years. They suggest that an early intervention can have long term benefit but that it is not sufficient. Repeated on-going interventions are need to lift pupils from deprived backgrounds out of their impoverished circumstances. This analysis provides confirmation of Ramey and Ramey’s view suggesting that long term educational advantages to individuals are most likely to come from a series of positive experiences over a sustained period. Sanders and Rivers (1996) found that the teacher was a key to later progress and they suggested that three years of ineffective teachers has serious consequences. This study cannot
40
confirm such dramatic claims but it does in broad terms support the same sentiments. If a pupil experiences the first three years of schooling at a school which had low value-added scores each year such that the school was in the bottom 16% of a value-added league table the pupils would have a long term disadvantage of nearly a half a standard deviation in maths or reading. For vocabulary the disadvantage would be about a third of a SD. The results of this research have important messages for policy makers since they underline the value of good education or rapid progress in the first few years of schooling. In England the league tables have prompted schools to concentrate on generating the greatest gains in Y6 – the year for which test results are published. This would appear to be an unfortunate consequence of league tables. This paper reinforces that view that not only should attention be given to every year of education but more value should be placed on the most sensitive times – the first few years. The analyses also confirm the positive advantage of progress in early reading and mathematics for children as young as 4 years old. The results also suggest a refinement of the term school effectiveness which has largely been taken to mean the relative progress of similar pupils in different schools. Not only does this analysis underline the importance of looking at progress year on year but it also highlights the importance of taking the long view. An effective school must surely be the school which has a long term positive impact on its pupils and this may not be the same as relative attainment at the end of the last year. The analyses presented in this paper suggest that large gains in a single year of schooling are likely to have just a little impact in the long run. It is the cumulative effect of the whole school experience which matters.
References Campbell, J., Kyriakides, L., Muijs, D. and Robinson, W. (2004). Assessing Teacher Effectiveness: Developing a differentiated model. London, RoutledgeFalmer. Dickens, W. (2007) IQ-Environment Reciprocal Effects and the meaning of General Intelligenc, Paper presented at the Annual Meeting of the American Educational Research Association Chicago April 2007 Ramey, C.T. and Ramey, S.L. (1998). Early Intervention and Early Experience. American Psychologist 53(2): 109-120. Sanders, W.L. and Rivers, J.C (1996) Cumulative and Residual Effects of Teachers on Future Student Academic Achievement, University of Tennessee Value-Added Research and Assessment Center. Tymms, P., Merrell, C. and Henderson, B. (1997). The First Year at School: A quantitative investigation of the attainment and progress of pupils. Educational Research and Evaluation 3(2): 101-118. Tymms, P. (1999). Baseline Assessment and Monitoring in Primary Schools: Achievements, Attitudes and Value-added Indicators. London, David Fulton Publishers. Tymms, P.B., Merrell, C. and Henderson, B. (2000). Baseline assessment and progress during the first three years at school. Educational Research and Evaluation 6(2): 105-129.
41
