Learning to Learn in Schools Phase 3 Evaluation Technical ...

Learning to Learn in Schools Phase 3 Evaluation Technical Appendices to the Final Report May 2007 Steve Higgins, Kate Wall, Viv Baumfield, Elaine Hall, David Leat, David Moseley and Pam Woolner School of Education Durham University

Centre for Learning and Teaching Newcastle University

Institute of Education, London

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

CONTENTS CONTENTS ...................................................................................................................................................................... 2 Table of Figures .............................................................................................................................................................. 3 Table of Tables................................................................................................................................................................ 3

1 RESEARCH APPROACH ......................................................................................................... 5 1.1

Collaborative and informed professional inquiry ............................................................................................. 5

1.2

Professional learning in a project network........................................................................................................ 6

1.3

Summary............................................................................................................................................................... 7

2 IMPACT ON LEARNERS AND LEARNING.............................................................................. 8 2.1

Evidence from the case studies.......................................................................................................................... 8

2.2

Pupils thinking about their learning................................................................................................................. 11

2.3

Background to the use of Pupil Views Templates ......................................................................................... 19

2.4

Impact on national test results: school level analysis ................................................................................... 26

2.5

Impact on L2L pupils attainment reported in the case studies ..................................................................... 35

2.6

Method for school-level attainment analysis................................................................................................... 36

2.7

Summary Table of Reported Attainment Data in the Case Studies............................................................... 42

2.8

Impact on learners’ attitudes to learning......................................................................................................... 49

3 IMPACT ON TEACHERS AND TEACHING.............................................................................53 3.1

Evidence from the case studies........................................................................................................................ 53 The third year of teacher interviews and case study reports ..................................................................................... 56

4 IMPACT ON SCHOOLS ...........................................................................................................63 4.1

What do we know?............................................................................................................................................. 63

4.2

Evaluation and conclusions.............................................................................................................................. 70

4.3

Summary............................................................................................................................................................. 72

5 WIDER IMPACT: INVOLVING PARENTS AND THE WIDER COMMUNITY...........................73

2

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

6 SO WHAT IS LEARNING TO LEARN? ...................................................................................78 6.1

Summary............................................................................................................................................................. 91

7 CONCLUSIONS .......................................................................................................................92 8 BIBLIOGRAPHY ......................................................................................................................93 8.1

References in the main report........................................................................................................................... 93

8.2

References in the Case Study reports ............................................................................................................. 98

Table of Figures Figure 1: Figure 2: Figure 3: Figure 4: Figure 5: Figure 6: Figure 7: Figure 8: Figure 9: Figure 10: Figure 11: Figure 12: Figure 13: Figure 14:

Example of a completed pupil views template ............................................................11 Graph showing affective comments across the year groups........................................13 Percentage of comments categorised as Moseley et al.’s (2005) cognitive skills ........14 Percentages of comments categorised by metacognitive knowledge and skilfulness .15 Graph showing the overlap between cognitive skills and metacognition .....................16 Pupil views template with evidence of metacognitive knowledge and skilfulness ........18 Comparison of comments relating to cognitive skills and metacognition .....................19 Average decline across the project..............................................................................49 Attitude changes in schools over three years ..............................................................51 Claxton’s (2004) four generations of teaching learning..............................................79 Campaign for Learning ‘Five R’ structure...................................................................80 Focus of case studies in the first year across the 5Rs...............................................81 Focus of case studies in the second year across the 5Rs .........................................81 Focus of case studies in the final year across the 5Rs ..............................................81

Table of Tables Table 1: Table 2: Table 3: Table 4: Table 5: Table 6: Table 7: Table 8: Table 9: Table 10: Table 11: Table 12:

Number of templates and number of text units for analysis across year groups ...........12 L2L schools: Predicted and actual results .....................................................................27 L2L Schools ..................................................................................................................28 A comparative sample of 10 other schools....................................................................28 L2L secondary schools: Predicted and actual results (2004) ........................................28 2004 results for L2L and matched secondary schools ..................................................28 L2L secondary schools: Predicted and actual results (2005) ........................................29 2005 results for L2L and matched secondary schools* .................................................29 L2L secondary schools: Predicted and actual results (2006) ........................................30 2006 results for L2L and matched secondary schools* ...............................................30 L2L primary schools: Predicted and actual results (2004)...........................................31 2004 results for L2L and matched primary schools *...................................................31

3

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

Table 13: Table 14: Table 15: Table 16: Table 17: Table 18:

L2L primary schools: Predicted and actual results (2005)...........................................32 2005 results for L2L and matched primary schools *...................................................32 L2L primary schools: Predicted and actual results (2006)...........................................33 2006 results for L2L and matched primary schools *...................................................33 Comparison of pupils attitude scores in schools..........................................................50 Results from the Year Three case studies relating to parents .....................................76

4

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

1 Research Approach

1.1

Collaborative and informed professional inquiry

This project was based on the assumption that teachers’ learning and development is central to educational change and therefore adopted an action research methodology for the school-based professional enquiries. Following this approach, teachers identified their own research focus as well as their own intervention methods. The locus of control for the focus of pedagogical change was therefore firmly in the teachers’ domain rather than that of the research team. Using Stenhouse’s (1975; 1981) overarching model of ‘systematic enquiry made public’, the teachers were encouraged to initiate changes they felt were are appropriate and then the research team supported their enquiry, giving advice on existing research evidence which was likely to be helpful as well as possible research techniques for the collection of data, analysis of result, and the writing up the projects. The model used in the project (Higgins et al., 2006) always intended three cycles of action research Kemmis and McTaggart (1988), one in each school year of the project in order to provide a cumulative and progressive investigation of the impact of Learning to Learn approaches in the schools involved in the project.

The writing of the case studies was intended both as a way to make the smaller action research projects public and as a way to support reflection on the process they had been involved in. The different sections of a write-up template provided by the evaluation team aimed to link research and practice explicitly. The overall intention was therefore that a set of case studies from a particular school would not only report on the research and learning from each specific year’s project but would also relate progress of the development of Learning to Learn over three years. The findings from each of the years of the project have been presented to the teachers annually in order to provide feedback about the progress of the research and to validate the emerging findings.

5

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

1.2 Professional learning in a project network The role of the project network in supporting the teacher-researchers should not be underestimated given the complexity and variation across the project as a whole. We believe that it is an important feature underpinning the success of the Learning to Learn Phase 3 project (Higgins et al., 2006). We have identified a number of different features in the network which we believe help to make it effective. First, the quality of the research partnership between the University team and the teachers is crucial. For such a partnership to work it is important that those involved understand and respect each others’ perspectives. The project is dependent on the relationship that is established and therefore on the knowledge that is created through this collaborative learning. Integral to this connection and therefore another major feature is the role of the local project coordinator supporting both the schools directly and facilitating development of Learning to Learn as well as supporting the action as well as the support of the Campaign for Learning in providing logistical and administrative support for the conferences and in-service days. Second, the close associations between learning and research which have been a key feature to this project have promoted professional development in schools. We have found in this project, as in our other work (Leat and Higgins, 2002; Baumfield et al., 2002) that particular professional tools, such as pedagogic strategies, or practical methods for data collection such as pupil reflection templates (Wall et al., 2005) can be supportive in the development of practice by altering the focus of attention in classrooms. The resulting talk and engagement in learning at both pupil and teacher levels creates an opportunity to alter the course of teaching and learning interactions. The change in focus provides feedback to teachers about aspects of the learning that is taking place and enables them to make choices about how to respond (Hall et al., 2006). Third, the concept of pedagogical risk within the project has been an important theme. Change requires a degree of confidence on the part of the teacher, which in turn is influenced by the school culture and the management and leadership of the school. The nature of this project has been to encourage and provide legitimacy for teachers to innovate and undertake what might otherwise be seen as more speculative activities within the teaching and learning processes of their classroom. Working with on a national project with a leading campaigning body, the Campaign for Learning, and alongside a University, with acknowledged LA support (through the local co-ordinator) has

6

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

appeared to give the teachers the opportunity to innovate and undertake research in their classrooms. The fourth aspect of this project has been the collaborative nature of the enquiry into learning and teaching (Cordingley et al., 2003), as this helps to encourage participation and the development of a professional discourse about learning. Professional networks are already emerging as a significant aspect of professional development in schools – the Learning to Learn project appears to endorse this approach. Some factors, however, may be hard to replicate as teachers have commented that they value being involved in an innovative, national network with strong local support. The involvement of both the Campaign for Learning and the University research team has also been acknowledged as significant by the teachers.

1.3

Summary

The range and complexity of the approaches developed under the broad ‘Learning to Learn’ heading have made evaluation of the project challenging. In addition the support structures in the project are clearly important feature. Involvement in a large-scale project, co-ordinated by a national organisation, the Campaign for Learning, with support from University researchers and identified Local Authority staff all helped to create a climate in which investigating and researching learning to learn in classrooms was encouraged and productive of professional development. Furthermore, when collaborative enquiry into learning is undertaken in schools with supportive leadership it tends to be conducive of professional learning and effective development. These features may help to explain aspects of the findings of this report and should be taken into account in interpreting the impact of the specific Learning to Learn approaches which the teachers and schools have adopted.

7

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

2 Impact on learners and learning

2.1

Evidence from the case studies

The involvement of pupils and inclusion of their perspective has increased as the project has progressed. The teachers have indicated as part of the interviews that the role and characteristics of learning to learn pupils are important, and seemingly as a direct result of this, every case study in the second and their years of the project included consultation of pupils about their learning in some way. In this section, the analysis of the pupils’ perspective has been separated into two separate subsections looking at those who investigated the pupils’ perspective using traditional interviews or questionnaires, and those who used the pupil views templates (Wall and Higgins, 2006). The latter has been presented separately due to interesting findings from earlier in the project which prompted more in depth research across schools.

Comments directly from pupils were included in seventeen of the third year case studies; these were collected using interviews, questionnaires and observations. As in previous years (Higgins et al., 2005; Higgins et al., 2006) we took the comments reported in the case studies and analysed them for common themes. The analysis examined the quotations from the pupils and used an iterative process of theory and construct generation (Glaser, 1992).

As in previous years the comments from the pupils were overwhelmingly positive. In that pupil motivation is one of the important characteristics that the teachers perceive L2L pupils should have, this is an important finding. These affective comments tended to cluster around attitudes to the different, specific Learning to Learn (L2L) approaches which were being adopted: PEPI is good to learn with. He helps you with the skills. D’s dad showed me how to do kick off’s at the park. Now I can do them. I was at home writing with my mum. My 17 year old brother had a science book. This is what PEPI does. He does research like this book. I told my mum and brother all about it. I help my brother wash his car. I organise my mum. I fold my clothes. She says don’t bother they are going in the wash but I say I need to so I know where everything is all the time. I have a big folder of all my certificates plus my PEPI ones. I always hide it behind my back and she says…well done. Sometimes she buys me a little toy when I bring home a certificate. (Hazelbury Infant School, Enfield) 8

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

I thought Wake and Shake was great because it got me awake and ready for learning. Also it really made me want to come to school. (Alverton Primary School, Cornwall) I was good because, in other subjects, Mum didn’t really know what pressure I was under but because she came to school to see what we had to do in DT, it made her more understanding and helpful. (Camborne High School, Cornwall) They also commonly commented on the benefits they perceived for their own learning and achievement: First I was scared then I got a bit confident because you were giving us new stuff to try and being nice. You were kind you helped, you helped me learn. You made me feel clever because you said I could do hard things (Oakthorpe Primary School, Enfield) Project pupils were able to identify what constituted a positive outcome of learning and were able to articulate how this made them feel and why. This could be due to the increased focus on different elements of assessment which has become an increasingly prominent aspect of the project: Asking questions helped me with further research – Asking one question led to me asking more detailed questions. (Woodford Lodge High School, Cheshire) So that higher ability can help lower ability. Lower ability would be able to help higher ability as long as they understood the mark grid. (Fallibroome High School, Cheshire) The pupils also continued to show a clear understanding of how they learned and were positive about this. The comments extend the findings from the teacher interviews in the second year of the project (Higgins et al. 2005) where teachers indicated that L2L pupils should be aware of the process of learning. For example, Persevere or you won’t know what you are doing. (Wilbury Primary School, Enfield) It helps to know what the overall plan is so you can stick to task and know where the work is leading you. (The Roseland School, Cornwall) When I was asked to improve my work at the start of the year I thought it was because it wasn’t very good. But now I know that we are asked to do it because it will help us get a

9

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

better level or grade. So I don’t feel that my work is not good, but I know we are doing it to get a higher level. (Fallibroome High School, Cheshire) Pupils appeared to be able to see beyond a particular teaching strategy and its related effects and apply the learning across the curriculum and also to their learning outside of the confines of school. There was also evidence that pupils were able to see the impact that this kind of approach had on the school’s philosophy: Everyone is given a chance. (Over Hall Primary School, Cheshire) There were a limited number of negative comments and these tended to be related to the process of change which the project teachers were undertaking or to particular difficulties pupils were having with a teaching and learning strategy. For example, I don’t like it because they mark things wrong when they are right. It really irritates me and I don’t know how well I have done, therefore getting confused. (Fallibroome High School, Cheshire) Negative comments were also evident in the case studies where they were related to teachers investigating pupils’ attitudes to different learning situations. For example in the CPR Learning Space (Cornwall) pupils were asked about their feelings in the morning before the intervention and then were re-asked the same question at the end of the day. The enquiry explored how these different pupils’ perspectives could be managed in the classroom. This kind of research was more common in the third year of the project and might indicate increased confidence from the teachers to ask the pupils for their views and for this perspective to be taken on board and used in a formatively.

Overall the analysis of pupils’ responses to the impact of Learning to Learn is clearly positive with evidence that learners believe that such approaches help them to learning and support the development of positive dispositions towards learning.

10

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

2.2

Pupils thinking about their learning

The use of pupil views templates (Wall and Higgins 2006) has increased over the three years of the project. In the second year of the project pupil views templates were used in eight of the case studies: Fleecefield, Oakthorpe and Wilbury Primary School and Hazelbury Juniors from Enfield, Winsford High Street (Cheshire) and Treloweth and Pennoweth Primary and St Meriadoc’s Infant School in Cornwall. Three of these schools had used this method in the first year.

Figure 1: Example of a completed pupil views template (Winsford High Street Primary School, Cheshire)

Early use of the templates from pupils across the primary age phase had indicated that pupils involved in Learning to Learn were very eager to talk about their learning. This included how they learned (what went on inside their head), as well as the different strategies which supported their learning and the different contexts they found beneficial for their learning. However, the templates also revealed some of the issues which need to be considered when transferring Learning to Learn strategies to other classes or situations. It was also evident that there needs to be a vocabulary for learning (Hargreaves, 2005) which should be explicitly developed as part of the strategy to support pupils in understanding the learning processes that were occurring (Higgins et al., 2006).

11

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

Due to the potential importance of these findings from individual schools, in the final year of the project further analysis of templates across the project was undertaken. All teachers who had used templates as part of their research were involved. A total of 210 templates (a total of 674 text units) were received from seven primary schools, with at least one school from each Local Authority. These were then individually transcribed and a scanned copy taken. The spread across the year groups can be seen in the table below.

Table 1: Table showing number of templates and number of text units for analysis across year groups

No of templates

No. of text units

Reception

6

46

Year 1

36

135

Year 2

81

236

Year 4

10

47

Year 5

57

206

Year 6

20

4

210

674

Year

TOTAL

Comments were categorised using three broad frames: comments that were dispositional, comments which represented Moseley et al.’s (2005) cognitive skills, and comments which fitted with Veenman et al.’s (2005) definitions of metacognitive knowledge and skilfulness. For more information on this process please see below.

Dispositional comments were shown to have a relationship with pupil age. They appeared to tail off as the pupils got older: pupils in Key Stage 1 were more likely than those in Key Stage 2 to comment on feelings they associated with learning. For example, Little bit hard, little bit easy, little scary. (Year 2 pupil, Oakthorpe Primary School, Enfield) It should be recognised that there could be some kind of teacher effect acting on any of these results. The teacher acts as facilitator when pupils complete the templates and therefore their expectations and talk are likely to affect way in which the templates were completed. This 12

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

interpretation could be supported by the anomaly for Reception aged pupils in the graph below, which does not appear to follow the trend, showing fewer dispositional comments than Year 1 and 2 pupils. However, the number of completed templates from this youngest year group was low and therefore the proportions being shown could be seen as unrepresentative, or it may relate to the developing language skills of this age group.

Percentage of text units

70 60 50 40

Negative affective

30

Positive affective

20 10

Figure 2:

Ye ar 6

Ye ar 5

Ye ar 4

Ye ar 2

Ye ar 1

Re ce pt io n

0

Graph showing affective comments across the year groups

When comments categorised as the different cognitive skills according to Moseley et al.’s (2005) framework were explored there seemed to be a general increase in this type of comment as the pupils got older (see Figure 00). However, Year 5 pupils provide an exception to this rule. This could be due to a teacher effect, although the numbers of analysed templates within this year group should have produced representative statistics.

13

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

140

Percentage of text units

120 100 Productive thinking

80

Building understanding 60

Information gathering

40 20 0 Reception

Figure 3: skills

Year 1

Year 2

Year 4

Year 5

Year 6

Graph showing percentage of comments categorised as Moseley et al.’s (2005) cognitive

It is interesting that comments related to ‘information gathering’ (e.g. remembering, recalling, comprehending) are seen to remain relatively constant across the different year groups, from Reception through to Year 6. These comments remain very similar in content with the pupils showing comprehension of lesson and curriculum content, recalling activities they have been involved in and giving detail of the learning strategies they have used. For example, James is doing well and I think he’ll get to a high standard. Although I get different types of books as him. (Year 5 pupil, Oakthorpe Primary School, Enfield) If I put my hand up Miss Lewis will ask me. (Year 2 pupil, Winsford High Street Primary School, Cheshire) ‘Building understanding’ (e.g. organising ideas, working with patterns and rules, forming concepts) does not appear to follow any consistent trend across the age groups, although there appears to be some evidence that there is an increase in this type of comments in templates from Year 6.

Trends with ‘productive thinking’ (e.g. reasoning, problem-solving. creative thinking) comments do appear to increase as pupils get older: pupils in Key Stage 2 were more likely to include comments under this category than pupils from Key Stage 1. This could be interpreted as some kind of Piagetian progression, with older pupils with experience and competence in cognitive skills related to gathering information and building understanding being more likely to lead to productive thinking.

14

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

Having said this it is worth noting that some pupils in the younger age groups were making comments which were classified under this heading.

Within each of these cognitive skills categories it was possible to see a number of sub-themes within the comments and therefore a further analysis was undertaken. Within this analysis the number of comments that were classified as ‘cognitive skills’ but were also classified as relating to metacognition was interesting. Indeed this cross over was apparent across all the different cognitive skills sub-categories. There seems to be evidence within the templates of the pupils being thinking about their thinking or being cognitive about metacognition.

Percentage of text units

70 60 50 40

Metacognitive skilfulness

30

Metacognitive know ledge

20 10 0 Reception

Year 1

Year 2

Year 4

Year 5

Year 6

Figure 4: Graph showing percentages of comments categorised using Veenman et al.’s (2005) metacognitive knowledge and skilfulness

With regard to the different elements of metacognition that were explored, Figure 4 shows that comments categorised as metacognitive knowledge and skilfulness were apparent across the year groups. This goes in contradiction to the findings of Bratsch et al. (2003) and Kuhn (1999): pupils within the L2L project as young as four and five years old showed that they not only have metacognitive knowledge, but also metacognitive skilfulness, something that these researchers found did not emerge until secondary school. For example, When I read in my head when I come to a long word I skip it. (Year 5 pupil, Oakthorpe Primary School, Enfield) I made some chicken – I thought about how mummy did it. (Reception pupil, Leaf Lane Infant School, Cheshire)

15

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

At the current stage of the research it is not possible to say whether this metacognitive skilfulness in these younger pupils is apparent because of the L2L interventions. However, it could be argued that the interventions included under the L2L umbrella give the pupils knowledge and vocabulary with which to talk about learning and therefore support this kind of articulation about their thinking.

Figure 5 shows the overlap between the different cognitive skills and metacognitive knowledge and skilfulness. It can be seen that there is a definite progression, with the least amount of overlap in comments categorised as gathering information through to most being related to productive thinking. Moseley et al. (2005) say that no hierarchy is implied by the different cognitive skills they present, however these findings do seem to suggest that metacognition is more likely when related to productive thinking and less likely with building understanding and information gathering skills.

Percentage of text units

80 70 60 50

Know ledge

40

Skilfulness

30 20 10 0 Information gathering

Figure 5:

Building understanding

Productive thinking

Graph showing the overlap between cognitive skills and metacognition

Having said this, it is interesting to look in more detail at the differences between metacognitive knowledge and skilfulness when related to the specific cognitive skills described above. Metacognitive knowledge is seen to be most common within the building understanding skill, for example Sometimes when I read by myself if it is funny I laugh. (Year 5 pupil, Oakthorpe Primary School, Enfield) I think its good working on your own because no one is here to distract you. (Year 1 pupil, Winsford High Street Primary School, Cheshire)

16

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

Using Moseley et al.’s (2005) definition, this skill is all about development of meaning and working with patterns and rules, concept formation and organising ideas. If pupils are thinking strategically and reflectively about their learning, then this skill would be a baseline in beginning to develop that knowledge about learning and thinking. You can gather information about learning, but understanding only comes when you start cognitively being aware of the relationships and interactions between the different elements which would correspond to the skill ‘building understanding’.

Metacognitive skilfulness, however, is more likely to interact with productive reasoning and again it would appear logical that if you are learning about and building understanding about learning, you will be able to recall the use of strategies which aid learning and thinking and be able to make links between the different of metacognitive skills and knowledge, but to truly have metacognitive skilfulness than there needs to be some understanding of ‘causal relationships’ and also some ‘systematic enquiry’ (Moseley et al., 2005: 314) into what works best for an individual. Quotes from the templates which exemplify this type of comment include, Its okay if you just try and I just ignore people if they laugh cos I’ve got it wrong (Year 2 pupil, Winsford High Street Primary School, Cheshire) We was thinking of describing words and thinking of some information and some ideas and I like James being my Literacy partner. (Year 4 pupil, Fleecefield Primary School, Enfield) The final analysis undertaken explored the structure of the templates (see Figure 6 for a further example), the use of the speech and thought bubble by the pupils, and the extent to which they were used to record cognition in general (speech) and metacognition (thought bubble). This has been validated in other research projects and contexts (Wall et al., 2006; Wall et al., 2005). Within the Learning to Learn Phase 3 Evaluation there is a significant emphasis on talk about learning and which might have an impact on the way that the templates were completed.

17

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

Figure 6: Example of a pupil views template completed by a L2L pupil showing evidence of metacognitive knowledge and skilfulness (Oakthorpe Primary School, Enfield)

There is no doubt that the templates did explore the metacognitive dimension. However, it is possible to see in the example in Figure 6 that there was not always as much difference between what was written in the speech and thought bubble as might have been expected. This is different to the result obtained from previous research with the templates (Wall et al., 2005). We speculatively suggest that this might be the result of making talk about learning more explicit through learning to learn, as the relationship between thought and speech is a complex one.

18

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

70

Percentage of text units

60 50 40

Cognitive skills Metacognition

30 20 10 0 Speech bubble

Thought bubble

Figure 7: Comparison of speech/ thought bubble comments relating to cognitive skills and metacognition

Pupils generally are thinking about their learning and thinking, but in an L2L classroom they are also positively encouraged by an explicit focus on talking about their learning which might then impact on their thinking and so on. When compared to the similar data from pupils learning with interactive whiteboards (Wall et al., 2005), conducted in non-L2L schools this relationship was not apparent. The nature of the project is such that it is not possible to draw firm conclusions about the impact of Learning to Learn approaches as a cause of these differences. However we can say that in Learning to Learn schools pupils are able to talk about their learning and that there is a progression evident in the way that they describe this, with a decrease in affective responses as pupils get older and an increase in the complexity of the way that they think about their learning. We believe that this is a result of the development of a vocabulary and language for learning being developed in the classrooms.

2.3

Background to the use of Pupil Views Templates (Exploring pupil views of metacognition)

There is no doubt that an important part of learning is metacognitive thinking. The term metacognition was introduced in the 1970s by Flavell (1979) to encompass learners’ knowledge of their own cognition. Veenman and Elshout (1999, p.510) state, “Metacognitive skills… concern the

19

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

procedural knowledge that is required for the actual regulation of control over one’s learning activities”. Metacognition has been subsequently given high status as a feature (Georghiades 2004), with characteristics of transferable learning skills, awareness of the process of learning and sustained benefit of metacognitive knowledge. This means that it can be argued to be a powerful and important aspect of teaching and learning, and therefore, worthy of research. In Moseley et al.’s book (2005) a model for mapping frameworks for thinking is outlined (see figure 1). The three cognitive components, arising from Bloom’s taxonomy (1956), plus a selfregulatory/metacognitive system is useful in beginning to think of knowledge about thinking and learning. Although the authors do not imply a hierarchy of thinking within this model, they do distinguish between what can be an automated cognitive process within the cognitive skills section, and reflective and strategic thinking which is seen as conscious and ‘harder work’. It is the author’s belief that this latter facet is where metacognition is apparent.

STRATEGIC AND REFLECTIVE THINKING Engagement with and management of thinking/learning, supported by value-grounded thinking (including critically reflective thinking)

COGNITIVE SKILLS Information-gathering Experiencing, recognising and recalling Comprehending messages and recorded information

Building understanding Development of meaning (e.g. by elaborating, representing or sharing ideas) Working with patterns and rules Concept formation Organising ideas

Productive thinking Reasoning Understanding causal relationships Systematic enquiry Problem-solving Creative thinking

Moseley et al.’s (2005) Model of frameworks for thinking (p.314) If the reflective and strategic thinking category within Moseley et al.’s model is where metacognition lies, then within this category it is useful to use the duality of Veenman et al.’s (1997) concepts of metacognitive knowledge and metacognitive skilfulness. This splitting of metacognition is based on Flavell’s (1979) definition of metacognitive knowledge: “…declarative knowledge one has about the

20

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

interplay between personal characteristics, task characteristics and the available strategies in a learning situation (Veenman et al. 2005); and metacognitive skilfulness: “reflecting on the nature of the problem, predicting consequences of an action or event, planning and monitoring the ongoing activity, comprehension monitoring, checking the results of one’s actions, testing for plausibility and reflecting on one’s learning performances” (Veenman et al. 1997). The author would argue however, that there is an element about L2L that is also about the critical and rational thought about this knowledge and skilfulness. In other words, to what extent is an individual being cognitive about their metacognition.

Metacognition, because of its inwardness, is difficult to observe. Within Moseley et al.’s model it could be argued that evidence of cognitive skills might be more observable and recognisable across individuals, whereas strategic and reflective thinking are more hidden and private. Even with adults it is difficult to identify and reflect on what metacognition is to them and with pupils this is increased. Indeed, it has been argued that metacognitive skills do not develop until the age of ten to twelve years old (for example, Kuhn 1999), although metacognitive knowledge can be present at a much younger age. In contrast, Bartsch et al. (2003) found that young pupils have knowledge of what they have learnt, but not of how and when they learnt it. However, in the author’s experience (Wall et al. in press; Wall and Higgins 2006; Wall et al. 2005) and as others have documented (Leat and Higgins 2002), including L2L project teachers (Higgins et al. 2005; 2006), this is not always the case. It has become necessary to develop a tool that might be able to capture evidence of its occurrence particularly within the Learning to Learn Phase 3 Evaluation.

21

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

An example of a completed L2L pupil views template Data Collection Method Most data collection methods within the field of pupil views tend towards the use of interviews, either one-to-one or focus groups (for example, McCallum et al. 2000; Bullock and Muschamp 2006), although some researchers have used questionnaires, with varying degrees of success (for example, Black et al. 2006). It was felt within the L2L Phase 3 Evaluation where teachers were doing the data collection within the action research methodology, it was important to try and transcend any division between teaching and learning in the classroom and empirical research. Therefore a research tool needed to be developed that offered something new to both research and teaching and learning perspectives. A tool which will inform about pupils’ development and understanding of metacognition in different learning contexts emerges as a principal consideration. With this in mind pupil views templates were designed (for example see figure 2). The method has its origins in educational action research, with the templates aiming to be a ‘pragmatic tool’ (Dewey 1931; Leont'ev 1981) which has meaning and value across both learning and research contexts. In other words, it aims to be a research tool that can be empirically 22

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

influential and powerful, while also having an impact upon the pedagogical processes within classrooms. The theory behind the tool and its use is fully described in Wall and Higgins (2006), however a brief summary is included below.

Model of interaction using the template The template provides an image of the learning situation on which the research is focused, the process becomes a three-way interaction between the researcher (or teacher), the pupils and the template (see figure 3). The template design has its inspiration in work completed by the Bubble Dialogue team1; for example, McMahon and O’Neill (1992) and Jones and Price (2001) and also the research of Hanke (2001). The key idea in all these projects is that pupils can be asked, using a cartoon representation, to reflect on their thinking on different aspects of their experience. The templates aim to gather information on pupils’ attitudes and beliefs about teaching, curriculum content and school/classroom structures (the process of teaching), but also to go further into the realms of metacognition (thinking about the process of learning). This is done through a superimposed structure of speech and thought bubbles. The speech bubble looks at factors external to the individual: the learning of other pupils, teachers and parents and practicalities of learning in the specified context (cognition in general). In contrast, the thought bubble is intended to look at the ‘internal’ processes: the learning of the individual - ‘what is going on inside their head’ (metacognition). An overlap between the two fields is expected with regard to advantages and

1

http://www.dialogbox.org.uk/BubbleDialogue.htm

23

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

disadvantages and subject differences: the impacts on the learning of themselves and others. A diagram of this rationale is included below in figure 4. The speech bubble and the thought bubble on the template means that there is an automatic prompt for the pupil to talk about what they are thinking. This could very simply be what they think about a specific activity, for example independent reading, or it could be more sophisticated with regard to the more abstract thinking processes which they associate with or utilise during a specific activity. The latter abstraction into metacognitive process can be seen to link with Veenman and Spaans’ (2005) concepts of metacognitive awareness and metacognitive skilfulness: ideas which will be used in the cross case study analysis of pupil views.

Venn diagram of thought and speech bubble rationale on pupil views template

Cross case study analysis The data within this paper comes from seven schools, all based in the primary age phase (catering for pupils aged four to eleven), in the L2L Phase 3 Evaluation. All three LAs were represented in the sample. In total 210 pupil views templates were analysed. This meant 674 units of text: each text unit was isolated on the basis of sense. The templates included in the sample were completed by pupils ranging from Reception (four years) through to Year 6 (11 year old). Analysis of the data 24

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

from the templates was conducted by the author as an iterative process of theory and construct generation (Glaser, 1992) and these theories and constructs were tested and analysed using NUD*IST qualitative software (Richards and Richards 1993). Firstly, the text units on the templates were coded according to school type, age of pupil, and in that some pupils had completed more than one template, they were marked with the sequence in which the template was completed. The text units were also tagged at this stage with whether they were written in the speech bubble or thought bubble. The second stage of the analysis saw the statements categorised using Moseley et al.’s (2005) model (figure 1). The statements were categorised as to whether they were predominantly evidence of cognitive skills: information gathering, building understanding, or productive thinking; and/or whether they were evidence of strategic and reflective thinking. The following definitions based on the model were used: • Information gathering: Comments in this category tended to be characterised by recall of ideas and processes, comprehension of information they have been told or have read; • Building understanding: This needed the concepts of information gathering, but also required some organisation to be given to these ideas and recollections, some idea of relationships were looked for, plus some development of meaning about implications and patterns that could be applied. • Productive thinking: These comments tended to show reasoning, problem solving and some movement of understanding beyond the concrete and towards the abstract. Ideas that were generalisable and creative were placed in this category • Strategic and reflective thinking: This category looked at whether the comments represented an awareness of the process of learning. It needed a reflective or strategic element to the statement; that this comment represented thinking about learning. An additional category was introduced at this stage for dispositions and attitudes to learning which appeared in the pupils’ comments. Thirdly, the statements which were labelled as strategic and reflective, and therefore indicative of metacognition, were then reanalysed for evidence of metacognitive knowledge and metacognitive skilfulness (Veenman et al. 2005). These categories were characterised in the following ways:

25

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

•

Metacognitive knowledge: Comments in this category demonstrated an understanding that the child could think about learning, and that the individual understood some of the processes which supported their own learning.

•

Metacognitive skilfulness: Comments within this category represented a movement beyond knowledge towards the application and translation of thinking and learning skills across different contexts or for different purposes.

The coding system was checked for inter-rater reliability. A colleague not associated with the project or the templates was asked to code comments from 20 templates, this included 75 text units. The inter-rater agreement was 82% which was felt to be good. It should be noted in all the graphs that the categories used were not necessarily mutually exclusive and a single text unit could be classified as fitting under more than one category and therefore percentages may not add up to 100%.

2.4

Impact on national test results: school level analysis

Within the case studies in each of the L2L schools various research methods were used to identify and, where appropriate, measure any effects. However, an over-arching analysis across the schools of school level data seemed a worthwhile addition and one that could be expected to add to the explanatory value of any results reported by individual schools below (for full method see below and Higgins et al., 2006). Furthermore, the current importance at the policy level of ideas about school effectiveness and judgements based on school performance indicators suggests that for an intervention to be seriously considered it would be desirable to have a demonstrable impact on school attainment. However it should also be borne in mind that the implementation of L2L varied widely across schools and only sometimes included whole year groups or years where national tests were taken by the pupils.

Phase 1 and 2 schools had been involved in the work of the Campaign for Learning, and therefore L2L, since 2001. We do not have any detailed record of the extent to which the L2L innovations have been implemented in each school, so it is not possible to make comparisons between

26

Learning to Learn Phase 3 Evaluation: Technical Appendices to the Final Report

schools. However they all made a commitment to this philosophy for two years and therefore the results from these schools as a group are of interest.

The following table (Table 2) shows how predicted and actual percentages of pupils achieving five or more GCSEs at grades A* to C compare for the secondary schools involved in phases 1 and 2 of L2L.

Table 2: L2L schools: Predicted and actual results

School

Actual

1 2 3 4 5 6 7 8 9 10

36 68 71 40 52 53 65 38 75 66

2002 Predicted 35.159 50.367 64.608 40.522 49.607 48.631 61.129 35.224 68.122 73.263

Test stat 0.193 5.788 2.686 0.063 0.653 1.356 1.054 0.642 1.990 2.436

Sig?

Actual

ns ↑ ↑ ns ns ns ns ns ↑

41 68 73 33 60 53 65 51 65 67

p