ECP

REFERENCE No. RES-000-23-1564

OUTPUT

Effective Classroom Practice (ECP) Project Report

Using Systematic Classroom Observation Schedules to Investigate Effective Teaching: Overview of Quantitative Findings

Pam Sammons & James Ko

School of Education University of Nottingham August 2008

Contact Prof Pam Sammons [email protected]


OUTPUT

CONTENTS EXECUTIVE SUMMARY ................................................................................... III I:

INTRODUCTION ........................................................................................ 1 1.0 Teacher Effectiveness Research ........................................................... 2 2.0 Inspection Evidence ........................................................................... 5 3.0 Systematic Observations..................................................................... 7 4.0 Objectives of the quantitative analyses ................................................. 8 II: CHARACTERISTICS OF THE TEACHER SAMPLE............................................... 9 1.0 Gender........................................................................................... 10 2.0 Age................................................................................................ 10 3.0 Educational levels ............................................................................ 11 4.0 Teaching experience......................................................................... 11 5.0 Number of school(s) taught and years in current school ........................ 12 6.0 Post(s) of responsibility .................................................................... 13 7.0 Current motivation, commitment and job satisfaction............................ 16 8.0 Hours the teachers worked per week and stress level............................ 17 9.0 School Context (Free School Meal Band) ............................................. 18 III: INTERNATIONAL SCHEDULE FOR TEACHER OBSERVATION AND FEEDBACK (ISTOF): SUMMARY OF RESULTS ..................................................................... 19 1.0 Missing data.................................................................................... 19 2.0 Bimodal distributions ........................................................................ 23 3.0 Correlations Among ISTOF items........................................................ 37 IV: QUALITY OF TEACHING LESSON OBSERVATION INDICATORS (QOT) .............. 41 1.0 Background..................................................................................... 41 2.0 Missing data.................................................................................... 43 3.0 Negatively skewed distributions and high positive kurtosis ..................... 43 4.0 Bimodal and normal distributions ....................................................... 46 5.0 Correlations among QoT Indicators..................................................... 52 V: EXPLORING THE UNDERLYING STRUCTURE OF THE TWO OBSERVATION SCHEDULES USING CONFIRMATORY FACTOR ANALYSIS ..................................... 57 1.0 A CFA model based on ISTOF items .................................................... 57 2.0 A CFA model based on ISTOF indicators .............................................. 60 3.0 A CFA model based on QoT indicators ................................................. 62 4.0 Variations in observed teacher behaviours in terms of the underlying factors identified by the CFA models of the two instruments................... 65 5.0 Comparisons of teachers by their scores in two instruments................... 74 6.0 Associations Between Observed Teaching Factors and Pupils’ and Teacher Perceptions ......................................................................... 78 VI: DISCUSSION AND CONCLUSIONS ............................................................. 82 REFERENCES ................................................................................................ 86 APPENDICES

........................................................................................... 89

ii


OUTPUT

EXECUTIVE SUMMARY 1.0 Background The ESRC funded Kington, 2006)

‘Effective Classroom Practice Research, (Day, Sammons &

is a mixed methods investigation conducted in English primary

and secondary schools during 2006-2008 that aims to :

•

Describe, analyse and explain the variation in primary and secondary school teachers’ classroom practice using two different observational instruments and pupil and teacher perceptions, focussing on English and mathematics teaching

•

Explore typical and more effective classroom practice of teachers in Years 2, 6 rand 9, across different school contexts, career phases and ages in relation to observations of practice and professional, situated and/or personal factors which are perceived to affect observed practice over time.

The ECP study collected a range of qualitative and quantitative data to address these aims. Quantitative data include systematic observations of practice using two different instruments conducted in two successive terms in one academic year (2006/2007), for a sample of 81 teachers in Year 2, 6 or 9, a survey of pupils’ views and experiences for a class or teaching group taught by the teachers and a short teacher survey. In addition, field notes made during the observations and pre and post lesson observation interviews to seek teachers’ perspectives provide rich qualitative data for the sample. This paper contributes to both the main aims from a quantitative perspective. It focuses on the description and analysis of variation in the observed classroom behaviour and practices of a purposive sample of English primary and secondary teachers using two different systematic observation instruments and data from the teacher and pupil surveys. This element of the ECP research complements the qualitative evidence obtained from teacher interviews and observers’ field notes. Results from the quantitative analyses were used to develop various groupings of teachers that fed into the qualitative data analysis by incorporation into individual teacher profiles drawing on methodological approaches developed in an earlier mixed methods study conducted by members of the project team (Sammons et al, 2007; Day et al 2007).

iii


OUTPUT

2.0 Structure of the Paper The paper consists of several sections. First a brief summary of teacher effectiveness research and inspection evidence on the characteristics of effective practice is given to contextualise the findings and rationale for the choice of systematic observation instruments used in the ECP research. Second the study describes the characteristics of the teacher sample and summarises some of the findings in relation to their self reports of commitment, motivation, self-efficacy, stress and workload. Third, the paper describes the results of descriptive analyses of the observed variation in practices found for the sample and identifies items where there was either little or alternatively greater variation among teachers (particularly

bimodal

distributions)

in

the sample. The relationships with

important features such as school sector (primary/secondary), school context (measured by the indicator of social disadvantage of pupil intake, % pupils eligible for Free School Meals) and teacher characteristics (e.g. experience, responsibility) are also investigated. Fourth the paper identifies a number of models summarising the underlying dimensions (factors) of teacher behaviour measured by the two observation instruments and compares these.

Fifth the

paper examines how teachers are grouped on the basis of the various factors measured by the two instruments Sixth; the paper explores patterns of association between the teacher observation factors and separate measures of pupils’ views collected from a pupil survey conducted for a class or teaching group taught by teachers in the sample. The concluding section summarises the main findings and their implications.

3.0 Choice of Systematic Observations The use of systematic observations is intended to allow for comparisons of teachers in terms of certain pre-determined and agreed categories of behaviour and practice and originated in teacher effectiveness field of enquiry (Muijs & Reynolds 2005).

Muijs and Reynolds (2005) note the difference between value

oriented instruments that require a value judgment from the observer compared with behavioural instruments that are designed to look at the occurrence or not of specific behaviours without passing global judgments on whether the observed behaviour was ‘good’ or ‘poor’. This is related to the distinction between high and low inference measures. They argue that teacher effectiveness research has tended to focus on mathematics and reading and on primary classes thus the generalisability of the findings to other subjects or activities and to the secondary sector requires further study.

iv


OUTPUT

Teddlie et al (2006) suggest that the successful international comparative research will depend on the development of internationally valid instruments, particularly teacher effectiveness protocols. The

International System for Teacher Observation and Feedback (ISTOF) was

developed using an iterative, multiple step, internet based modified Delphi technique involving teams from 21 participating country experts (researchers, practitioners and education advisers/inspectors) whose opinions about what constitutes effective teaching were used to generate the components in the instrument. The ECP team was asked to contribute to the international pilot phase of the ISTOF project and chose to use the ISTOF schedule as one of the two instruments in the study to enable its applicability to the English context to be explored. In addition, the ECP team chose an additional instrument based on an international collaboration between two inspectorates (senior HMI and Dutch inspectorate equivalents). The Lesson Observation Form for Evaluating the Quality of Teaching (QoT) is intended to utilise professional judgment and has a deliberate high inference evaluative framework based on professional judgments. The inspectorate research project team that developed the Form investigated reliability and validity using data collected from primary school classes in England and the Netherlands (van de Grift et al 2004). The ECP team decided to adopt the QoT Form as an additional observational schedule in the ECP research and trial its use in both primary and lower secondary classes. Inspection has a high profile in evaluating schools and the quality of teaching and learning in England and therefore inspection based instrument might be expected to provide an additional source of relevant evidence to contribute to understandings of what constitutes effective classroom practice. The team also sought to explore the utility of the QoT instrument for use in lower secondary classes as it had been developed in the primary sector.

4.0 Objectives of the quantitative analyses The main aim of the ECP quantitative analyses is to explore the extent of variations in teachers’ observed classroom practice for a purposive sample of relatively effective teachers in English primary and secondary schools

v


OUTPUT

The objectives are:

•

To examine the utility of the ISTOF and QoT instruments in the English context in identifying features of effective classroom practices.

•

To examine the relationships between features of teachers’ observed practices measured by the two instruments.

•

To explore variations in observed practices between primary and lower secondary teachers and identify the key underlying dimensions of practice.

•

To

explore

variations

in

observed

practice

between

English

and

Mathematics teachers in the secondary school sample.

•

To explore variations in observed practice according to school context (in terms of level of social disadvantage measured by the percentage of pupils eligible for free school meals indicator).

•

To investigate the relationships between variations in observed practice and teachers’ self reported commitment, motivation and stress levels.

•

To investigate any relationships between pupils’ views and experiences (derived from surveys) and variations in teachers’ observed classroom practice.

•

To produce different groupings of teachers according to overall and specific features of their observed classroom practice to incorporate into teachers’ individual profiles (in terms of whether teachers were relatively above, typical or below the sample average in specific aspects.

5.0 Characteristics of the teacher sample The sample consisted of 81 teachers in two groups1, 26 (32%) were chosen based on results of an earlier study, VITAE, (Day et. al. 2007) after analysis of existing data on pupil outcomes (pupil views and value added attainment data) as being typical or more effective. In addition, another 55 (68%) participants were newly recruited for the ECP study. These newly recruited teachers were drawn from an ongoing study of Leadership and Pupil Outcomes funded by DCSF (Day et al, 2007). It was anticipated that they might be more likely to show behaviours

associated

with

academic

effectiveness,

because

they

were

purposively selected from schools identified as academically highly improved/r

1

The sample size is relatively small which reduces the potential to test validity and reliability. Nonetheless, given the mixed methods design of the current study, the use of quantitative data can be triangulated with qualitative data to explore the potential utility of the two instruments.

vi


OUTPUT

effective using national assessment and examination data for three years 20032005 (GU & Sammons, 2007). The teachers selected were also identified by head teachers and, in secondaries, by Heads of Departments, as particularly effective teachers in these improved/effective schools The numbers of teachers in primary and secondary sample were not equal, 45 (55.6%) in the former and 36 (44.4%) since it the primary sample covered Years 2 and 6 and it was desirable have reasonable numbers in both years for comparison. The teachers selected taught either Year 2 or Year 6 classes in the primary and Year 9 in the secondary sample and the class observations took place in Year 2, 6, or 9 classes. While most primary teachers taught only one year group, secondary teachers generally taught several year groups.

6.0 Methodology 6.1

Current

self

reported

motivation

and

commitment

levels The vast majority (N=64, 86.5%) of teachers in the ECP study indicated that their current motivation as a teacher was either high or very high with only one teacher indicating that his motivation was low and nobody giving the lowest response rating (very low) Similarly, teachers’ self-reported commitment was extremely high in the sample, and more teachers (N=34, 45.9%) claimed that their commitment was very high than was found for the percentage reporting very high current motivation (N=22, 29.7%). For this sample there was no indication of any significant association between self reported motivation and commitment and the number of posts or the number of years in teaching. There was, as might be expected, a significant positive association (rho=0.61) between teachers’ self reported motivation and satisfaction and between motivation and commitment (rho=0.57).

6.2

Hours the teachers worked per week and stress level

The number of hours the teachers reported they worked per week during the term gives an indication of workload. Over half indicated they worked 52 or more hours a week in term time. A little over a quarter (29%) reported high or very high stress levels compared with 20% reporting low or very low stress. Hours of reported work were moderately correlated with levels of reported stress while the association with current commitment to teaching was positive but weaker and that with current motivation non-significant. In addition, there was a moderate

vii


OUTPUT

but significant negative association (rho=-0.30) between teachers reported stress and their motivation levels and between stress and job satisfaction (rho=-0.34). There were no statistically significant associations between the level of social disadvantage of context (measured by % pupils’ eligible for FSM) and teachers’ self reported motivation, commitment job satisfaction or stress levels for this sample. However, there was a weak significant positive association between teachers feeling they could make a difference to pupil learning and the level of disadvantage of intake. Interestingly, in this sample of typical to more effective teachers those in more disadvantaged schools were more likely to feel they could make a difference to pupil learning (rho= 0.24) but were somewhat less likely to report they could get parents involved in school activities (rho= -0.24).

6.3

Variations in Teachers’ Observed Practice: Underlying

Dimensions A range of statistical approaches were used to explore variations in teachers’ observed behaviours in terms of specific items, indicators and components of the two instruments used for the quantitative strand of the ECP research. On some measures there was much less variation than others. In many aspects teachers scored highly suggesting these were more universal features of effective practice. On others bimodal distributions were found. Both exploratory and confirmatory factor analysis were used to identify the main underlying dimensions or factors in each of the two schedules. Several models were compared (e.g. item based and indicator based). The results revealed five underlying indicator model and six for the QoT indicator model .

viii

factors for the ISTOF


Factor Name Structured lessons that maximize learning and involvement (ECP Core)

High Quality Communication & Interaction (ECP Skilled) Responsiveness to Students’ Needs (ECP Advanced)

Ind. No 2.1 3.3 4.1 6.3 6.4 7.1 1.1 3.1 4.2 6.1 2.2 4.3

Active Learning

Purposive Learning

5.2 5.3 1.2 3.2

N=79

OUTPUT

Indicator Name The teacher creates an environment in which all students are involved Lessons are well structured The teacher is able to engage students The teacher interacts with all students The teacher communicates high expectations Learning is maximized The teacher gives explicit, detailed and constructive feedback The teacher shows good communication skills The teacher possesses good questioning skills All students are valued The teacher takes full account of student differences The teachers uses various teaching methods and strategies The teacher gives students opportunities to be active learners The teacher fosters critical thinking in students Assessment is aligned with goals and objectives Clear explanation of purpose Total

Square Reliability Factor multiple Cronbach's Loading correlation alpha 0.69

0.48

0.93 0.81 0.70 0.71 0.62

0.86 0.66 0.49 0.50 0.39

0.60

0.36

0.82 0.45 0.92

0.67 0.20 0.85

0.82

0.68

0.87

0.77

0.72

0.52

0.86 0.69 0.63

0.74 0.40 0.48

Table 1 illustrates and summarizes the results for the ISTOF indicator model equivalent analyses for the QoT instrument.

ix

0.68

0.84

Table 1: Indicator-based CFA Model of ISTOF scale

where five underlying factors are identified.

0.80

Table 2 illustrates the results of

0.71 0.54 0.83


FACTOR NAME Supportive and stimulating lesson climate

Square Reliability Factor Multiple Cronbach's loading Correlation Alpha 0.94 0.89 0.9 0.8 0.98 0.95 0.87 0.91 0.83

IND. INDICATOR CONTENT 1.1 1.2 1.3 2.1 2.2 2.3 2.4

Proactive lesson management Well orgainzed lessons with clear objectives

Effective classroom layout

0.82 0.66 0.86

0.67 0.43 0.74 0.83

involves all pupils in the lesson

0.91

8.1

gives a well structured lesson

0.91

0.83

8.2 8.3

ensures the orderly progression of the lesson uses leanring time efficiently clarifies the lesson objectives at the start of the lesson evaluates whether the objectives have been achieved at the end of the lesson

0.93

0.86

0.91

0.82

0.45

0.2

0.56

0.31

1

1

0.87 0.96

0.76 0.93

0.96

0.92

0.81

0.65

0.92

0.84

0.96

0.92

0.63

0.39

0.76

0.57

0.99

0.98

3.1 3.2 4.1

7.1

gives clear instructions and explanations gives clear explanations of the learning materials and the assignments gives feedback to pupils ensures the classroom layout supports the pupil activities the teaching environment is educational and contempory adapts the instruction to the relevant differences between pupils adapts the assignments and processing to the relevant difference between pupils ensures that the teaching materials are orientated towards transfer

7.2

stimulates the use of control activities

7.3

provides interactive instruction and activities

9.1 9.2

Adapation of teaching

ensures a relaxed atmosphere promotes mutual respect supports the self-confidence of pupils ensures cohesion stimulates the independence of pupils promotes cooperation between pupils there is good individual involvement by the pupils

5.1

4.2 4.3

6.1 6.2

Teaching learning strategies

OUTPUT

0.88

0.73

0.79

0.90

0.69

0.91

Total

Table 2: CFA Factors of QoT (van deer Graft) indicators In general, the intra-scale factor correlations for ISTOF were not strong as those for the QoT, only the first two factors, structured lessons that maximize learning and

involvement

and

High

Quality

Communication

&

Interaction,

were

significantly correlated with each other (rho 0.49) and with most other factors. The fifth factor, Purposive learning, was only significantly correlated with the first two factors. In contrast, the factors of the QoT scale were all significantly correlated with one another. Their correlations were also somewhat stronger, ranging from 0.3 to 0.75. The strongest association was between the factors Proactive lesson management and Well-organised lessons with clear objectives (rho 0.75) followed by the factors Proactive lesson management and Supportive lesson climate (rho 0.63). The field researchers’ overall judgment of the quality of teaching made at the end of the QoT observation was significantly correlated with all the individual QoT factors. The factor Proactive lesson management appeared to contribute most to

x


OUTPUT

the field researchers’ overall judgment with a strong positive correlation (rho 0.8). 6.3.1 Grouping teachers by their scores in two instruments In order to investigate how consistent individual teachers might have been in their behaviour in different factors, their factor scores were categorized into three broad groups with reference to the distributions of the respective factors: above average for the sample (i.e., ABOVE), average (i.e., AVG) and below average (i.e., BELOW) for the sample. Results are summarized in Appendix 4. 1. Teachers grouped by factors in the ISTOF item-based model In all 24 teachers (30% of the sample) were categorized as ABOVE for their mean aggregate factor scores in the item-based ISTOF model based on frequencies of observed behaviours. However, of these 24 cases, only three teachers were categorized as ABOVE in all the individual factors identified. The results for these cases suggest that a generic concept of teacher effectiveness may be helping in identifying unusually effective outliers. For the remaining 21 cases, the majority (N=15, 19%) were rated at least at average range in all aspects, while only six teachers2 (25%) were categorized as BELOW (average) in one aspect of their observed teaching practices. In particular, just three

were rated as BELOW in

Purposive learning, two in Quality questioning skills and feedback towards students, and one in Positive classroom management. These findings therefore lend support for a more differentiated concept of teacher effectiveness. Nonetheless, it is still rare that teachers in the ECP sample were found to be particularly weak in any one aspect. Moreover, these teachers who had a mean aggregate score in the ABOVE (average) range received at least an AVERAGE or ABOVE rating in the first three factors. This shows that the most effective teachers (based on observation results) are likely to have particular strengths in these factors. Of the 25 teachers (or 32%) who were categorized as BELOW (average) in the mean aggregate factor score, only one was rated as BELOW in all aspects of teaching practices and nine (11.4%) teachers were rated as BELOW in most aspects and without any ABOVE rating in any aspect. Moreover, eighteen teachers

2

These six teachers were L3933006BG, V8782008AG, V8782058DH, V8783004SB, V8882083BR and V9374601AT.

xi


OUTPUT

who were in the BELOW category for their mean aggregate score were still identified as in the ABOVE category for at least one teaching practice. This tends 2. Teachers grouped by factors in the QoT model Twenty three teachers out of 70, or nearly a third (32.9%) in the sample, were categorized as ABOVE for their mean aggregate factor scores in the QoT model. Of these 23 cases, three teachers were categorized as ABOVE in all aspects. Again this finding gives some support to the generic concept of teacher effectiveness. For the remaining 20 cases, while the majority (N=19, 24%) were rated at least at average range in all aspects, four teachers (5%) were categorized as BELOW (average) in one aspect of teaching practices. As in the indicator-based model, four teachers had a mean aggregate score in the ABOVE (average) range but were categorized as BELOW in one aspect of their observed teaching practices. Of the 24 teachers (or 34%) who were categorized as BELOW (average) in the mean aggregate factor score, two were rated as BELOW in all aspects of teaching practices. No teacher was rated as BELOW in most aspects and without any ABOVE rating in any aspect. By contrast, nine teachers who were in the BELOW category for their mean aggregate scores also were categorized as ABOVE for at least one aspect of the teaching behaviours. Teachers were compared to establish whether those who were rated relatively highly in one instrument were also rated relatively highly in the other. The results showed that there were nine teachers (out of 70 for which data were collected on both instruments, or 13%) in the ABOVE range and five teachers (7%) in the BELOW range in both instruments. This suggests that around a fifth of the sample were broadly consistently categorised. Nonetheless, there were six teachers (8.6%) in the ABOVE range for ISTOF who were in the BELOW range for the QoT factors. Moreover, there were also six teachers classified in the BELOW range in ISTOF but in the ABOVE range in QoT. It is likely that time and lesson differences (one term) in the observations taken as well as differences in the foci of the two instruments affect the consistency in groupings. Teachers may vary in their effectiveness over time short or longer term. In addition, different raters may vary in their judgments on different occasions and contexts to some degree, even when inter-rater reliability is generally sound. When factor scores obtained in the two instruments were

xii


OUTPUT

compared the correlation of the mean aggregate scores in both instruments was very low (r=0.04). Of the thirty possible correlations between factors of the two scales, there were only ten significant but weak correlations.

The strongest

significant associations were between the QoT Supportive lesson climate and the three ISTOF factors Structured lessons that maximize learning and involvement (rho 0.25), High quality communication and interaction (rho 0.27) and Purposive learning

(rho

0.23).

The

QoT

factor

Proactive

lesson

management

was

significantly correlated with Structured lessons that maximise learning and involvement (rho 0.24). The field researcher judgment of overall teaching quality on the QoT instrument correlated only modestly but significantly with the overall mean ISTOF factor score (rho 0.25) and with the ISTOF factor Purposive learning (rho 0.28) and also with the ISTOF factor Structured lessons that maximise learning and involvement (rho 0.25). Interestingly there were two QoT factors that correlated weakly but negatively with the ISTOF factor Active Learning. These were Adaptation of teaching (rho 0.21) and Teaching learning strategies (rho -0.28). Earlier results had pointed to these later aspects as not necessarily being core features of effective classroom practice for this sample. School context (in terms of the measure of social disadvantage of intake

%

pupils’ eligible for FSM) was not significantly correlated with teachers’ observed practice except for weak but negative associations with the overall mean score for the ISTOF factors (rho -0.26), and with the factor Responsiveness to student needs that deals with differentiation (rho -0.31) and with Active learning (rho0.26).

This suggests that teachers in the sample may find such strategies less

appropriate or less useful in such contexts. There was, by contrast, a positive correlation between % pupils’ eligible for FSM and the QoT factor Adaptation of teaching. In general, the intra-scale factor correlations for ISTOF were not strong as those for the QoT, only the first two factors, Structured lessons that maximize learning and

involvement

and

High

Quality

Communication

&

Interaction,

were

significantly correlated with each other (rho 0.49) and with most other factors. In contrast, the factors of the QoT scale were all significantly correlated with one another. Their correlations were also somewhat stronger.

The strongest

association was between Proactive lesson management and Well organised

xiii


OUTPUT

lessons with clear objectives (rho 0.75) followed by Proactive lesson management and Supportive lesson climate (rho 0.63). The field researchers’ overall judgment of the quality of teaching made immediately after the QoT observation was finished was significantly correlated with all the individual QoT factors but most closely with the factor Proactive lesson management (rho 0.8). Overall, the results suggest that the ISTOF and the GRIFT instruments measure rather different aspects of teaching behaviour (one focusing on observed frequencies of behaviour, the other on judgments of strengths and weaknesses). In addition it is likely that teachers probably vary in their observed behaviour over time because the two instruments were used in two different terms, thus the study of consistency is confounded by the time factor. Further research using both of the two scales at a common time point and

rating the same teachers on

several occasions over a set period of time would be needed to investigate the extent of correspondence in more depth.

6.4

Differentiated

versus

Generic

Models

of

Teacher

Effectiveness Campbell, Kyriakides, Muijs and Robinson (2004) proposed a differentiated model for investigating teacher effectiveness, which was further elaborated by Creemers and Kyriakides (2008) into a dynamic model of educational effectiveness. This refers to multiple factors of effectiveness that operate at student, classroom, school and context levels. Campbell et al. (2004) argued that multiple factors in various dimensions may affect a teacher’s effectiveness. For example, they made a distinction between teacher effectiveness observed with time stability (such as over different lessons or periods of an academic year) and that observed with subject consistency (such as across various subjects or across diverse areas within a subject). In the present ECP study, teachers in the sample were observed for two observations in different terms and with different observation schedules.

Any consistent practices observed in the teachers on the two

instruments across the two observation occasions would imply some time stability and subject consistency for the secondary sample and at the least some time stability for the primary sample. Because teachers were rated on two different observation instruments, a third dimension of consistency, namely, instrument or measurement consistency can

xiv


OUTPUT

be studied3. If the two instruments were measuring broadly the same aspects of teacher behaviours, then any inconsistency problems would be reduced, but if they were measuring different aspects of teaching behaviours, inconsistencies might arise when teachers were found to score highly in some areas but not in the others, irrespective of time or subject areas taught. In comparing the two instruments, there are some similarities in content. For example,

both

seek

to

measure

features

classroom

climate,

classroom

management, instructional qualities and student involvement/response. In other words, when the relationships between features of teachers’ observed practices measured by the two instruments are explored, positive correlations might be expected in some of the CFA factors of the two instruments, especially in these three emphasized areas. The present results illustrate variations in teachers’ observed behaviour in relation to two possible sources of influence the: school sector (between primary and secondary) and school context (as measured by the %FSM indicator). Some sector differences (primary and secondary) and contextual differences (by level of social disadvantage of the school’s pupil intake) have been identified which indicate that effective teachers’ practice may be affected by and/or be responsive to these differences in context. Nonetheless, in most respects the differences were not strong, which provides support for the view that effective teachers share many common features across sectors and different contexts. The most important assumption of the differentiated model proposed by Campbell et al. (2004) is that teacher effectiveness should not be viewed as a generic concept. They argued against the assumption that effective teachers are likely to be effective with all students, in all contexts, in all aspects of teaching and so on. A model based on a generic concept of teacher effectiveness, they claim, is too simple. It would tend to identify a general set of characteristics that define the effective teachers and might ignore the possibility that teachers may have a range of strengths and weaknesses in their teaching practices and that their behaviour may change over time. The distribution patterns of the factor scores of the various CFA models of the two scales used provides a degree of support for

3

Instrument or scale consistency here refers to participants being rated as effective across different scales, and probably also effective in their subscales. It is expected that when the two instruments show greater similarities, it is more likely that the teachers observed would be rated similarly by appropriately trained field researchers.

xv


OUTPUT

these arguments because it is clear that there is variation across teachers in the sample of most factors, although this is greater in some areas measured than in others. Nonetheless the high scores of this purposive sample of primary and secondary teachers a number of the underlying factors and on particular items, also provide some support for the generic view of overall effectiveness.

In these areas all

teachers in the sample tend to be rated highly.

7.0 Key Factors of Effective Practice This sample of effective and more effective teachers score highly in terms of the following factors related to judgments of the quality of teaching from the QoT schedule (where higher ratings imply predominant strengths).

•

Supportive lesson climate

•

Proactive lesson management

•

Well organized lesson with clear objectives

•

Environmental and teacher support

From the ISTOF observations that explore frequency of observed behaviours we also find that the sample scores very highly in terms of the following factors:

•

Clear and coherent lesson in a supportive learning climate

•

Engaging students with assignments and activities

•

Positive classroom management

•

Purposive learning

•

Quality questioning and feedback for students

These various features can be seen as necessary and central or core characteristics of effective or high quality teaching across different sectors, subjects and contexts.

7.1

Non Key Factors with More Variation

There was greater variation within the teacher sample in other areas observed particularly in relation to the use of differentiation of work and instructional strategies for different groups of students related to the ISTOF factor Teacher strategies with respect to teacher expectations. There was also notable variation in the Responsiveness to student needs and the Active learning factors. Similarly,

xvi


OUTPUT

in the QoT schedule the factor Adaptation of teaching showed greater variation as did the factor Teaching learning strategies. These again involved the extent of observed adaptation of teaching approaches or work to meet students’ perceived needs. Secondary teachers tended to obtain lower scores than primary teachers in the sample on these factors. This may relate to differences in organization such as greater use of setting in Year 9 than in primary schools where mixed ability classes are more prevalent and therefore there may be greater diversity in approaches. Also in primary schools a teacher tends to work with the same class and therefore will have greater knowledge of individual pupil needs. Interestingly, teachers in schools in more disadvantaged contexts (higher FSM) showed lower scores on these factors. It may be that such approaches are found to be less appropriate in such contexts and this may reflect the findings of earlier teacher effectiveness studies that conclude that low SES/low ability students tend to benefit more from approaches linked to the concept of interactive direct instruction (van de Werf 2006).

7.2

Factors Associated with more Positive Pupil Views

The quantitative analyses linking teacher and pupil survey data with two observation measures enabled the identification of significant patterns of association. Although pupils’ views were generally quite or very positive (as was the case with most measures of observed practice) there was some variation amongst in these measures of pupil outcomes for this sample of typical and more effective teachers. Given the small sample size of teachers for each year group studied the results should be treated with caution. Nonetheless there were indications of significant positive links between a number of aspects of teachers’ observed practice and more favourable pupil views. The patterns of associations were strongest for the Year 9 sample, and this may reflect the more detailed instrument used for this older age group. In general teachers’ scores on the QoT instrument showed clearer links with more favourable pupil views particularly in terms of the two observation factors Proactive lesson management and Supportive lesson climate. The QoT schedule was developed in collaboration between the Dutch and English inspectorate (led by the Dutch framework) and is a higher inference instrument based on the judgment of strengths and weaknesses by trained observers (in this case field researchers) rather than frequency of observed behaviours. Field researchers commented that they found the aspects covered rather more straightforward ISTOF.

xvii

to rate than was the case for


OUTPUT

Although originally developed for observation in the primary sector, the findings here suggest the QoT instrument may well be useful in studies of lower secondary classes, at least in the areas of English and mathematics covered here.

8.0 Conclusions The evidence from the systematic observation schedules indicates that they can provide a valuable source of evidence about effective classroom practices and their interrelationships for teachers in England in this purposive sample. This evidence supports that derived from qualitative analyses of observation field notes and teacher interviews that can help to illuminate participants’ perspectives further as part of an integrated mixed method design. CFA proved a useful tool to explore the underlying dimensions in teachers’ observed behaviours. In addition they identified potential core features of effective teachers’ classroom practices, and those that showed more variation and may be viewed as non-core. The results provide some support for a generic model of teacher effectiveness but also support for a differentiated model. They also highlight a small number of significant associations between contextual factors and observed practice and a small number of sector differences. In addition, there is evidence of significant positive associations between certain core factors of teachers’ classroom practices and more favourable pupil views and perceptions, particularly for lower secondary pupils.

Further research using both observation instruments at the same time

point and in other countries would help to illuminate the extent of differences/ similarities among in terms of variations in their observed practice. In addition, it would further extend the ECP research via further exploration of the way teachers can be grouped on the basis of similarities in the variations in their observed practice. The quantitative strand of the ECP study shows that this purposive sample of effective teachers (typical and more effective) can produce useful groupings of teachers that can form part of individual teachers’ profiles or cameos. These have proved valuable in facilitating the integration of qualitative and quantitative evidence on effective classroom practice.

xviii


I:

OUTPUT

INTRODUCTION

The ESRC funded Kington, 2006)

‘Effective Classroom Practice Research, (Day, Sammons &

is a mixed methods investigation conducted in English primary

and secondary schools during 2006-2008 that aims to :

•

Describe, analyse and explain the variation in primary and secondary school teachers’ classroom practice using two different observational instruments and pupil and teacher perceptions, focussing on English and mathematics teaching

•

Explore typical and more effective classroom practice of teachers in Years 2, 6 rand 9, across different school contexts, career phases and ages in relation to observations of practice and professional, situated and/or personal factors which are perceived to affect observed practice over time.

The ECP study collected a range of qualitative and quantitative data to address these aims. Quantitative data include systematic observations of practice using two different instruments conducted in two successive terms in one academic year, for a sample of 81 teachers in Year 2, 6 or 9, a survey of pupils’ views and experiences for a class or teaching group taught by the teachers and a short teacher survey. In addition field notes made during the observations and pre and post lesson observation interviews to seek teachers’ perspectives provide rich qualitative data for the sample. This paper contributes to both the main aims from a quantitative perspective. It focuses on the description and analysis of variation in the observed classroom behaviour and practices of a purposive sample of English primary and secondary teachers using two different systematic observation instruments and data from the teacher survey. This element of the ECP research complements the qualitative evidence obtained from teacher interviews and observers’ field notes. Results from the quantitative analyses were used to develop various groupings of teachers that fed into the qualitative data analysis by incorporation into individual teacher profiles drawing on methodological approaches developed in the earlier VITAE research (Sammons et al, 2007; Day et al 2007).

1


OUTPUT

1.0 Teacher Effectiveness Research Muijs and Reynolds (2005) have highlighted the lack of a strong research base on teacher effectiveness in the UK which they partly attribute to a cultural belief that teaching is an art not a science, the lack of a tradition of studying learning and instruction within educational psychology in the UK and the dislike of making comparisons of performance between individual teachers. Elsewhere, educational effectiveness researchers have sought to study and explore a range of actual behaviours of teachers in classrooms, observing individual lessons and linking the behaviours observed to data about student outcomes such as scores on standardised tests. The early teacher effectiveness researchers in the US, who studied teacher behaviours using classroom observation instruments, gradually started to find patterns which indicated that more effective teachers (a concept of relative effectiveness defined by value added approaches as teachers whose students made stronger gains on standardised achievement tests) tended to actively teach the whole class, spending significantly more time than ineffective teachers explicitly lecturing, demonstrating or interacting with the class as a whole. As well as actively teaching the whole class for more of the time, effective teachers were found to be characterised by a number off observed behaviours. Despite the lack of a strong teacher effectiveness tradition some British studies have investigated classroom practice and pupil outcomes. Classic examples include the ORACLE study (Galton & Croll, 1980; Galton & Simon, 1980; Galton, 1995) and School Matters (Mortimore et al, 1988). The ORACLE research indicated that teachers identified as ‘Class Enquirers’ generated the greatest gains for pupils in the areas of mathematics and language whereas pupils in classes of teachers identified as operating a highly individualised approach termed ‘Individual Monitors’ made least progress. The Class Enquirers group of teachers were observed to utilise significantly more time in whole class interactive teaching than the Individual Monitors (Croll, 1996). Further analyses showed an association between progress and non-individualised interaction. The ORACLE study found that whole class interaction was positively associated with high levels of pupil time on task. The later PACE study of primary schools investigated the impact of major educational reforms such as the introduction of curriculum and national curriculum changes (Pollard et al, 1994). This also reported high levels of whole class interaction associated greater pupil task engagement. Mortimore et al (1988) studied primary school effects using an educational effectiveness design including information on children, their classrooms and

2


OUTPUT

teachers, their primary schools following children age 7 to 11 years. The research showed that teachers were spending much more time communicating with individual children than they were doing whole class teaching or facilitating collaborative group work. At classroom level the effective teacher characteristics identified were: • teacher responsibility for ordering activities during the day for pupils, i.e. structured teaching; • pupils having some responsibility for their work and independence within these sessions; • a limited focus covering only one curriculum area at a time; • high levels of teacher interaction with the whole class; • providing ample, challenging work; • high levels of pupil involvement in tasks; • a positive atmosphere in the classroom; • greater use of praise and encouragement. Teachers who spent more time interacting with individual pupils tended to focus on routine, i.e. non-work, matters and made less use of higher order questioning, or work related feedback while teachers who interacted more with the whole class made relatively more use of higher order communication and work related feedback. Mortimore et al (1988) concluded that the key classroom factors contributing to effective outcomes were structured sessions, intellectually challenging teaching, a work orientated environment, communication between teacher and pupils and a limited focus within the sessions. Recent research in the US in elementary schools has drawn attention to the independent contribution to the prediction of achievement growth of students across 1st, 3rd and 5th grade classrooms of the quality of teacher-child interactions and quantity of exposure to instructional activities, but also to their interactive effects Pianta et al, 2008).

Their review found that for first graders ‘teachers’

focus on concepts and provision of feedback have an even stronger positive effect for students whose parents have low levels of education themselves’ (Pianta et al 2008, p 4).

It concludes that ‘there is accumulating evidence that teachers’

instructional interactions with children have the greatest value for student performance when they are focused, direct, intentional and characterized by feedback loops involving student performance.’ In addition it is noted that, ’The value of intentional, focused interaction and feedback is not limited to reading but appears to be a key component in other skill domains that may extend to

3


OUTPUT

cognition and higher order thinking.’ (Pianta et al 2008, p 4) These conclusions are in accord with those of van der Werf (2006) whoh reviewed evidence concerning more recent constructivist teaching approaches with alternatives such as direct instruction using empirical data from the Netherlands. She argues that there is little evidence of positive effects of constructivist approaches on student attainment

whereas,

in

contrast,

she

concludes

that

direct

instruction

emphasizing learning as an active process is more effective strategy for raising attainment especially for lower ability students and those from disadvantaged backgrounds. Recent research on 125 primary schools in England conducted in 2005-2006 as part of the Effective Pre-School and Primary Project (EPPE 3-11) adopted two US observational protocols to explore variation in teachers observed practice in Year 5 classes (Sammons et al 2008). These were the Instructional Observation Schedule (Stipek, 1999) and the Classroom Observation Schedule Grade 5 (NICHD, 2001).

The study indicated that there was significant variation in

teachers’ observed classroom practices and revealed the existence of a number of underlying dimensions. Higher quality was found to be significantly associated with the use of a plenary session in literacy and numeracy lessons, although only half the sample studied were found to use a plenary. The authors summarized the findings that explored the relationship between children’s progress and social behavioural outcomes (measured over four years) and classroom practice as follows:

‘What matters in the classroom • The observations reveal considerable differences in teachers’ and children’s behaviours and classroom practices indicating significant variation in the quality of children’s educational experiences during Year 5. We found that teachers can be classified into different groups in terms of overall Teaching quality, and that observed Year 5 overall Teaching quality is a significant predictor of better cognitive progress from Year 1 to Year 5 in both Reading and Maths. • Quality of pedagogy in Year 5 (richness of instructional method, a positive climate, productive use of instructional time, the use of evaluative feedback, teacher sensitivity and lack of teacher detachment) was found to be a significant predictor of progress in Maths, but not in Reading. It was also

4


OUTPUT

important for children’s progress in terms of reducing Hyperactivity, and promoting Pro-social behaviour and Self-regulation. • Classroom Attention and control was related to better progress in Maths and better development in Self-regulation. Child positivity (which involved childteacher relationships, children’s co-operative skills and their self-reliance) was related to better progress in Reading. • High levels of observed classroom Disorganisation in Year 5 (related to teachers’ organisation and the behavioural climate of the classroom) were predictors of poorer progress in both Reading and Maths and increased Hyperactivity’. (Sammons et al 2008, p 2)

The results indicated that global measures of overall teacher quality could predict variation in pupil progress in both reading and mathematics (effect Sizes 0.350.37) but in addition teachers varied in more specific features of practice and these tended to relate to social behavioural as well as academic outcomes, particularly

the factor Disorganisation.

2.0 Inspection Evidence In England Inspection evidence has also been used to study teaching quality and to comment on teacher effectiveness especially since 1993 after the creation of Ofsted and introduction of regular and high stakes inspection with publication of schools’ reports. Although such evidence is not based on process product research directly linked to data on pupil outcomes (in contrast to the teacher effectiveness research tradition) it relies on professional judgment and experience and many of the findings are in accord with the results of educational effectiveness research.

The HMCI annual report has provided a regular and

detailed commentary on standards of teaching in England and specific reports on teaching different subjects have received widespread attention during the period 1993 to 2007 (the most recent inspection framework draws on the Every Child Matters agenda and involves much less detailed focus on teaching). general teacher/teaching factors were identified in Primary Matters

A number of

associated with positive outcomes for pupils (Ofsted, 1995)

and these provide a good

indication of the aspects of effective teaching considered relevant by the inspection model of teaching quality. They include: • good subject knowledge;

5


OUTPUT

• good questioning skills; • an emphasis upon instruction; • a balance of grouping strategies; • clear objectives; • good time management; • effective planning; • good classroom organisation; • effective use of other adults in the classroom. Later the development of the National Literacy and Numeracy Strategies evolved drawing on research and inspection evidence and had a major impact on practice and teachers’ perceptions of what characterised effective primary practice in England, as did the Key Stage 3 Strategy in lower secondary school. Evaluations of the strategies noted positive relationships with improvements in the quality of teaching in terms of inspection data (Ofsted, 2005), as well as links with improved pupil outcomes in national assessment outcomes and in international comparisons (PIRLS and TIMSS), although as with most educational and social research

it

is

not

possible

to

draw

causal

connections

between

such

improvements and the introduction of the strategies, nor to separate their influence from that of other initiatives intended to raise attainment. Commenting on the literacy and Numeracy strategies Webb & Vulliamy (2007) note that teachers generally perceived them to have improved primary practice in England considerably, but found from interviews that there were objections to their perceived prescriptive nature. ‘In general the Strategies were viewed very positively because they promoted continuity of teaching and learning in literacy and numeracy across all English schools, gave a structure and learning objectives for lessons, helped pupil understanding and provided ideas and resources for teachers—particularly important for those lacking specialist knowledge in English and/or mathematics. Despite their support for the Strategies, teachers were highly critical of the government for imposing them ‘in a such a way that “You don’t have to do it, it is an option, but woe betide anybody who doesn’t!”’. The Strategies were viewed as another expression of the government’s lack of trust in the teaching profession and a public declaration that teachers lacked expertise in teaching basic subjects which further lowered morale and reduced teacher selfconfidence.’ Webb & Vulliamy (2007, p 567-568)

6


OUTPUT

3.0 Systematic Observations The use of systematic observations is intended to allow for comparisons of teachers in terms of certain pre-determined and agreed categories of behaviour and practice and originated in teacher effectiveness field of enquiry (Muijs & Reynolds 2005). Muijs and Reynolds (2005) note the difference between value oriented instruments that require a value judgment from the observer compared with behavioural instruments that are designed to look at the occurrence or not of specific behaviours without passing global judgments on whether the observed behaviour was ‘good’ or ‘poor’. This is related to the distinction between high and low inference measures. Muijs and Reynolds also note that teacher effectiveness research has tended to focus on mathematics and reading and on primary classes thus the generalisability of the findings to other subjects or activities and to the secondary sector requires further study. Teddlie et al (2006) argued that the successful international comparative research will depend on the development of internationally valid instruments, particularly teacher effectiveness protocols. The International System for Teacher Observation and Feedback (ISTOF) was developed using an iterative, multiple step, internet based modified Delphi technique involving teams from 21 participating

country

experts

9researchers,

practitioners

and

education

advisers/inspectors) whose opinions about what constitutes effective teaching were used to generate the components in the instrument. The ECP team were asked to contribute to the international pilot phase of the ISTOF project and chose to use the ISTOF schedule as one of the two instruments in the study to enable its applicability to the English context to be explored. In addition, the ECP team chose an additional schedule also based on an international collaboration this time between two inspectorates (senior HMI and Dutch inspectorate equivalents). The Lesson Observation Form for Evaluating the Quality of Teaching is intended to utilise professional judgment and has a deliberate high inference evaluative framework based on professional judgments. The inspectorate research project team that developed the Form investigated reliability and validity using data collected from primary school classes in England and the Netherlands (van de Grift et al 2004). The ECP team decided to adopt the Quality of Teaching (QoT) Form as an additional observational schedule in the ECP research and trial its use in both primary and lower secondary classes. As noted above inspection has a high profile in England and therefore inspection based instrument might be expected to provide an additional source of relevant

7


OUTPUT

evidence to contribute to understandings of what constitutes effective classroom practice. The team also sought to explore the utility of the QoT instrument for use in lower secondary classes as it had been developed in the primary sector.

4.0 Objectives of the quantitative analyses The main aim of the ECP quantitative analyses was to explore the extent of variations in teachers’ observed classroom practice for a purposive sample of relatively effective teachers in English primary and secondary schools. The objectives were:

•

To examine the utility of the ISTOF and QoT instruments in the English context in identifying features of effective classroom practices.

•

To examine the relationships between features of teachers’ observed practices measured by the two instruments.

•

To explore variations in observed practices between primary and lower secondary teachers and identify the key underlying dimensions of practice.

•

To

explore

variations

in

observed

practice

between

English

and

Mathematics teachers in the secondary school sample.

•

To explore variations in observed practice according to school context (in terms of level of social disadvantage measured by the percentage of pupils eligible for free school meals indicator).

•

To investigate the relationships between variations in observed practice and teachers’ self reported commitment, motivation and stress levels.

•

To investigate any relationships between pupils’ views and experiences (derived from surveys) and variations in teachers’ observed classroom practice.

•

To produce different groupings of teachers according to overall and specific features of their observed classroom practice to incorporate into teachers’ individual profiles (in terms of whether teachers were relatively above, typical or below the sample average in specific aspects.

8


II:

OUTPUT

CHARACTERISTICS OF THE TEACHER SAMPLE

The sample consisted of 81 teachers in two groups4, 26 (32%) were chosen based on results of an earlier study, VITAE, (for details, see Day et. al. 2007) and were chosen for the ECP study based on VITAE data on pupil outcomes (views and value added attainment data). In addition, another 55 (68%) participants were newly recruited for the CP study because insufficient teachers were still contactable and willing to engage in a second longitudinal study from the original VITAE sample. These newly recruited teachers were drawn from an ongoing study of Leadership and Pupil Outcomes funded by DCSF (Day et al, 2007). It was anticipated that they might be more likely to show behaviours associated with academic effectiveness, because they were purposively selected from schools identified as academically highly improved/r effective using national assessment and examination data for three years 2003-2005 (Gu & Sammons, 2007) and were identified by head teachers and, in secondaries, by Heads of Departments, as particularly effective teachers. The sample of teachers from the original VITAE study represented only a small proportion of the original sample (less than one tenth), and included those willing to participate and who had been identified as having typical to above average effectiveness in terms of either academic outcomes or pupil views.5. Comparisons were made between the sub groups (VITAE and LEADERSHIP) in the sample to establish whether their observed classroom practice differed in particular areas but few differences were found. The numbers of teachers in primary and secondary schools were not equal, 45 (55.6%) in the former and 36 (44.4%)

since it the primary sample covered

Years 2 and 6 and it was desirable to have reasonable numbers in both years for comparison. The teachers selected for the sample were those who taught either Year 2 or Year 6 classes in the primary and Year 9 in the secondary sample and the class observations took place in Year 2, 6, or 9 classes. While most primary

4

The sample size is relatively small which reduces the potential to test validity and reliability. Nonetheless, given the mixed methods design of the current study, the use of quantitative data can be triangulated with qualitative data to explore the potential utility of the two instruments

5

Unfortunately, teachers in VITAE had already been involved in a lengthy 4 year longitudinal research project and it proved difficult to recruit as many of them as originally planned to a further study due to attrition/mobility and concerns over workload. Nonetheless the inclusion of a further sample of teachers drawn from a more academically effective group of schools allowed additional exploration of the potential relationships between school effectiveness and teachers’ classroom behaviour to be conducted.

9


OUTPUT

teachers taught only one year group, secondary teachers generally taught several year groups. This explains why the bars of secondary year levels in Figure 1 are higher than those of primary ones. Figure 1 shows that the majority of secondary teachers taught classes across 4 to 6 different year groups (unfortunately 7 teachers, 8.6%, did not return the biographic survey and so certain background data for these are missing).

1.0 Gender There were more female (N= 53, 65.4%) than male teachers (N=28, 34.6%) in the sample, reflecting sector differences in the wider teaching population. Only about one-fourth of the primary school teachers were male. For primary teachers, there were no subject distinctions as they typically taught all subjects, but for the secondary sample, taught either Mathematics or English as their prime subject.

40

COUNT

30

20 35

31

21

18

22

22

13

9

11

vel

12 yea r

r 11 yea

9 yea r

10

8 yea r

yea r

7 yea r

r6 yea

5

r4

5

yea r

yea

2 yea r

r1 yea

rec ept ion

r3

4

3

0

yea

7

A le

10

29

Figure 1: Year group which the teachers taught

2.0 Age Excluding the 7 cases that had not returned the questionnaire, the age distribution of the remaining 74 teachers was slightly positive skewed and bimodal (Figure 2, nearly one-third was 30 or under and almost half was under 35, while one-fifth was in the age range of 50-55 years. Given the age distribution in Figure 2, four main age groups were identified and compared in

10


OUTPUT

subsequent analyses: 30 or under (N=22, 29.7%), 31-40 (N=18, 24.3%), 41-50 (N=16, 21.6%), and 51 or above (N=18, 24.3%).

25

Frequency

20

15

10

22

14

14

5

10

6

4

2

0 30 or under

31-35

36-40

41-45

45-50

51-55

56-60

1

1

61-65 over 65

Age of the teacher

Figure 2: Frequency distribution of age of the teacher

3.0 Educational levels Approximately half of the teachers (N=38, 51.4%) listed only one qualification. The most common educational qualification among the participants was a degree (N=60, 81.1%), while about two fifths (N=31, 41.9%) also had a postgraduate certificate or diploma. The highest level of qualification reported by the participants in the sample was a Masters. Of the 10 teachers (12.4%) who had a Master’s degree, five also possessed a postgraduate certificate or diploma.

4.0 Teaching experience The distribution pattern of the length of time in teaching (i.e., teaching experience) was similar to that of age distribution Out of the 74 teachers who returned a questionnaire, about thirty percent (N=27) had 4-7 years of experience, while about 25% (N=18) had taught for 8-15 years and another 25% (N=19), 24 years or more.

As shown in Figure 3, primary teachers were in

general slightly more experienced than the secondary teachers in this sample.

11


OUTPUT

12.5 Level at which the teacher teaches primary or secondary

Count

10.0

Primary Secondary

7.5 13

14 12

11

5.0

7

7

7

7

2.5 2

1

0.0

0-3 years 4-7 years 8-15 years

16-23 24 years years or more Total Length of time teacher has been a teacher

Figure 3: Length of teachers’ experience

5.0 Number

of

school(s)

taught

and

years

in

current school

25

Level at which the teacher teaches primary or secondary

Count

20

Primary Secondary

15 25

10 17

14

15

5

1

0

1 2-5 6-9 Number of schools teacher has taught in

Figure 4.

Number of school in which the teacher has taught

Figure 4 shows that over half the teachers (N=41, 56.9%) had taught in more than one school, although the proportion of teachers that had taught in only one

12


OUTPUT

school (N=31, 43.1%) is still large. Figure 5 shows that this relates to experience. As might be expected, teachers who have taught only one school were more likely to have taught in their current school for fewer than 8 years.6

Length of time teacher has taught in current school

20

0-3 years 4-7 years 8-15 years 16-23 years 24 years or more

Count

15

10 16

5

9 5

0

6

10

7

4

1

1

2

1

1 2-5 6-9 Number of schools teacher has taught in

Figure 5: Length of time in current school and number of schools in which the teacher has taught

Again, as might be expected teachers who have taught in only one school were less experienced. Primary school teachers in the sample tended to have taught in their current school for a longer period than secondary teachers in the sample. In particular, there were more primary than secondary school teachers who had taught for over 8 years in their current school.

6.0 Post(s) of responsibility Most participants (N=56, 69.1%) reported their post as class teachers. However, a number were also Key Stage managers (N=11, 13.6%) or subject coordinators (N=32, 39.5%), some were department heads (N=13, 16.1%), and a number had other posts or duties (N=14, 17.3%). There were also 13 (16.1%) teachers in senior rank as assistant heads, deputy heads or heads of the schools. The details of the posts of teachers are summarized in Table 3.

6

χ2=12.002, df=8, p=0.151, but Linear by Linear Association Test=4.192, df=1, p=0.41 and Somers’d =0.247, p=0.018.

13


OUTPUT

Level at which the Post held by the teacher

teacher teaches Primary

Secondary

Total

Class teacher

37

19

56

Advanced Skills teacher

1

2

3

KS Manager

10

1

11

Subject coordinator

30

2

32

Deputy Head of Year

0

1

1

Head of Year

0

2

2

Deputy Head of Department

0

5

5

Head of Department

2

6

8

Assistant Head

1

3

4

Deputy Head

6

0

6

Head

2

0

2

Other post or responsibility

10

4

14

TOTAL

43

31

74

Table 3: The post(s) of teachers in primary and secondary schools

12

Length of time teacher has taught in current school

12

10

0-3 years 4-7 years 8-15 years 16-23 years 24 years or more

10

Count

8 6

6 5

4

4

4 3

4 4 4

3

2

2 1

1

1

0 1

2 3 4 The number of post(s) held

5

Figure 6: Number of post(s) held and years taught in current school

14


OUTPUT

Figure 6 indicates that the number of posts held is negatively associated with the number of years that the teacher has taught in the current school, especially among teachers who have taught in the current schools for fewer than 8 years7.

20

Level at which the teacher teaches primary or secondary

20 19

Primary Secondary

Count

15

12

10 8

8

5 3

3

1

0

1

2 3 4 The number of post(s) held

5

Figure 7: The number of post(s) holding in primary and secondary school Since primary school teachers tended to have taught in the current schools longer than their counterparts in the secondary schools and the number of post(s) held is very much related to the number of years in the current school, more primary schools teachers in the sample tended to hold more posts than their secondary counterparts, as shown in Figure 7.8

7

χ2=17.54, df=4, p=0.02, Linear by Linear Association Test=14.92, df=1, p=

IND.

ITEM

(0.05)

(0.01)

0.5

NO.

NO.

2

2

1.2

4

2

2

0

1

ITEM CONTENT The teacher explains how assignments are aligned to the learning goals of the lesson

2○6

The teacher motivates the students to think about the

1○2

The teacher asks students to identify the reasons why

30

Students are invited to give their own examples

advantages and disadvantages of certain approaches

5.3

5

4

1

0

3.2

5

3

2

1

5.4

specific activities take place in the lesson

The length of the pause following questions varies according 6

3

3

0

4.2

18

to the difficulty level of the questions The teacher invites students to explain the different steps of

6

5

1

0

5.1

22

the problem solving strategy which they are using The teacher clarifies the lesson objectives at the start of the

7

5

2

0

3.2

11

lesson The teacher encourages students to ask one another questions and to explain their understanding of topics to one

24 ○

another

7

5

2

0

5.2

9

4

5

1

1.1

1

or not.

9

8

1

0

7.1

39

Teacher starts lesson on time

The teacher makes explicitly clear why an answer is correct

Table 13: The 10 ISTOF items ranked by the lowest number of significant correlations

Table 13 identifies ten items with the lowest number of significant correlations. These items were significantly correlated with ten or fewer of these items and

38


OUTPUT

also have a low number of correlations significant at p