Teacher evaluation: A comprehensive framework for ... - Springer Link

Journal of Personnel Evaluation in Education 1:157-180, 1987 © 1987 Kluwer Academic Publishers, Boston - Manufactured in the United States of A m e r i c a

Teacher Evaluation: A Comprehensive Framework for Supervisors JOSEPH MURPHY

College of Education, Department of Administration, Higher, and Continuing Education, University of Illinois at Urbana, 1310 South Sixth St., Champaign, IL 61820

What can we say about teacher supervision and evaluation in our schools today? In many places there is no systematic supervision whatsoever of teachers by school administrators. In a great number of schools, perhaps the majority, this process is performed in a ritualistic manner at best. That is, what passes for supervision is really little more than a paper audit for organizational record-keeping (Stiggins and Bridgeford, 1985; Wise, Darling-Hammond, McLaughlin, and Bernstein, 1984). Although some scholars contend that some symbolic good accrues from perfunctory teacher evaluations (Davis and Stackhouse, 1977; Meyer and Rowan, 1975), there are few real benefits and a number of disadvantages. Teachers learn little about their skills and obtain limited feedback on their performances. An opportunity to provide a much needed structure for the instructional process and reinforcement for teachers is not realized. Worse yet, token supervision degrades both the importance of teaching and learning and the professional positions of supervisors and teachers. When supervisors simply go through the motions of supervising and evaluating teachers~ they communicate two messages to staff: (1) supervisors are not particularly concerned about teaching and learning; and (2) they are not competent to make credible judgments and decisions in these areas. This pattern in turn diminishes the possibility that administrators and other supervisors can meaningfully fulfill their roles as instructional leaders. Token supervision does little to help individual teachers and administrators. It is also rarely, if ever, used to guide school wide improvement efforts. It does not lead to the selection of meaningful staff development activities for staff, to improved instructional decision-making, or to better curricular programs. Nor does it help produce the kind of information that is needed in the evaluation process to make basic decisions about retention and promotion of staff (Stiggins and Bridgeford, 1985). What is the picture in those schools where the evaluation process is taken more seriously? Two approaches are most common in these schools--checklist and clin-

158

J. MURPHY

ical supervision approaches. Many schools use a checklist system to supervise teachers. In these schools a formal process generally guides the reporting of the supervisor's judgments about teaching--principals have a standard form or checklist which they complete and share with teachers. However, in these schools, there is often little substance to the process (Wise et al., 1984). There is neither a systematic method of collecting supervisory information nor a focus in the categories used in the process on the critical elements of teaching and learning (Natriello and Dornbusch, 1980-81). Such systems generally require the principal to make high inference judgments based on global impressions rather than upon specific, objective facts (Bridges, 1984; Levin, 1979). Many checklist systems are designed to provide an assessment of the teacher as a person rather than an analysis of the teaching-learning process in the classroom (Peterson, 1985-86). For example, in their review of district checklists, Wood and Pohland (in Stiggins and Bridgeford, 1985) found that only 28% of the items related to the instructional role of the teacher. This global, checklist approach to supervision suffers from almost all of the same problems as perfunctory evaluations (Bridges, 1984; Medley, Soar, and Coker, 1983). In a growing number of districts there is a movement toward the use of systematic and meaningful supervision and evaluation procedures. Many of these fall under the rubric of clinical supervision, a generic model of supervision emphasizing the use of formal conferences, focused observations, and systematic data analysis (see Glickman, 1985; McGreal, 1983). While these more systematic procedures offer the potential to overcome many of the inadequacies of token and checklist approaches to supervision, they are not without problems themselves. Five of these problems are of particular concern to principals and other supervisors who use clinical supervision as the primary basis for working with teachers to improve instruction. First, there is a tendency for process and procedure to push out content and substance when these systems are employed (Stiggins and Bridgeford, 1985). Teacher-principal conferences and classroom observations often assume greater significance than the objectives and content embedded in these activities. While it is important to recognize the significance o f process in the effective use of clinical supervision models, it is equally critical that the process be grounded on a firm knowledge base about effective teaching and learning. Second, clinical supervision models rely exclusively on formal observations as a method of collecting information about what is happening in classrooms. Although it is clear that formal observations are needed, reliance on this approach provides too limited a strategy for effective supervision. A number of students of organizational theory (Cohen and Miller, 1980; Cohen, Miller, Bredo, and Duckworth, 1977; Duckworth, 1981; Owens, 1984) have argued that administrators need to rely heavily on informal and indirect methods of control to be successful instructional leaders. This has been corroborated in studies of effective schools which find high levels of principal visibility in classrooms but relatively few formal observations (Firestone and Wilson, 1985). Supervision systems that rely almost exclusively on

A COMPREHENSIVE FRAMEWORK

159

formal procedures will not be as productive as ones that incorporate both formal and informal strategies for collecting information. Supervisors also need to be aware that most clinical supervision approaches use a rather narrow database--an often limited number of formal observations and principal-teacher conferences. In addition, most clinial supervision models focus on only a few specific objectives during these conferences and observations. This approach to collecting information about what is occurring in classrooms is too narrow for a number of reasons. Teaching is a complex activity, and it is the responsibility of the supervisor to develop a comprehensive picture of the teachinglearning process in classrooms. This cannot be done adequately by focusing solely or primarily on the handful of objectives generally emphasized in clinical supervision. Second, in spite of the great strides made in understanding effective teaching (see Brophy and Good, 1986; Rosenshine, 1983), there is still a good deal of uncertainty about the connections between instructional activities and student learning. The teacher effects and the school effectiveness literature suggests that a variety of weak to moderate effects come together to create student learning. If this is the case, principals need to gather as much information in the evaluation process as possible. They also need to employ an array of strategies to collect that information. Another problem with clinical supervision is that it is often unconnected to school goals and other important schoolwide activities. While one of the great strengths of clinical supervision is the emphasis on individual diagnosis, reflection, and prescription, it is important that such activities be undertaken within the framework of school goals. The evaluation process affords supervisors a natural opportunity to reinforce the mission of the school. For example, if a schoolwide goal is to implement an interactive model of teaching mathematics (see Good, Grouws, and Ebmeier, 1983), then objectives emphasized in clinical supervision should relate directly to that goal. Differences in approach would be possible, but differences in direction would not. Along these same lines, the focus of clinical supervision on individual growth often obscures its use in schoolwide decisionmaking in such areas as instruction, curriculum, staff development, and long-range planning. Finally, most clinical supervision programs are nonsituationally based--they treat everyone the same. All teachers in a school are subject to the same set of conferences, observations, data analysis strategies, procedures, and so forth. Such an approach ignores the fact that the supervisory orientation of principals and other supervisors and the procedures and methods they employ should vary depending upon the characteristics of individual staff members. For example, in advocating a developmental approach to supervision, Glickman (1981) argues that principals should emphasize one of three approaches or orientations to supervision (directive, collaborative, or nondirective), depending on two characteristics of teachers--their level of commitment to teaching and their level of abstraction, or ability to reason abstractly, think symbolically, and solve problems.

160

J, MURPHY

Purposes of supervision and evaluation programs The ultimate purpose of instructional supervision and evaluation is to improve the quality of teaching and learning in a school. Supervision/evaluation activities help reach the goal of improved instruction, both directly and indirectly. Direct efforts to improve instruction occur when the teacher and supervisor work to improve existing teaching behaviors or to develop new ones that are positively related to student outcomes. Often overlooked, however, are the indirect ways that evaluation promotes improved instruction. This less-direct route to better instruction occurs when the evaluation system: (1) provides organizational structures that teachers often need but generally lack, e.g., formal structures for instructional selfassessment and peer observations; (2) raises instructional issues to the forefront in schools; (3) improves the level of professional dialogue between teachers and administrators; (4) contributes to higher quality and more frequent collegial discussions among staff; and (5) helps to develop a sense of instructional continuity and consistency in the school. Supervision and evaluation systems can be used by principals and other supervisors to reach many ends. We have just reported on some of the indirect benefits such as greater professional collegiality that can accrue to a school. We also noted that the primary purpose is the improvement of instructional practice in individual classrooms. Holding teachers accountable for the implementation of the schools' instructional focus and curricular objectives is another function of teacher evaluation programs. However, many current supervision systems are as inept in the area of accountability as they are in the area of promoting meaningful instructional improvement. A third goal of the supervision and evaluation process, schoolwide decisionmaking, is often overlooked, even in good systems, when the sole focus is the instructional improvement of individual teachers. Supervision data should be aggregated to the school level to provide a basis for schoolwide decisions, e.g., the selection of staff development programs and the determination of school goals. In many schools, the principal is the only person who is capable of developing an instructional gestalt. Synthesizing evaluation data about the instructional strengths and weaknesses of teachers can assist supervisors in developing this larger perspective.

Principles of quality evaluation systems A comprehensive listing of quality indicators for an evaluation system could be quite lengthy. Therefore, in this section our discussion is limited to nine principles that we believe to be the most critical and that capture important elements missing from many current evaluation programs. 1. Support of school mission. One of the fundamental conclusions of recent school improvement research is that schools work better when the parts fit


161

together, when plans and activities are coordinated in a common effort to reach important school goals (Purkey and Smith, 1983). Yet one of the characteristics of many evaluation systems is their lack of connectedness to other school activities. Stiggins and Bridgeford (1985, p. 89) report, for example, that "teacher evaluation operates, more often than not, as an independent, self-contained system." In addition, the idiosyncratic nature of the evaluation process does not encourage the selection of objectives derived from a common framework. If school districts are going to invest the resources necessary to meaningfully evaluate teachers, they should structure these programs so that results are achieved in a variety of important areas--instruction, staff morale, student achievement, and so forth. One major payoff would be the promotion of school purpose or mission. In order for this to occur, evaluators should insure that mutually selected objectives relate directly to school goals. For example, if one school goal is to provide students with opportunities to improve their writing skills in all classes, then strategies for infusing writing skills into the curriculum should be a focus of supervision and evaluation for all teachers in the school. 2. Instructional research focus. Teachers generally have a rather large array of objectives from which they can make selections for inclusion in the supervision and evaluation process. If evaluation activities are to provide significant benefits for teaching and learning, supervisors should guide the objective selection process toward those areas where the research about teaching reveals highest payoffs for students (Stiggins and Bridgeford, 1985). Just as many school goals are developed through the use of needs assessments which measure very little of importance, teacher objectives are often selected with little regard for what is known about quality instruction. In addition, as noted earlier, many evaluation instruments are only distally connected to the teaching-learning process that unfolds in the teacher's classroom (Peterson, 1985-86). It may be a truism, but it is worth emphasizing that evaluation systems that do not focus on teaching and learning are unlikely to promote improvement in either area. 3. Multiple sources of data. A number of researchers have called for the use of multiple sources of information in teacher evaluations (Epstein, 1985; Levin, 1979; Stiggins and Bridgeford, 1985). Yet even in the best evaluation programs too much emphasis is devoted to formal observations. As a result, other methods of collecting information, e.g., review of teacher and student work products, student interviews, analysis of test results from various sources, peer evaluations, and so forth, are often underemployed (Levin, 1979; McLaughlin, 1982). Given the complexity of teaching and learning, a well-thought-out supervision and evaluation system should not rely solely, or primarily, on one data source. Rather, multiple sources of information should be used to make decisions about the appropriateness and quality of teaching. 4. Continuous process. Two of the erroneous ideas that continue to be perpetuated in educational practice are, first, that significant improvements can occur in a relatively short period of time and, second, that the introduction of change strategies insures improvement. Nowhere are these beliefs more heavily ingrained

162

J. MURPHY

than in supervision and evaluation programs (Stiggins and Bridgeford, 1985). Even many of the better systems emphasize very short timelines and noncontinuity of review, i.e., substantively unconnected assessments of instruction every second or third year. Unfortunately, the evidence from the school improvement literature indicates that neither of these two embedded assumptions is accurate (FuUan, 1982). Research leads to the conclusion that administrators should use multiyear evaluation systems. Multiyear approaches are needed to provide sufficient time for developing a reasonable composite picture about teaching and learning in a given classroom. Equally important, a multiyear framework for supervision and evaluation provides sufficient time for skills and practices to become fixed or institutionalized within a teacher's repertoire of behaviors. If evaluation systems are to reach their full potential for the improvement of both individual teachers and the overall school learning climate, lock-step, uncoordinated, yearly regimentations must give way to more long-term, coordinated, and ongoing approaches. 5. Focus on student outcomes. Although it may seem strange to suggest that the evaluation system should focus on student outcomes, there is good justification for doing so (Stiggins and Bridgeford, 1985; Lezotte & Bancroft, 1985). A primary purpose of an evaluator is to insure quality and equality of student learning. The only way to ascertain whether this is actually occurring is to examine proximate student outcomes such as time-on-task and more direct measures such as scores on various achievement tests. While emphasis on the instructional strategies to reach these outcomes is important, and often the direct focus of teacher evaluations, analysis of these strategies should not be the end product. To put it another way, change/improvement strategies are successful not because they are successfully implemented but because they produce improvements in important organizational outcomes. There are strong pressures on supervisors to emphasize processes and to avoid inspecting outcomes when assessing the quality of classroom instruction (Meyer and Rowan, 1975). Embedding the principle of outcome focus in the structure of the evaluation system will help supervisors surmount some of these pressures. 6. Integration with other school and district systems. Evaluation programs need to be connected to and coordinated with other important organizational systems. We have already discussed the importance of grounding the evaluation program on the goals of the district and school. Evaluation systems should also be carefully integrated with organizational rewards and sanctions, a connection conspicuous by its absence in most schools (Natriello and Dornbusch, 1980-81). Many evaluation programs are also only tenuously connected to important school and district planning activities in the areas of curriculum and instruction (Stiggins and Bridgeford, 1985). For example, the aggregation of the evaluation results of individual teachers for use in the school and district level decision-making processes is rare (Wise et al., 1984). Likewise, the link between aggregated evaluation information and systemwide staff development activities is weak (Stiggins and Bridgeford, 1985). Yet the absence of integration between the evaluation system and other organizational activities substantially reduces the quality and usefulness of the evaluation program.


163

Integration of the evaluation program with these and other organizational systems is essential for meaningful school improvement. 7. Outcome significance. Most supervision and evaluation systems accomplish none of the three ends for which they were established. They neither promote instructional improvement nor insure accountability. Few capitalize on the potential for gaining schoolwide benefits. They do provide a paper audit and fulfill some symbolic functions, but the results of most evaluation systems are simply not used (Natriello and Dornbusch, 1980-81; Wise et al., 1984). There are few direct or indirect outcomes from the process. For evaluation programs to be effective, they must result in some effects or outcomes. Minimally, this means improvement in teaching and learning in individual classrooms. Ideally, it means this plus accountability, schoolwide improvement, and such indirect benefits as greater professional collegiality and greater staff morale. 8. Active involvement of staff. A great deal has been learned in the past 15 years about the importance of ownership, or the active involvement of the people who will be expected to carry out proposed changes (Fullan, 1982). On the other hand, research designating the bottom-up process of change as the only way to make improvements in schools has been oversold (Guskey, 1986; High and Achilles, 1986; Miles, 1983). The collective message from the change literature seems to be that superintendents, principals, and other supervisors can exercise considerable control by coordinating school goals and change strategies for their staffs (Schwille, 1986). Yet they need to work hard to involve staff, or to bring them along, as improvement efforts progress. In the specific area of evaluation, the research would indicate that maximum involvement of staff should be the goal. Because the emphases in evaluation are on the change of often complex behaviors, the individual, and judgments of competency, it is imperative that teachers be deeply involved in all aspects of the process (Darling-Hammond, Wise, and Pease, 1983; Stiggins and Bridgeford, 1985). 9. Commitment of adequate resources. Many teacher evaluation programs are ineffective because sufficient resources to fuel the systems have not been provided (Natriello and Dornbusch, 1980-81). In the absence of needed resources, evaluations are often conducted in a ritualistic manner with emphasis on minimum performance standards and bureaucratic procedures (Stiggins and Bridgeford, 1985). On the other hand, Wise and his colleagues (1984) found that school districts with particularly effective evaluation systems devoted more resources to these programs than did other districts. Sufficient resources for useful evaluation programs would include at least the following: funds, time, and training. Additional money and time are needed to free administrators and other supervisors to complete carefully crafted evaluations--to conduct regular observations, to collect and analyze evaluation data from a variety of sources, and to meet with teachers to provide specific suggestions for improvement. In addition, all staff in a school or district must be trained in the evaluation program. Training should be based on the purposes, procedures, and content of the specific evaluation system.

164

TIMELINE Year 1

© Z ~0

©

O u. Z

Year 2

Observations Formal Informal_ teacher work products Record-keeping systems Lesson flans Grade books Home-school contacts Student works products Daily irk Home ork Proj Test results Teacher/textbook l-referenced lized Teacher If-evaluation Interviews Parents Students Peer Review Questionnaire Parents Students Meetings Evaluatee Central office s t a f f Other hool personnel Professional development log School develooment loe

.=

,.,

o

•~ "= ~

Figure 1. The supervision matrix.

0

~

,.

"~

~

~.~ "~ -6

~

PROCESS

165 Year 3

DIAGNOSIS: ASSESSING THE LEARNER

Formal assessment Informal assessment Monitoring pupil progress Communicating assessment information CURRICULUM DECISION-MAKING: DETERMINING/ORGANIZING LONG-TERM INSTRUCTIONAL PROGRAM

Specifying goals and objectives Curriculum analysis skills Awareness/selection alternative programs Sequencing instruction Individualization/needs of special students Grouping strategies INSTRUCTIONAL DESIGN; PLANNING THE LESSON

Using general learning theory General lesson planning skills Selecting/planning for alternative teaching strategies INSTRUCTION" TEACHING THE LESSON

Applying general learning theory Questioning skills Responding skills Explaining skills Behavior management skills Using alternative teaching strategies Evaluation and monitoring strategies

O

CREATING AND MANAGING THE LEARNING ENVIRONMENT

Use of time and space Use of materials and media Classroom management strategies Social climate and multicultural dimension Learning style/social pattern accommodation Record-keeping/program monitoring SUBJECT AREA COMPETENCE AND SPECIALIZATION

Knowledge of concepts and principles Knowledge of specific teaching techniques Curriculum development Knowledge of resources, instructional programs and instructional reserach HOME/SCHOOL/COMMUNITY RELATIONS

=

.=.

Parent communication Parent involvement Community relations HUMAN RELATIONS

,

Communication practices and skills Participation in school and district activities Leadership skills PROFESSIONAL GROWTH

Keeping up-to-date with development in field Building personal growth plan Participating in professional development activities Contribution to field

Z

166

J. MURPHY

The supervision matrix In this section we present an evaluation matrix that supervisors can use to build their own evaluation programs. The matrix is designed both to overcome the deficiencies common in many approaches to teacher supervision and to incorporate the principles of quality evaluation just discussed. As noted in Figure 1, the matrix has four dimensions: a timeline for collecting and analyzing information; content areas on which information is collected; a process for working with teachers; and information sources from which data about the quality of the teaching-learning process can be extracted. Below, we briefly review the first three components of the matrix and analyze the information sources in detail. An underlying assumption of the matrix is that there is no single best way to evaluate staff. While the structural dimensions that define evaluation are fixed, the actual array of sources selected and content areas analyzed can be tailored to meet the needs and interests of both the supervisor and the individual staff members. Another assumption is that teachers can work on a variety of strategies to improve classroom instruction. The supervision process does not need to be exclusively devoted to improving specific skills and meeting narrowly defined goals.

Timeline

The supervision timeline presented in Figure i extends over three years. This seems to be a reasonable period of time over which to collect and analyze data about teaching performance. One question that may be raised is, which teachers get supervised when? We recommend a four-year cycle with 25% of the staff in each of the three years of the supervision process and 25% out of the cycle for any given year. If the principal cannot work with everyone, the rule to follow is this: it is better to supervise fewer people in more depth than to provide token or perfunctory reviews for the full staff every year.

Content

While it is beyond the scope of this article to provide a full description of all the content areas that can be assessed in the evaluation process, the nine teaching functions shown in Figure 1 provide a good basis around which to collect data. These functions provide the substance which information sources are designed to assess. Stated more directly, the information sources should tap into the content areas. For example, supervisors should focus discussions on these content areas when gathering information about staff members from district coordinators and school resource teachers. The combination of clear specification of teaching functions (content) with the use of mutliple sources of information will (1) direct super-


167

visory focus to areas that really make some difference in student outcomes, and (2) allow supervisors to make strong inferences about what is occurring in classrooms.

Process

The process used in the evaluation matrix can be flexible. The process goals are as follows: to involve individual staff actively in the supervision process; to keep lines of communication open and discussions flowing; and to insure that the evaluation program leads to regular formative improvement as well as summative statements about the quality of teaching and learning at the end of the three-year cycle. Regular meetings and open discussions form the heart of the process component of the evaluation matrix. At the initial meeting, the supervisor and teacher begin to build an individualized evaluation matrix. The supervisor and teacher each bring to this meeting whatever information is available to help them focus on the content areas and data collection sources that will be emphasized. At a minimum this would include the following: a teacher self-assessment; copies of school goals; results from previously completed teacher evaluations; the teacher's professional growth objectives; and test data on student achievement patterns for the school, grade level, or subject area, and the individual teacher's current and former students. The point to stress here is that one of the most powerful features of the evaluation matrix is its comprehensiveness, its focus on a broad definition of teaching rather than on a few specific skills. At the same time, it is important to note that every content area need not be assessed and that all the information sources need not be used with each staff member. The goal is to maintain the comprehensiveness of the program while tailoring the focus of the content areas and information sources to the needs of the individual staff members, as determined within the context of the school mission and goals. As noted in Figure 1, regular meetings should be scheduled throughout the year. Although no hard and fast rules apply here, it is recommended that meetings be held about every three weeks, or that there be approximately 12 meetings per year. Six of these will be used in conjunction with three formal observations (a meeting before and after each observation), and one should be held at both the beginning and the end of the school year. This leaves four scheduled meetings to be spread over the year as needed. The reason that they are built into the structure of the evaluation matrix is to insure that they actually occur, that is, that they are not neglected in the crunch of everyday work pressures. The supervisor should also be available for and encourage the use of less formal and less structured visits throughout the year. Meetings should be held in both the principal's office and the teacher's classroom or office. In addition to these meetings, the supervisor should provide teachers with an opportunity to discuss all written feedback received in the evaluation process throughout the year, e.g., notes on review of student work products and comments on teacher lesson plans.

168

J. MURPHY

It should not come as a surprise that the supervision approach emphasized in this article is very systematized. The structure of the matrix is fairly well set in advance. The major array of data sources are thought out, content is focused on the best available research, and the supervisory process is built around a general set of timelines. However, formalized structure should not be taken to mean uniformity of approach in the supervisory process. Leadership and evaluation are situational in nature. Different contexts within a school and varying conditions in the school environment will make behaviors that are highly effective in some situations much less effective in others. In short, there is no one best set of leadership behaviors generally, nor is there one best approach for supervising all teachers.

Information sources The sources of information that can be used to improve instruction in a school are potentially quite large. Unfortunately, as noted earlier, most supervisors continue to rely on only a few sources when evaluating teachers. Below we take a closer look at some of the types of information sources that administrators can use in assessing the quality of classroom instruction.

Formal observations. In many scools formal observations are an exercise in futility. Teachers receive little reinforcement for effective teaching practices and few meaningful suggestions for improvement; student learning remains unaffected; and teacher perceptions of supervisors as curricular and instructional leaders are not enhanced. There is little advance planning for supervisory observations and teachers are often unaware of either the purpose of the visits or the scope of the review. Casual exchanges in the hall often suffice for conference meetings (Natriello and Dornbusch, 1980-81). Yet formal observations can form an integral part of an evaluation program if they are properly conducted. A well-conducted observation schedule includes: (1) a preconference in which the substantive focus and procedural conditions for the observation are established; (2) an observation during which the supervisor objectively describes and records activities that are occurring in the classroom; (3) a period of analysis and interpretation of the data collected during the observation; (4) a postconference during which the supervisor and the teacher review the analysis of the lesson together for the purpose of planning further efforts at instructional improvement; and (5) a postconference analysis in which the supervisor and teacher analyze the usefulness of the first four phases of the observation cycle. Informal observations. Informal observations in classrooms by supervisors should occur on a regular basis. Principals in small schools should be able to visit the classroom of each teacher being evaluated daily. In larger schools it may take the principal up to a full week to make the rounds of all teachers in the evaluation


169

cycle. While many informal visits should be unplanned, the majority of them should be loosely scheduled. By this we mean that the principal (1) plans to be in a given room a certain number of times a year; (2) visits with a specific focus in mind; and (3) provides brief written feedback on what transpired during the visit. The focus for these visits and the basis of the feedback provided are the content areas noted in Figure 1. An example of an informal observation feedback system is presented in Figure 2. It immediately focuses attention on four important factors during each visit: (1) student activity (what students are doing and rates of engagement with learning materials); (2) teacher activity; (3) subject matter content (relation of the lesson to the school curriculum); and (4) school/teacher objectives (any evidence of activity, either of a positive or negative nature, that relates to school goals or teacher objectives for the year). Twenty or more of these informal observation records per class per year can help supervisors to develop a clear picture of teaching and learning in each classroom. Combined with other data sources, informal observations allow administrators and other supervisors to make judgments about teacher performance with a good deal of surety.

Review of teacher record-keeping systems. Once a year the supervisor should review classroom record-keeping systems with each supervised teacher. Specific factors that principals should look for include evidence of systematic pupil-monitoring systems, the connection between assessment and instructional planning, and the relation of the monitoring systems to the approved curriculum. Review of lesson planning. Administrative review of teacher lesson plans is often attacked as an infringement upon the professionalism of staff. We believe that this need not be the case. Teachers who understand the place such a review has in the larger evaluation system and who receive meaningful and useful feedback on their lesson plans are less likely to view this process negatively. The analysis of lesson plans can yield a good deal of information about both short-term and long-term curriculum planning, teacher expectations, instructional design, and so forth. Supervisors should be leery of neglecting such a rich source of instructional information. Review of home-school contacts. Supervisors should encourage teachers to keep records of their contacts with parents. Maintaining a bank of parent calls and visits, filing copies of weekly or monthly parent letters, and recording the names of students whose parents receive notes are not difficult tasks. Yet most teachers do not keep these records. Those who do rarely, if ever, have a chance to share the evidence of their work with their principal. Systematic review of these records by supervisors communicates the message that home-school relations are important, provides a substantive basis for assessing that teaching function, and acknowledges an area of teacher activity that is often overlooked. We recommend a semiannual review of contacts between the teacher and the home.

170

J. MURPHY

William James Middle School INFORMAL OBSERVATION NOTE

Teacher Name:

Date & Time:

Room Number:

Subject:

Student Activity (include engaged time):

Teacher Activity:

Content (include connection to course objectives):

Relations to Yearly Goals/Objectives:

Note:

(signature) Jose Martinez, Principal

Figure 2. Informal observation record.


171

Review of student work products. Most principals do not systematically review student work products. Yet these artifacts provide one of the most important sources of data available for understanding how teaching and learning unfold in specific classrooms. A general framework for reviewing the artifacts of teaching developed by McGreal, Broderick, and Jones (n.d.) is presented in Table 1. An analysis of student work products should be made within this larger framework and with reference to the content areas listed in Figure 1. Figure 3 provides an example of a checklist that meets these criteria and can be used by supervisors to review student work products. We suggest that for a two-week or three-week period all student work for each teacher being evaluated should be collected and reviewed. Two such reviews each year for each supervised teacher should provide a sufficient amount of material. Homework is one type of student work that deserves the special attention of the supervisor. Well-thought-out and prepared homework assignments can raise the level of academic press at the school and classroom levels and can increase student learning and achievement (Keith, 1985; Paschal, Weincstein, and Walberg, 1984). In order for homework to be effective, it must be: (1) systematically and regularly assigned; (2) structured so students achieve high rates of success; (3) collected, marked, and recorded; and (4) incorporated into the course grades of the students. These are the criteria by which supervisors should review completed homework assignments. A concomitant review of completed homework papers and teacher grade books should provide evidence about whether homework is a meaningful extension of learning time and content or a waste of student time and effort.

Review of test results. Analysis of outcomes generally and student achievement results specifically are limited in many schools. Student achievement scores are rarely used in either teacher evaluations (Glassman, 1984; McLaughlin, 1982) or principal evaluations (Peterson, 1984). (See Haertel, 1986, for a discussion of student achievement results in teacher evaluations.) Yet they do provide a valuable source of information for decision-making within the evaluation process (Stiggins and Bridgeford, 1985). Three points must be emphasized here. First, multiple sources of test data, such as teacher-made tests, commercial curriculum tests, and criterion-referenced tests, should be examined. Second, gain scores or scores which reflect growth in student learning over a period of time are the most important factors to consider when using test scores in the evaluation process. Third, test scores need to be disaggregated by gender, race, socioeconomic status, language proficiency, and so forth, to assess for equity of learning outcomes. Teacher self-evaluation. Instructional self-assessment is an important component of the overall supervision process, yet it is generally absent from most teacher evaluation programs (Darling-Hammond, Wise, and Pease, 1983; Stiggins and Bridgeford, 1985). Self-analysis should occur within the context of school goals and the nine teaching functions listed in Figure 1. This self-review process can help the

172

J. MURPHY

Table 1. A framework for analyzing the artifacts of teaching. Content Quality of artifacts can be considered from the point of view of content or essential meaning. Some considerations related to quality of content are: 1. Validity--Is the artifact materially accurate and authoritative? 2. Appropriateness--Is the content appropriate to the level of the intended learner? 3. Relevance--Is the content relevant to the purpose of the lesson? 4. Motivation--Does the artifact stimulate interest to learn more about the subject? Does it encourage ideas for using the material? 5. Application--Does the artifact serve as a model for applying learning outside the instructional situation? 6. Clarity--Is the content free of words, expressions, and graphics which would limit its understandability? 7. Conciseness--Is the artifact free of superfluous material? Does it stick to the point?

Design Design of artifacts should proceed from an analysis of the content of the lesson or instructional unit. High quality artifacts are those which conform to instructional objectives. The quality of an artifact is the product of its design characteristics, its relevance to instructional objectives, and its application to content. 1. Medium selection--Is the most appropriate medium used for meeting each objective and presenting each item of content (e.g., films, textbooks, teacher-prepared handout)? 2. Meaningfulness--Does the artifact clearly support learning objectives? If so, is this made apparent to the learner? 3. Appropriateness--Is the design appropriate to the needs and skill levels of the intended learner? Are time constraints considered in the artifact's design? 4. Sequencing--Is the artifact itself sequenced logically? Is it employed at the appropriate point in the presentation? 5. Instructional strategies--Is the artifact format appropriate to the teaching approach? Does its construction incorporate sound learning principles? 6. Engagement--Does the artifact actively engage the learner? Does it reinforce the content with appropriate practice and feedback questions? 7. Evaluation--Is there a plan for evaluating the effectiveness of the artifact when used by the intended learner? Can the success rate for the artifact be easily determined?

Presentation Presentation considerations include physical and aesthetic aspects of an artifact as well as directions for its use. 1. Effective use of time--Is the artifact suitable for the time allotted? Is learner time wasted by such things as wordiness or extraneous information unrelated to learning objectives? 2. Pace--Is the pace appropriate to the level of the learners, neither too fast nor too slow? Does the pace vary inversely with difficulty of content? 3. Aids to understanding--Are directions clearly explained? Are unfamiliar terms defined? Are important concepts emphasized? 4. Visual quality--Do the visuals show all educationally significant details? Is composition uncluttered? Does the composition help the learner to recognize important content? Are essential details identified through appropriate use of highlighting, color, tone, contrasts, position, motion, or other devices? Is the type size of any text legible from the anticipated maximum viewing distance? 5. Audio quality--Can the audio component be clearly heard? 6. Physical quality--Is the artifact durable, attractive, and simple? Are size and shape convenient for hands-on use and storage?

Source: From Thomas L. McGreal, Eileen Broderick, and Joyce Jones. Artifact Collection: An Alternative Source of Data about Teaching for Use in Instructional Supervision. Unpublished manuscript. University of Illinois, UrbanaChampaign. Used with the permissionof the authors.

.. Time on task (student engagement) a. Do students complete most work assigned b. Is there evidence that enough exercise or practice has been assigned to provide for student improvement and expected growth c. Is there evidence of given massed practice when new skills or concepts are introduced d. Is there evidence that class, as a group, achieves expected mastery before moving onto another topic or assignment 2. Feedback to students by teacher a. Is there evidence that eacher has processed the work; i.e., evaluated, commented, corrected, made suggestions, recommendations b. Is feedback clear and enable student to follow course of action c. Is there evidence of rapid return of feedback to student d. Is comment or criticism constructive; i.e., rc-inforces desired behavior; accurately describes the problem or need, nonjudgmental 3. Coherence or logical sequence of assignments, activities, exercises a. Is there evidence of logicat sequence of steps to assignments given b. Is there evidence of student understanding, of the sequence and logic of the assignment or subject matter c. To what extent is students' work free of confusion d. Does the assignment or exercise move the student toward the objective (Is there clear connection between activity and objectives) 4. Clarity of directions, explanations, etc. a. Are the directions, explanations, etc., clear to the evaluator b. Is there evidence of student understanding (few or no students misinterpreting directions, explanations, etc.) 5. Structure, clarity of objectives a. Is there evidence of organization of planning of the exercises b. Is there evidence that purpose and objectives of lesson have been understood, carried out c. Is there evidence that students understand purpose of exercise, its objectives, how the exercise helps them improve where needed d. Is there evidence that exercise is based on diagnosis or assessment of students needs, aptitudes or interests

Source: The Milpitas United School District, Milpitas, CA. Used with the permissionof the district. Figure 3. Checklist for analysis of student work folders.

174

J. MURPHY

supervisor and teacher establish: (1) an indication of which content areas will receive the major focus throughout the three-year evaluation cycle; and (2) specific objectives to emphasize in the formal observation component of that cycle. Supervisors should ask teachers to assess themselves on the nine content areas and the school goals before the first conference of the three-year cycle (see Process, Figure 1). Teachers should be able to present support for their judgments during this initial conference. Similar assessments should occur at the mid-points and endpoints of the three-year cycle. Again, emphasis should be placed on support for teacher judgments about their performance. No matter how many sources of information are used by supervisors to assess teaching and learning, reasoned teacher analysis of their own skills provides an important base for decision-making in the evaluation process. Peer review. Although there is some use of peer review by department chairpersons at the secondary level, in most schools it is an underutilized resource (McLaughlin, 1982; Stiggins and Bridgeford, 1985). Recent research indicates that professional collegiality can be an important catalyst for improving classroom instruction and the overall academic milieu at the school level (Little, 1982; Rosenholtz, 1985). Peer review can be a key ingredient in promoting such staff collegiality. Two points about peer review should be underscored. First, there must be a clear focus and rationale for these visits. Visiting for the sake of visiting will not do much to improve instruction. The rationale is to improve teaching and learning. The focus should be on those content areas from Figure 1 that are being emphasized by the host teacher in the evaluation process. Feedback must be provided to teachers. The organizational structure of these visits must be carefully laid out by the principal. Second, the process can contain a number of voluntary and required phases. For example, the principal can mandate the use of a schoolwide peer review system. Decisions about which teachers review the instruction of their peers are made collaboratively by the teacher and the supervisor, with teachers being permitted to express their ideas and feelings about whom they feel comfortable working with. The supervisor emphasizes his or her knowledge about which teachers are particularly skilled in given content areas. Finally, the peer team decides which information gained from the peer review process is shared with the supervisor and is incorporated into the larger evaluation system. Parent and student interviews and questionnaires. Parents and students can be important sources of information about teaching quality. By and large, however, both sources of data are misued (Epstein, 1985; Levin, 1979). Student data are generally not collected at all. When parents are asked to assess instruction, they are often provided with needs assessments of questionable value, i.e., they are often asked either high inference questions about topics that they cannot reasonably be expected to answer and/or questions that measure very little of real importance. The principle in developing questionnaires and assessments for parents and students is quite simple. Specific questions should be asked about the teaching-learning

A COMPREHENSIVE F R A M E W O R K

175

process as reflected in: (1) the evaluation content from Figure 1; and (2) school and teacher goals and objectives for which parents and students can be reasonably expected to have answers. Obviously, this will require different assessment instruments for parents and students. Figure 4 provides an example of a student questionnaire that meets the criteria noted above. The choice of whether to use questionnaires and/or interviews can be made by the supervisor. Small group interviews often encourage students to provide richer detail than is obtainable through questionnaires. If small group interviews are used, all students from each teacher being evaluated can be interviewed each year. Questionnaires can be given during each of the three years of the evaluation cycle.

Meetings with other staff. Other staff members, such as resource teachers and district coordinators, are generally able to provide specific information about the teaching-learning process in classrooms. These staff members often have information that is not easily attainable in other ways. Supervisors can conduct both formal and informal meetings for the purpose of collecting such information. Formal meetings in which discussions are conducted about all teachers in a department or grade level provide a good context for gathering information from resource teachers and district coordinators.

School and professional development logs. Supervisors should ask teachers in the evaluation cycle to keep records of all professional development activities attended. Most teachers participate in an array of inservice activities and formal courses at their own expense and on their own time, yet their efforts often go unnoticed and unrewarded. Similarly, teacher contributions to the organization outside their own classrooms need to be recognized in the evaluation process. Teachers are the logical ones to record this type of information. Our own experiences indicate that teacher resistance to this increased paperwork and/or administrative intrusion into their personal activities rapidly disintegrates when they see how the information is used to enhance assessments about their professional commitment. The information from these logs can be transferred to the supervisor's records on a yearly basis. This aggregated form allows principals to see patterns in professional development activities for individual teachers and for the staff as a whole. It also provides the supervisor with another leverage point for influencing the types of professional development activities that staff members select.

Conclusion

In this article we examined current practices in the area of teacher supervision and evaluation. We noted that in many schools supervision is a ritualistic activity. We also pointed out the most significant problems with "clinical" approaches to teacher supervision. Next we reviewed the purpose of supervision and evaluation programs and discussed nine principles of quality teacher evaluation systems. Building on this

176

J. M U R P H Y

Figure 4. Student assessment form. Please m a r k the following questions as follows:

Mways

A. Diagnosing and monitoring 1. The class moves at about the right pace for me. 2. I get e n o u g h practice with a skill or area before the class moves onto a new skill or area. 3. If I don't understand, the teacher finds a way to explain so that I do understand. 4. T h e work is not too hard or too easy for me. 5. The teacher tests often e n o u g h to let me know how well I understand the material. 6. T h e teacher uses different ways (i.e., homework, questions, projects, written reports) to measure how m u c h I know. 7. The teacher tells me what to study in preparation for a test. 8. T h e tests are based on what was covered in the course.

9. Assignments and tests are returned promptly. 10. R e t u r n e d assignments and tests have comments from the teacher which help me understand what I need to work on. 11, T h e teacher reviews/discusses work to help me understand m y strengths and weaknesses. 12. Everyday the teacher uses some way to determine what I need to know.

B. Decision-making 14. T h e teacher knows his/her subject area. 15. The teacher adjusts the course to the students' interests and needs. 16. T h e teacher relates events that happen in life to the subject being studied. 17. T h e teacher uses different grouping patterns (i.e. whole class, small group, pairs, etc.) depending upon the lesson.

Often ;ometimes

Rarely

Never


18. The teacher uses a variety of materials.

C. Planning Instruction 19. T h e teacher discusses objectives and expectations of the course. 20. T h e teacher discusses the objectives and standards for each lesson so that I know what is expected of me. 21. At the beginning of the class, the teacher tells me what activities we will be doing during the class. 22. The teacher explains how lessons are related to the previous lessons.

D. Instruction: teaching the lesson 23. The teacher uses questions and responses from the students duing a lesson. 24. W h e n students give wrong answers, the teacher helps t h e m develop the right answers. 25. T h e teacher encourages questions if material/ lesson is not understood. 26. The teacher gets responses from the students to determine if they are learning during the lesson. 27. The teacher uses more than one m e t h o d of teaching (i.e., lecture, worksheets, student presentations, etc.) during a lesson. 28. The teacher's teaching m e t h o d s encourage me to maintain interest during class. 29. The teacher explains difficult ideas in more than one way. 30. The teacher acknowledges good work and responses from the students. 31. The teacher leads discussions well.

E. Managing the learning environment 32. The teacher helps all students to feel comfortable in the class. 33. The teacher is fair with all students. 34. The teacher avoids offending or putting down students. 35. The teacher builds confidence and self-esteem in all students.

177

178

J. MURPHY

36. There is fair and consistent discipline in the class.

F. Home-school relations 37. When problems arise, the teacher works with me and if necessary with my parents to work out a satisfactory solution. 38. The teacher uses parent and other adult volunteers in the class.

G. Human relations 39. I feel comfortable going to my teacher with concerns or problems. 40. The teacher shows interest for and supports school activities outside the classroom.

Source: The Milpitas Unified School District, Milpitas, CA. Used with the permission of the district.

earlier analysis, we presented a matrix that supervisors can use to develop effective evaluation programs. The important aspects of the matrix were: timeline that is of sufficient length to allow important information to be collected; content areas that focus on teaching and learning; quality processes for working with staff during supervision; and multiple sources of information about activities unfolding in the classroom.

References Bridges, E.M. (1984). The Identification, remediation and dismissal of incompetent teachers. Burlingame, CA: Association of California School Administrators. Brophy, J. & Good, T.L. (1986). Teacher behavior and student achievement. In M. Wittrock (Ed.), Handbook of research on teaching (3rd ed.). New York: Macmillan. Cohen, E.G. & Miller, R.H. (1980, October). Coordination and control of instruction in schools. Pacific Sociological Review, 23(4), 446-473. Cohen, E.G., Miller, R.H., Bredo, A. & Duckworth, D. (1977). Principal role and teacher morale under varying organizational conditions: research memorandum. Stanford, CA: Stanford University, Stanford Center for Research and Development in Teaching (NIE Contract No. NE-C-00-3-00627). Darling-Hammond, L., Wise, A.E. & Pease, S.R. (1983, Fall). Teacher evaluation in the organizational context: a review of the literature. Review of Educational Research, 53(4), 285-328. Davis, M.R. & Stackhouse, E.A. (1977, April). The importance of formal appearances: the implementation of programs for the evaluation of elementary schools and teachers. Paper presented at the Annual Meeting of the Pacific Sociological Association, Sacramento. Duckworth, K. (1981, September). Linking educational policy and management with student achievement. Eugene: Center for Educational Policy and Management, University of Oregon. Epstein, J.L. (1985, August/September). A question of merit: Principals' and parents' evaluations of teachers. Educational Researcher, 14(7), 3-10. Firestone, W.A. & Wilson, B.L. (1985, Spring). Using bureaucratic and cultural linkages to improve


179

instruction: the principal's contribution. Educational Administration Quarterly, 21(2), 7-30. Fullan, M. (1982, February). Implementing educational change: progress at last. Paper prepared for a Conference on the Implications of Research on Teaching for Practice, sponsored by the National Institute of Education, Warrenton, VA. Glasman, N.S. (1984, Fall). Student achievement and the school principal. Educational Evaluation and Policy Analysis, 6(3), 283-296. Glickman, C.D. (1985). Supervision of instruction: a developmental approach. Boston: Allyn and Bacon. Glickman, C.D. (1981). Developmental supervision: alternative practices for helping teachers improve instruction. Alexandria, VA: Association for Supervision and Curriculum Development. Good, T.L., Grouws, D.A. & Ebmeier, H. (1983). Active mathematics teaching. New York: Longman. Guskey, T.R. (1986, May). Staff development and the process of teacher change. Educational Researcher, 15(5), 5-12. Haertel, E. (1986, Spring). The valid use of student performance measures for teacher evaluation. Educational Evaluation and Policy Analysis, 8(1), 45-60. High, R.M. & Achilles, C.M. (1986, April). Principal influence in instructionally effective schools. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco. Keith, T.Z. & Page, E.B. (1985). Homework works at school: national evidence for policy changes. School Psychology Review, 14(3), 351-359. Levin, B. (1979). Teacher evaluation--a review of research. Educational Leadership, 37(3), 240-245. Lezotte, L.W., & Bancroft, B.A. (1985, March). Growing use of the effective schools model for school improvement. Educational Leadership, 46(6), 23-27. Little, J.W. (1982, Fall). Norms of collegiality and experimentation: work place conditions of school success. American Educational Research Journal, 19(3), 325-340. McGreal, T.L. (1983). Successful teacher evaluation. Alexandria, VA: Association for Supervision and Curriculum Development. McGreal, T.L., Broderick, E. & Jones, J. (no date). Artifact collection: An alternative source of data about teaching for use in instructional supervision. Unpublished manuscript. Urbana-Champaign: University of Illinois. McLaughlin, M.W. (1982). A preliminary investigation of teacher evaluation practices. Santa Monica, CA: Rand Corporation. Medley, D.M., Soar, R.S. & Coker, H. (1983, May). The minimal conditions for valid evaluation of teacher performance. Unpublished manuscript; Bridges, Incompetent teachers. Meyer, J.W. & Rowan, B. (1975, August). Notes on the structure of educational organizations: revised version. Paper presented at the Annual Meetings of the American Sociological Association, San Francisco. Miles, M.B. (1983, November). Unraveling the mystery of institutionalization. Educational Leadership, 41(3), 14-19. Natriello, G. & Dornbusch, S. (1980-81). Pitfalls in the evaluation of teachers by principals. Administrator's Notebook, 29(6), 1-4. Owens, R.G. (1984, April). American high school: characteristics of organization and culture and their impact on the exercise of leadership by the principal. Paper presented at the Annual Meetings of the American Educational Research Association, New Orleans. Paschal, R.A., Weinstein, T. & Walberg, H.J. (1984). The effects of homework on learning: a quantitative synthesis. Journal of Educational Research, 78(3), 133-139. Peterson, D. (1985-86). Developing teacher evaluation systems with potential for increasing student performance. Educational Research Quarterly, 10(2), 39-45. Peterson, K.D. (1984, December). Mechanisms of administrative control over managers in educational organizations. Administrative Science Quarterly, 29,573-597. Purkey, S.D. & Smith, M.S. (1983, March). Effective schools: a review. Elementary School Journal, 83(4), 427-452. Rosenholtz, S.J. (1985, May). Effective schools: interpreting the evidence. American Journal of Educa-

180

J. MURPHY

tion, 93(2), 352-389. Rosenshine, B. (1983, March). Teaching functions in instructional programs. Elementary School Journal, 83(4), 335-351. Schwille, J. (1986, April). A longitudinal study of the effects of district curriculum policies on the content decisions of teachers. Paper presented at the Annual Meetings of the American Educational Research Association, San Francisco. Stiggins, R.J. & Bridgeford, N.J. (1985, Spring). Performance assessment for teacher development. Educational Evaluation and Policy Analysis 7(1), 85-97. Wise, A.E., Darling-Hammond, L., McLaughlin, M.V. & Bernstein, J. (1984, June). Teacher evaluation: a study of effective practices. Santa Monica, CA: The Rand Corporation.