Large Lecture Sections: An Experiment Test of Their Effectiveness Brian K. Arbour Assistant Professor
[email protected] Andreas Karras Lectuer
[email protected] Ernest Lee Lecturer
[email protected] Department of Political Science John Jay College of Criminal Justice, CUNY 899 Tenth Avenue New York, NY 10019
Abstract Can a large lecture sections be an effective means of student instruction? The creation of a single large section class we created provides a natural experiment to address this issue. Students in our large section class were indistinguishable from students in regular-sized classes on our Department’s assessment exam. On the other hand, we find that students gave lower ratings to the instructor and the class on post-semester surveys. We find that most, though not al, of these differences come from students who sat in the back half of our architecturally challenged classroom. Our results indicate that there are plusses and minuses to large section courses, and administrators and department chairs must weigh both in deciding whether to adopt large section classes.
Electronic copy available at: http://ssrn.com/abstract=1546252
Arbour, Karras, Lee TLC Paper To many professors, large lecture sections are a “necessary evil” of teaching at research or state universities. University administrators have created large lecture sections as a result of the pressure created by increasing enrollments and the demands of faculty members for reduced teaching loads. At John Jay College, we are facing these same demands, as enrollments have increased and the administration has worked to increase the coverage of lower-division courses by fulltime faculty members. In the Political Science Department, these pressures are most acute on our Government 101 course—Introduction to American Government. The course requires 30 to 40 sections of 32 to 38 students each semester. The vast majority of Government 101 sections are taught by adjunct professors. The demand for Government 101 and the preference to have tenure-track faculty teaching students prompted Professor Arbour to propose an experimental large lecture section for Government 101, which was offered in Fall 2009. This paper uses the leverage provided by the development of a single large lecture section as a natural experiment to test the effectiveness of large lecture sections. A large literature has developed on teaching large section classes. This literature identifies a number of drawbacks to large section courses, and recommends a number of ways that professors can mitigate these drawbacks. This literature has, to this point, not yet assessed the issue of large sections in the study of political science. We tried to implement several of the suggestions of the large lecture section literature. Here, we assess our large lecture section course. We assess the effect of large sections on learning outcomes by means of a pre-semester and post-semester assessment exam. We also test the effect of large classes on student satisfaction through an ad hoc post-semester survey and the college’s official student evaluations. Because the other Government 101 sections offered at 1
Electronic copy available at: http://ssrn.com/abstract=1546252
Arbour, Karras, Lee TLC Paper John Jay are regularly-sized (approximately 36 students), we compare the results for the large sections to those of sections taught by other instructors in Fall 2009 and Prof. Arbour’s regular sized sections of Government 101 taught in Spring 2009. Our findings are mixed. On one hand, we find that students in the large lecture section had similar results to those in the regular sized sections. This indicates that the large lecture section did not depress student learning outcomes. On the other hand, we find that students in the large lecture section gave lower scores on the ad hoc student survey and on the student evaluation survey than previous students of Professor Arbour, indicating that the large lecture section reduced student satisfaction with the course. Further analysis shows that while students in the back half of the classroom gave significantly lower ratings than those in the front half of the classroom, classroom seating does not explain the whole difference between outcomes on the surveys. These results presented here indicate that large lecture classes may not be an “evil” unto themselves, and that that with use of elements of the best practices literature, their negative effects can be ameliorated. Still, results do show some negative effects created by holding the large lecture section, specifically with regard to student satisfaction rates. Should political science departments adopt large lecture sections for introductory courses, or should our discipline move to abandon these classes? Our results here do not provide a conclusive answer, and indicate that department chairs and administrators must carefully weigh the costs and benefits of large lecture sections. In particular, the results speak to the importance of room design as an important feature of the success of a large section class. We also believe that the results show that instructors in large section classes must adopt some of the best practices recommended by the literature, 2
Electronic copy available at: http://ssrn.com/abstract=1546252
Arbour, Karras, Lee TLC Paper
The Pro and Con on Big Sections The literature on the pedagogical viability of large lecture sections is generally framed in terms of their potential adaptability to teaching strategies traditionally employed in small or medium-sized classrooms. As such, there are no affirmative arguments that point to the desirability of large lecture sections as pedagogical ends in and of themselves. Instead, the literature follows two general themes. First, large lecture sections have significant pathologies which inhibit student learning and satisfaction. Second, instructors can use corrective methods to mitigate the negative effects of large lecture sections to produce a rough equivalence with the experience to smaller, more intimate, classroom settings. Cuseo (2004) summarizes the empirical case against large size classes and points to a number of salient weaknesses inherent to the teaching of large groups of students. First, large section classes inhibit both the teaching and learning processes. Large classes encourage an overreliance on lectures on the part of faculty, and a corresponding passivity in the learning styles of the students. Because of the emphasis on lecturing and the lack of in-class student-faculty dialogue, large sections reduce the depth of student thinking. Also, instructors with large class sizes often have limited assumptions about possible learning outcomes, which leads to an overreliance on multiple choice examinations, and a dearth of writing opportunities for students. The second set of objections to large section classes are based on the lack of useful communication between instructors and students. Large classes discourage the active involvement of students—reducing the possibilities for active learning. These communications issues are exacerbated by a discrepancy between the perceptions of faculty members and students on the degree of student-teacher interaction that is actually taking place (Cuseo 2004: p 3
Arbour, Karras, Lee TLC Paper 4). Further, large class sizes reduce student-faculty interaction and reduce the frequency and quality of feedback to students. Put simply, it is easier to administer a large class by reducing the number of exams and assignments. In this sense, large classes can deprive students of the opportunities for positive academic socialization that can be provided through small richer and more frequent feedback in a small section class. The third set of objections to large lecture sections deals with student satisfaction. Students report less satisfaction with large classes as reflected in high absentee rates, higher incidents of class incivility (talking etc.) and cheating. Further, students reported that they were more easily distracted and less motivated given the impersonal nature of the class and the lack of accountability. Put together, these objections create an indictment against large section classes. Another set of literature on large sections comes to the defense of large section classes. This defense is not an argument in favor of large lecture sections as an end themselves, but a set of suggestions that instructors can use to reconfigure instruction methods to mitigate these drawbacks. Weimer (1987) offered up a compendium of techniques that enhance the interactions amongst students and between student and teacher so as to make the large class a less alienating and more intellectually rewarding experience. Lowman (1987) argued for diversity in incentive structures so as to appeal to both grade oriented (GO) as well as learning oriented (LO) students. However, the suggestions for GO oriented students appear to be more concrete while the countervailing suggestions for the LO students appear to be more attitudinally motivated. More generally, Peter J. Fredrich (1987) anticipates future developments more explicitly in his call for active learning in large lecture sections. He argues for multiple teaching strategies to accommodate the different learning styles of students through “energy shifts” within a given 4
Arbour, Karras, Lee TLC Paper class period. Fredrich groups these strategies into five suggestions:; interactive lecturing (or brainstorming), Socratic questioning, small group interactions, critical thinking and problem solving exercises, and large class debates, simulations and / or role playing. These strategies are not mutually exclusive and can be mixed in matched in experimental fashion to facilitate various forms of student engagement and interaction. The point is that the whole of the class is greater than the sum of its parts. Fredrich’s article is very synoptic and his strategies were echoed, in their various manifestations, by succeeding “practical advice” books on large sections. Carbone (1998) makes the point that social science research has established that the adult attention span is about twenty minutes (p 48). As such, professors who believe they are covering material over the course of a lecture, after a point, are deluding themselves. Finally, McKeachie (2002), frames his chapter on teaching large sections with the argument that you can still get active learning (pp 227-230). McKeachie emphasizes spot, low stakes, writing assignments (p 227), having students take turns at “fishbowling exercises” in which they present before the class, and “poster sessions” (p 229) in which students explain their work to smaller groups of roving questioners. Thus, this “best practices” literature suggests a number of improvements that instructors can adopt to reduce the negative features of large lecture sections. Do these improvements make large lecture sections effective? Maxwell and Lopus (1995) analyzed the cost effectiveness analysis of large sections and concluded that, indeed, economic savings were achieved “without sacrificing quality as measured by student achievement”, but at the disturbing expense of student retention for the major in question (p 168). And in the case of the political science major, the problem is similar. The college freshman surveys for 1999 (Mann p.263) and 2001 (Bennett and Bennett p 295-96) both indicate that interest in the political science major has remained static in 5
Arbour, Karras, Lee TLC Paper terms of the percentage of first year students who expressed an interest in continuing with political science as their major. This suggests that retention is a goal that may be at odds with the goal of saving money through the re-introduction of large lecture sections. Thus, the large section literature shows that there are number of drawbacks to large lecture sections, but that instructors can work to ameliorate these drawbacks through application of active learning techniques in large lecture sections. In the next section, we discuss our large section class at John Jay College, and how we tried to adapt these practices in our classroom.
Government 101 at John Jay College At John Jay College, we implemented a large section Government 101 in the Fall 2009 semester. The proposal to create a large section course at John Jay was driven by two factors. First, a large section course could help increase the number and percentage of students taught by full-time faculty members. Currently, the Political Science Department1 offers 30 to40 sections of Government 101 each semester. These classes are capped at 40 students per section, and that size often compels instructors to follow traditional lecture methods of instruction. Because of the large number of students served by a relatively small department, roughly 90 percent of these introductory courses are taught by adjuncts.2 Second, our administration has asked the Department to develop a common course design across the various Government 101 sections. Adopting a large section course helps reduce the
1
Yes, it is the Political Science Department which runs Government 101. It is bureaucratically easier to change department names than course names and numbers 2 From Spring 2006 through Fall 2009, the Political Science Department at John Jay offered 289 sections of Government 101, a mean of 36.1 per semester). Of these, 26 were taught by tenure track faculty members and 263 by adjunct instructors of varying levels of experience
6
Arbour, Karras, Lee TLC Paper number of sections offered each semester,3 while providing a common experience to a large number of students. The experimental large section was implemented as a possible solution to the these two issues. Our Provost approved a single large section course in Fall 2009 on an experimental course, contingent on implementation of an assessment program for the course The goal of our course was to explore the impact of large sections on the department’s agreed upon Government 101 learning outcomes as well as to have a full time faculty member teach the material. The experimental large section was comprised of 150 students that met in one of the college’s auditorium style classrooms for a pair of 75 minute classes each week. No discussion sections were offered. In addition to the full time faculty member teaching the class (Professor Arbour), there were also two teaching assistants to assist the instructor. In this case the teaching assistants were advanced graduate students who had significant experience teaching the Government 101 at John Jay (co-authors Karras and Lee). While the course had a lecture focus, there were several other components to the class that were designed to foster active learning and participation. Several of these components overlap with those purported to improve student participation and evaluation in large-group college courses (Geske, 1992). These included: 1. One minute writing assignments—Several times during the semester students were instructed to take out a piece of paper and to reflect on a specific question/issue from that day’s lesson. The instructor then led a short discussion based on student answers to the writing question.
3
In Fall 2009, the large section course helped reduce the number of Government 101 sections offered to 28, a four year low.
7
Arbour, Karras, Lee TLC Paper 2. Game play—To illustrate specific topics, the class was divided into three equal sections with the instructor and teaching assistants each guiding students through playing a role-playing game. 3. Current events—The teaching assistants alternated one class a week to present current events that related to the class as well as to moderate the class discussion and answer questions. 4. As a general rule, the instructor did not stand at the lectern in the front of the room, but instead moved around the auditorium to engage students and to answer questions. In addition to the above in-class activities, there were two online components to the class. 1. Online assignments—Students completed several assignments and quizzes via the course’s Blackboard page. The online quizzes covered textbook material, which urged students to complete and understand textbook readings. In addition, these assignments gave students additional ways to demonstrate their knowledge and accumulate points. They also allowed the instructor to assess student progress in learning the stated outcomes at specific points in the semester. 2. Facebook—We created a John Jay Gov 101 Facebook group. While the instructor and teaching assistants were responsible for submitting course relevant articles and topics on which the students could comment, students were also encouraged to submit articles. This element of the course was designed to cultivate an online interactive community where students could share their thoughts to supplement the in-class instruction. Students were required to join the group and comment on articles for course credit. 8
Arbour, Karras, Lee TLC Paper Each of the above practices were adopted based on a review of the teaching and learning literature before the semester. Each was adopted as a means of enhancing student engagement with the class and the professor. As noted, the large section literature does not argue that large sections are good for students on their own. Instead, the literature argues that with the judicious application of active learning practices, large lecture sections can produce the same or similar results to smaller classes. We adopt this as the hypothesis for this paper—our large section class will produce results that are statistically indistinguishable from regular sized classes on measures of learning and student satisfaction We now address how we conducted this assessment.
Research Design In this paper, we analyze the effect of our large section Introduction to American Government on learning achieved in and student satisfaction with the class. Because Professor Arbour’s class is the only large section of Government 101 that John Jay has offered, it provides a natural experiment to test the effectiveness of large section classes. We compare the learning outcomes and student satisfaction in the large section class to the same measures in other regular sized classes at John Jay. We use three methods to conduct this assessment—improvement between a pre-semester and post-semester evaluation exam, an ad hoc post-course survey, and results of the college’s evaluation survey. Below, we discuss each of these three evaluations tools. The pre-semester and post-semester exams consist of 25 multiple choice questions. The questions were created by various Government 101 professors based on the Department’s Government 101 learning objectives (see Appendix 1 for a copy of the exam). Exam questions 9
Arbour, Karras, Lee TLC Paper were the same on both exams. After a pilot post-semester exam in Spring 2009, a full program of pre- and post-semester exams was offered in Fall 2009. In addition to the experimental large section, four other Government 101 sections were randomly selected to take the evaluation exams. Each of these sections was taught by an adjunct professor, though each of the four adjuncts had taught the course at John Jay on many occasions. The ad hoc student survey was created by Professor Arbour in Spring 2009 as he developed the proposal for a large section Government 101 class. Questions focused on topics such as student satisfaction with the class, engagement in course material, engagement with the professor, and interaction with the professor. The questions were developed by consultation with different student evaluation surveys from colleges and universities across the country, and were designed to address specific issues with large lecture classes not covered by the College’s student evaluation surveys. Professor Arbour administered the same survey questions at the end of his Spring 2009 Government 101 sections, and again in the large section in Fall 2009. The survey provided statements about the instructor or class, and students assessed their agreement or disagreement on a 7-point scale. The instructions told students that a 1 meant that they “strongly disagree” with the statement, 4 meant that they were “neutral,” and that 7 meant that they “strongly agree” (see Appendix 2 for a copy of the survey). John Jay College conducts its own student evaluations for each class at the end of each semester. These results are part of a professor’s personnel file. Students are provided statements about the instructor and can agree or disagree on a 5-point scale. We have been provided early access to the quantitative results from the large lecture section class, and can compare those results to the results from Professor Arbour’s Spring 2009 Government 101 sections.
10
Arbour, Karras, Lee TLC Paper This multi-faceted approach to evaluating the large section of Government 101 has several advantages. First, by using measures of both student opinion as well as performance, we can assess different aspects of the large section course. Again, the literature on large section classes suggests that large sections can reduce performance in these two areas. We test these different contentions here. Second, our approach tries to control over the quality of the instructor (by comparing not only different instructors, but also comparing Professor Arbour (the professor in the experimental condition) across time. In addition, by using multiple instructors from the same semester, we also control for any effects based on the time-frame of the course.
Results Again, we assess the effectiveness of the large section exam by assessing student performance in comparison to other Government 101 classes in the same semester. We also assessed student satisfaction by an ad hoc survey and through the college’s official student evaluation surveys.
Assessment Exam Results We first examine results from the Department’s multiple choice assessment exams. We employ both a pre- and post-semester exam to isolate the effect of learning by controlling for potential variation in student ability between classes. Figure 1 shows mean number of correct answers on both versions of the assessment exam, as well as the difference (improvement). The first panel of Figure 1 shows the results for the large section class and the results for all the other classes combined. Students in the large section class answered 3.5 more questions correctly at the end of the semester than at the 11
Arbour, Karras, Lee TLC Paper beginning, moving from 11.0 correct answers to 14.5. Students in the regular-sized classes answered 11.6 questions correctly before their American Government classes started, and 15.0 after all classes. The improvement of 3.5 correct answers in the large section class is basically indistinguishable from the 3.4 improvement in the smaller classes. FIGURE 1 About Here The second panel of Figure 1 breaks down the assessment exam results between the five different sections. The panel allows for a more detailed examination of the differences between the large and regular-sized sections. Students in Section 2 and Section 4 answered 4.1 more questions correctly on the post-semester exam. This level of improvement is statistically indistinguishable from large section class. Students in Section 3 improved significantly more than students in the large lecture class, while Students in Section1 improved significantly less. Overall, these results show that our large section class did not have a different impact on learning outcomes. Students’ improvement on learning outcomes in the large section was indistinguishable from those in regular sized classes. While one class improved significantly more than the large section class, one improved significantly less, and two were statistically indistinguishable.
Ad Hoc Student Survey Of course, learning outcomes are not the only measure of the effectiveness of a class or teaching method. As noted, objections to large lecture classes focus not just on what students learn, but also on their satisfaction with the learning experience in such classes. To assess student satisfaction, Professor Arbour crafted an ad hoc survey of students. The survey was administered during the final class of the semester in Professor Arbour’s two regular-sized sections in Spring 12
Arbour, Karras, Lee TLC Paper 2009 and his one large lecture section in Fall 2009. The survey instrument asked students questions about their satisfaction with the class, their engagement in course material, the level of engagement with the professor, and the level of interaction with the instructor. Figure 2 shows the mean results from 18 statements given to students. Answers are on a 7-point scale, with higher numbers reflecting strong agreement with the statement. The results frst show that in absolute terms, students were satisfied with the large section of Government 101. Students agreed with every statement presented, showing their general positive regard for the class. Students gave the highest marks in the engagement with the professor category, giving a mean 6.4 on the instructor’s interest in teaching the course, and 5.8’s to the instructor for his personal interest in students and his relationship with students. Students also gave relatively high marks in the engagement with course material section. Students thought the instructor encouraged active learning (5.7), facilitated note taking 96.0), and that exams fairly tested their understanding of course material (5.9). FIGURE 2 About Here The lowest marks concerned the class atmosphere, which students gave a mean 4.9 rating. Students also gave low ratings on the two questions that dealt with the impact of the class outside the classroom. Students gave a 5.1 on the instructors’ accessibility outside of class and a 5.0 on encouraging student-faculty interaction outside of class. The results in Figure 2 also show the results from Prof. Arbour’s Spring 2009 Government 101 sections. The comparative results show that students in the large section course gave comparatively lower ratings on every single question. The difference ranged from 1.5 points on the class atmosphere question to .5 points on the instructor’s interest in teaching the
13
Arbour, Karras, Lee TLC Paper course. Significant differences were recorded on a number of key measures between the two types of classes. In particular, large differences exist on areas of student-instructor interaction. In addition, the largest difference is on class atmosphere. As instructors, we noticed substantial issues with the back half of our classroom. First, our classroom has two large supporting columns in the middle of the classroom along the exterior aisles. These columns not only created a visual obstruction for students who sat behind them, but also provided a natural division in the classroom between front and back. Further, the classroom sloped only gently, making it difficult for students in the back half of the classroom to see the projection screen (especially the bottom of the screen). These specific issues with our classroom exacerbated the always present issues of distance to the back of a classroom and the tendency of less interested students to take seats in the back of classrooms. These issues with the back half of our classroom raise substantial questions in our mind about the satisfaction and engagement of students in the back of our classroom. By creating our own survey instrument, we were also to ask a number of questions that can be used as control variables. We asked students to report their extant GPA (in broad categories) and the number of hours each week in which they studied for the course. Most importantly, we asked students whether they sat in the front or back half of the room. Results from our survey bore out these concerns. Figure 3 shows the mean results to the ad hoc survey divided between students self-identified as sitting in the front and back halves of the room. Students in the back half of the classroom reported less agreement with every single statement. Further, these differences are significant on nearly every question. Are the issues with large classrooms really just issues with the geography of the room? The evidence presented here provides some support for that perspective. But compare the results for students who sat in the 14
Arbour, Karras, Lee TLC Paper front half of the class to those from Professor Arbour’s Spring 2009 classes. On each of the 18 statements, the mean agreement was higher in the regular sized Spring classes. Where you sit has some effect on where you stand on your satisfaction with the large lecture section, but that does not explain all of the differences. FIGURE 3 About Here As noted, students who sit in the back of a classroom are not randomly distributed. Instead, students who are less interested in the subject matter tend to sit in the back of the classroom, which is strongly correlated with lower grades. Are the relatively negative results from students in the back half of the classroom a result of the distance from the instructor, or from the type of students who would sit in the back half of the classroom? In an attempt to address this question, we constructed a series of multivariate models with the results from each statement as dependent variables. The independent variables remain constant in each model. They include Sat in Back Half of Classroom, which is coded 1 if the student reported that they sat in the back half of the classroom and 0 otherwise; 3.00 to 4.0 GPA, which is coded 1 if the student reported that their GPA was between 3.0 and 4.0, and 0 otherwise.4 The )o GPA Reported variable is coded 1 if a student did not report a GPA. Most students who did not do so are first-year students, who do not yet have a GPA. Finally, we include a variable for self-reported Hours Studied. Table 1 presents the results of these models. For 15 of the 18 models, the Back variable is significant, and in none of the models are the other three variables significant. While this provides a rudimentary attempt to disentangle the correlated effect of the architecture of the
4
Of the 82 students who reported their GPA, 55% reported a GPA between 3.0 and 4.0, 39% reported a GPA between 2.0 and 3.0, and 5% reported a GPA lower than 2.0.
15
Arbour, Karras, Lee TLC Paper room and the type of student who sits in the back of the classroom, this test does provide evidence that room architecture and not student type is the reason for the depressed ratings from students in the back half of the class. TABLE 1 About Here
Student Evaluations Like most colleges, John Jay uses a student evaluation survey as one means of assessing instructor performance. These surveys are created and scored by the College. While Fall 2009 results have not yet been made generally available, the Testing Office at John Jay provided us the results from the large section of Government 101. We are able to compare these results to those recorded in Professor Arbour’s two regular sized Government 101 sections from Spring 2009. Figure 4 presents these mean results for Spring 2009 and from the large section class in Fall 2009. In general, students gave very favorable scores in the large section class. The mean grade on all questions was 4.50. FIGURE 4 About Here But when comparing the results from the Fall 2009 large lecture section to the regularsized Spring 2009 section, a consistent pattern emerges. On 14 of the 15 questions asked on the evaluations, students gave higher scores in the regular sized sections in the Spring. Only in “deal[ing] fairly with different points of view” did large section students give Professor Arour a higher score. Yet despite the consistency, the differences between the two sets of evaluations are relatively modest. Only on five items are the differences statistically significant at the p < .05 level (to be fair, two items—“Lessons are well organized” and “material is presented clearly”— 16
Arbour, Karras, Lee TLC Paper are significant at the .10 level). For the other twelve questions, the differences do not rise to the level of statistical significance. So on one level, the differences between the regular-sized and large sections are not that great. But a look at the items where significant differences exist show that they are the ones most related to conducting a larger section. Two are related to the particular challenges of holding a larger section. Returning graded materials quickly and maintaining proper order in the classroom are both much more difficult when there are more students. Two significant differences are on items that a large section can exacerbate. Instructors are expected to handle questions or comments effectively or to clarify difficult points. But the structure of a large section discourages students from asking questions about interesting lecture topics, participating in a discussion, or asking questions to clarify difficult points. Thus, the pattern of the student evaluations follows the pattern of the ad hoc survey. Students gave high scores to the large lecture section class on an absolute level, but poorer scores compares to the same instructor’s regular sized classes. Students were less satisfied with the large section class.
Implications for Administrators and Instructors We present a paper with mixed results. On one hand, students in our large section class performed no better or worse than students in regular-sized sections on assessment exams. On the other hand, students reported lower levels of satisfaction and engagement with the same instructor in the large section class than in his previous regular-sized Introduction to American Government class.
17
Arbour, Karras, Lee TLC Paper As noted, the benefits of large section classes are often related to administrative efficiencies—the need to teach large numbers of students, reduced need for classrooms, promotion of a common curriculum. We did not directly attempt to measure these, as our goal was simply to create a large section course and leverage its potential as a natural experiment to assess student learning and satisfaction. Thus, it is difficult for us to fully address the costeffectiveness of large section classes more generally. Our results however do provide some perspective on what the costs and benefits of large section classes are. The results do indicate that the creation of large section courses can reduce the level of student satisfaction in comparison to results in regular-sized section course. The first question that administrators must answer is how highly they weight the negatives that large sections create for student satisfaction. And then administrators can consider whether these costs are greater than the benefits. The benefits of large lecture sections are clear In the face of fiscal constraints, college administrators can realize cost savings by reducing the costs of instruction per individual student. Large lecture sections, assuming that the space is available, offer a straight forward means to this end. As a matter of simple division, the expenses of teaching the class are spread over the prorated tuition contributions of many more students. Fewer instructors are involved in the teaching and so we would expect to see a reduction in the heterogeneity of teaching styles and learning outcomes. Finally, given the fact that the large lecture sections are assigned to tenure track faculty as a matter of facilitating the fulfillment of contractual teaching loads, we would expect to see an improvement in the quality of the instruction itself. Our brief experience with teaching a large section indicates that while these assumptions are based on solid rational thinking, “they aint necessarily so” in the actual practice of running a large lecture section. 18
Arbour, Karras, Lee TLC Paper As such, the worst reason to initiate large lecture sections is in the anticipation of cost savings. What may look good on paper may effectively translate into an unintended reassertion of traditional paradigms of passive learning. Students will listen to lectures, take their notes, and answer multiple choice exams en mass. Teaching assistants will hand out the class materials and facilitate the grading. Tenure track professors, those who lead the section, will apply superficial pointers about making the students feel comfortable and less alienated. But students see through this and they don’t like it. Key to the running of a large lecture section, as a successful pedagogical enterprise, is that administrators and faculty, as a matter of self-reflection, understand that they face an uphill battle in making up for the inherent alienation that most students experience in large sections. The point is to find ways to reconfigure how these sections are taught in order to make up for their deficiencies and that takes money and resources. For starters, lecture sections require newly equipped “lecture halls” that paradoxically allow professors to lecture less while facilitating more. The task at hand, in terms of a more contemporary pedagogy, is to promote student engagement and reflection on a mass scale. The goal is for students to realize, not just mastery, but the sense of relevance that will lead to their own ownership over the material in question. In this respect, audio-visual installations and wireless technologies that allow for instant feedback on quizzes and class exercises is not enough. Large lecture sections offer the promise of a new “agora,” a new form of public space that brings students, teachers, and administrators together so they can learn from each other. And they also offer the possibility for new synergies, with discussion sections, with other classes in broader learning communities, and with other student groups on campus. Professors, T.A.’s, and administrators must be willing to step up to initiate the institutional linkages that will allow
19
Arbour, Karras, Lee TLC Paper these communities to form. This is hardly the mechanistic vision of a teaching factory. It is, rather, in the best instances of education, an unceasing dialogue. Large lecture sections also place demands on instructors as well. As the large lecture section literature indicates, the negative effects of large lecture sections can be ameliorated with the application of techniques that reduce feelings of alienation and. We attempted to implement several of these recommendations in our large lecture section. While the class was lecture format, it was not exclusively conducted in this manner. In addition to the lectures, there were several one minute assignments to stimulate the students’ thinking on the class material. Additionally, role-playing and game activities were frequently incorporated into the class structure. These activities not only served to highlight important aspects of the courts materials, but it had the added benefit of getting the students out of their chairs and interacting with their peers. We also sought to facilitate contact with the students. In class, the professor moved around the room frequently to interact with students and to get questions from all areas of the room. We also created a class Facebook page as a way to create discussion of topics in current events that the large lecture section inhibited. Again, our results show that we had mixed success with these initiatives. Our selfcritique is that a number of our initiatives to increase student engagement with the course were useful, but that we needed to do a better job in executing these initiatives. In future versions of the large section course, we will work harder to post more topics on our Facebook page and in a timelier manner. We will also work to invite more students to office hours and to create a better sense of interaction inside and outside the classroom. In order for large section courses to succeed, instructors need to do more than just write up lecture notes and walk into class. In order to make a large section class work, instructors must 20
Arbour, Karras, Lee TLC Paper first be aware of the limitations of large section classes, and then plan their instruction to compensate for those limitations. Instructors can use techniques such as games, one minute writing assignments, guided class discussion to create greater engagement in the large section classes. As we noted at the beginning of the paper, some instructors see large lecture as a “necessary evil.” We show that large lecture sections are not necessarily evil all on their own. With application of practices that engage students in active learning exercises, instructors can ameliorate some of the drawbacks of large lecture sections. With the application of these principles in our classroom, we were able to blunt the negative effects of large lecture sections on learning outcomes. Despite our efforts, our students still gave lower satisfaction ratings in the large section class then they did in previous regular sized sections taught by the same instructor. The results indicate that there are both costs and benefits to large section classes. Instructors and administrators must weigh these costs and benefits to determine whether and how to implement large section courses.
21
Arbour, Karras, Lee TLC Paper References Cited Bennett, Stephen Earl and Linda L. M. Bennett. 2001. “What Political Scientists Should Know About the Survey of First Year Students in 2000.” PS Political Science and Politics 34(2): 295-299. Carbone, Elisa. 1998. Teaching Large Classes: Tools and Strategies. Thousand Oaks: Sage Publications. Cuseo, Joe. 2007. “The Empirical case Against Class Size: Adverse Effects on the Teaching, Learning and Retention of First Year Students.” Journal of Faculty Development 21(1): 5-21. Frederick, Peter J. 1987. “Student Involvement: Active Learning in Large Classes.” In Teaching Large Classes Well, ed. Maryellen Gleason Weimer. San Francisco, London: Jossey-Bass. Geske, J. (1992). “Overcoming the drawbacks of the large lecture class.” College Teaching, 20 (4), p151-54. Lowman, Joseph. 1987. “Giving Student’s Feedback.” In Teaching Large Classes Well, ed. Maryellen Gleason Weimer. San Francisco, London: Jossey-Bass. McKeachie, Wilbert J. 2002. McKeachie’s Teaching Tips: Strategies, Research, and Theory for College and University Teachers. Boston: Houghton Mifflin Company Mann, Sheilah. 1999. “What the Survey of American College Freshman Tells Us about Their Interest in Politics and Political Science.” PS Political science and Politics 32(2): 263-268. Maxwell, Nan L. and Jane S. Lopus. 1995. “A Cost effectiveness Analysis of Large and Small Classes in the University.” Educational Evaluation and Policy Analysis 17(2): 167-178. Weimer, Maryellen Gleason Ed. 1987. Teaching Large Classes Well. San Francisco, London: Jossey-Bass.
22
Arbour, Karras, Lee TLC Paper
Figure 1. Pre-and Post Semester Grades on Evaluation Exam Vs. All others
Improvement 3.5
3.4 15
14.5
Pre-Semester
11.6
11.0
Large Section
Post-Semester
All Others
Vs. Each Individual Class
Improvement 3.5
0.7
4.1
4.9
4.1
17.6 14.5 11.0
16.0
15.2 10.9 11.6
Large Section Section 1*
11.1
Section 2
Pre-Semester
12.7
11.9
Section 3*
* Significantly different from large section at p < .05 level. N’s; Large section pre: 126 , Large Secction post: 98 All others pre: 133; All others post: 98 Section 1 pre 34 Section 1 post: 22 Section 2 pre: 34; Section 2 post: 27 Section 3 pre: 33; Section 3 post: 21 Section 4 pre: 32; Section 4 post: 28
23
Section 4
Post-Semester
4.9
5.3
6.4 5.8
5.5
Exams fairly tested my understanding of course material
5.9
I would recommend this instructor to other students
6
5 Instructor conveyed the importance of teaching and learning
5.6
24
Responses on 7 point scale from 1 “strongly disagree” to 7 “strongly agree” N = 100.
The instructor was accessible to Encouraged student-faculty students outside of class. interaction outside of class
5.1
Level of Interaction with the Professor
The instructor made the subject The instructor was genuinely Displayed a personal interest in Instructor had a realistic interesting interested in teaching the students and their learning appraisal of students' course. background and experience
5.6
6
I enjoyed taking this class.
5.3
Instructor's presentation style Course developed appropriate facilitated note taking writing strategies
Engagement with the Professor
Instructor encouraged active learning
5.7
Engagement in Course Material
This course challenged me as a The class atmosphere was student. conducive to student learning
5.2
Satisfaction with the Class
Figure 2. Mean Responses to Ad Hoc Student Survey
Instructor's relationship with students was good
5.8
Instructor encouraged students to think critically
5.7
I would recommend this course to other students
5.7
Arbour, Karras, Lee TLC Paper
5.6 4.9
6.4
6.7
6.5 5.3
6
6.4
6.9 5.8
6.7
6.5
6.3
5.5
6.1
Exams fairly tested my understanding of course material
5.9
I would recommend this instructor to other students
6
6.1
5
6.1
6.6
Instructor conveyed importance of teaching.*
5.6
25
Significantly at p < .05 level. ; N Front of Classroom: 66; N Back of Classroom: 34.
The instructor was accessible Encouraged interaction outside outside of class.* of class.*
5.1
Level of Interaction with the Professor
5.7
6.3
5 Category 5
5.5
Instructor's relationship with students was good.*
5.8
6.7
Instructor encouraged students to think critically.*
Back of Classroom 5.5 5 Category 4
5.7
6.6
I would recommend this course to other students.*
Front of Classroom
The instructor made the subject Instructor was interested in Displayed a personal interest in Realistic appraisal of student interesting teaching the course.* students and their learning.* background and expertise
5.6
6.2
6
I enjoyed taking this class.*
5.3
6.3
Instructor's presentation style Course developed appropriate facilitated note taking writing strategies
Engagement with the Professor
Instructor encouraged active learning.*
5.7
Engagement in Course Material
This course challenged me as a The class atmosphere was student. conducive to student learning*
5.2
Satisfaction with the Class
Figure 3. Mean Ad Hoc Survey Responses, by Seat Location
Arbour, Karras, Lee TLC Paper
Arbour, Karras, Lee TLC Paper Table 1. OLS Models of Ad Hoc Survey Results
Sat in the Back Half of Classroom GPA 3.0 to 4.0 No Reported GPA Hours Studying R2
Sat in the Back Half of Classroom GPA 3.0 to 4.0 No Reported GPA Hours Studying R2
Sat in the Back Half of Classroom GPA 3.0 to 4.0 No Reported GPA Hours Studying R2
This course challenged me as a student. -0.27 (0.34) -0.20 (0.35) -.0.03 (0.44) 0.04 (0.04) .014
Atmosphere was conducive to student learning -1.49*** (0.37) -0.33 (0.38) 0.05 (0.49) 0.17 (0.05) .222
I enjoyed taking this class.
Instructor encouraged active learning
Presentation style facilitated note taking
Course developed appropriate writing strategies
- 0.76* (0.31) -0.07 (0.32) 0.16 (0..40) 0.04 (0.13) .073
0.61† (0.31) 0.92 (1.00) 0.33 (1.25) -0.13 (0.13) .021
0.97** (0.34) -0.16 (0.35) 0.06 (0.44) -0.12 (0.05) .087 The instructor made the subject interesting -1.38*** (0.32) -0.59† (0.33) -0.36 (0.04) 0.00 (0.04) .173
Instructor interested in teaching. -0.60* (0.24) -0.12 (0.26) 0.26 (0.32) -0.03 (0.03) .083
Accessible to Encouraged students outside of interaction outside class. of class Sat in the Back -1.03** -0.89*** Half of Classroom (0.33) (0.33) GPA 3.0 to 4.0 -0.25 -0.55 (0.34) (0.35) No Reported GPA -0.35 -0.61 (0.44) (0.44) Hours Studying 0.10 0.12* (0.05) (0.04) R2 .134 .128 N = 100; * = p < .05; ** = p < .01; *** = p < .001
-1.43*** (0.36) -0.30 (0.38) 0.15 (0.48) 0.06 (0.05) .160
Displayed a personal interest in students. -1.11*** (0.28) -034 (0.30) -0.10 (0.37) -0.05 (0.04) .154 Conveyed the importance of learning -1.09*** (0.30) -0.05 (0.31) 0.09 (0.39) 0.03 (0.04) .160
26
I would recommend this instructor -0.94** (033) -0.44 (0.34) 0.12 (0.43) 0.00 (0.05) .096
I would recommend this course -1.05** (0.34) -0.39 (0.35) 0.19 (0.44) 0.02 (0.05) .106
Exams fairly tested course material
Encouraged students to think critically
-0.78** (0.28) -0.14 (0.29) 0.77* (0.37) 0.03 (0.04) .145 Realistic appraisal of students' background -1.07** (0.33) -0.57† (0.34) -0.39 (0.43) -0.07 (0.05) .130
-0.97*** (0.29) -0.42 (0.31) 0.04 (0.38) -0.02 (0.04) .118 Instructor's relationship with students was good -1.10 (0.32) -0.15 (0.33) 0.14 (0.41) 0.00 (0.04) .123
4.78
4.66
4.74
4.42
4.82
Material is presented clearly
4.49
4.58
4.92
4.86
4.31
4.72
Is enthusiastic about teaching.
4.74
4.62
Fall 2009 Large Section
4.78
4.37
4.62
Treats students respectfully
4.72
Spring 2009 Regular Section
27
4.42
4.76
4.56
4.78
Overall, instructor is an effective teacher
4.60
Deals fairly with different points of view
4.59
Grades are determined Questions or comments fairly handled effectively
4.40
Demonstrates thorough Maintains proper order in Encourages students to knowledge.* classroom.* reason for themselves
4.61
4.68
Graded materials returned soon.
4.37
* differences significant at the p < .05 level. N for Fall 2009 = 98; N for Spring 2009 = 50.
Attempts to motivate student interest
4.41
Instructional class time is Efforts made to clarify well used. difficult points.*
4.57
Lessons are well organized
4.52
Figure 4. Mean Responses to College Student Evaluation Survey.
Arbour, Karras, Lee TLC Paper
Arbour, Karras, Lee TLC Paper Appendix 1 Appendix 1—Assessment Exam Instructions: Please write the best response to each question on the answer sheet. Do not put your name on this sheet, or the answer sheet. 1. One reason the Constitution has endured for such a long time is a. that it has been amended over 50 times to address changes in society. b. that it is subject to constant revision by state legislatures. c. that its broad language allows for varying interpretations over time. d. that its detail covers practically any issue that may arise about the power of government. 2. In our Constitutional system of checks and balances, a. each branch has some authority over the others. b. the President can hire and fire members of the Supreme Court at will c. the Supreme Court automatically reviews all acts passed by Congress. d. all branches of government must be elected at the same time. 3.
The Great Compromise led to the a. legalization of slavery. b. creation of a bicameral Congress. c. creation of the Supreme Court. d. peaceful conclusion of Shays’ Rebellion.
4.
What is the Three-fifths Compromise? a. It determined that three out of every five slaves would be counted for purposes of representation and taxation. b. It determined the ratio between free states and slave states. c. It created a bicameral legislature. d. It declared that the states would pay three-fifths of the Revolutionary War debt and the federal government would pay the rest.
5. The powers of Congress under the Constitution are __________ those of Congress under the Articles of Confederation. a. greater than b. weaker than c. similar to d. There was no Congress under the Articles of Confederation. 6. The ability of the president to veto a bill passed by Congress is a good example of what principle of limited government? a. separation of powers b. federalism c. checks and balances d. civil liberties 28
Arbour, Karras, Lee TLC Paper Appendix 1 7. Before adopting a federal system in the Constitution, the U.S. had experienced _______ with the Articles of Confederation and _______ under British control. a. a unitary government; a confederation b. dual federalism; shared federalism c. shared federalism; dual federalism d. a confederation; a unitary government 8. Our federal system can best be defined as a. a system of government where member nations meet in a multinational conference. b. a system of government in which power is divided between a national government and lower levels of government. c. a system in which the power of the central government is funded through taxation of local government. d. the sharing of legislative powers between an upper and lower house. 9. Over the course of American history, the federal government has grown __________ compared to the states. a. stronger b. weaker c. less costly d. in tandem 10. What is the legislative role of standing committees in Congress? a. To pass final legislation, which is then presented to the President. b. To create legislation reflective of compromises between the House and Senate. c. To debate and edit proposed legislation and report proposed legislation to the floor. d. To establish the rules of debate on the floor of the United States Senate. 11. The Rules Committee in the House serves the following purpose: a. Sets rules for debate and amendment for bills on the House floor b. Decides which committee or committees bills will be referred to c. Negotiates compromises between the House version of a bill and the Senate version. d. Negotiates compromises between the Congress and the President. 12. How long is the term of office for a U.S. senator? a. two years b. six years c. eight years d. Senators have lifetime appointments 13. Pork-barrel legislation a. deals with specific projects and their location within a particular congressional district. b. deals with specific agricultural subsidies. c. funds efforts to increase the levels of America’s meat exports. d. grants a special privilege to a person named in the bill. 29
Arbour, Karras, Lee TLC Paper Appendix 1
14. A filibuster allows members of the Senate to a. refer a bill to multiple committees. b. avoid a conference committee. c. prevent a vote on a bill by speaking continuously on the floor. d. call into question any action of the executive branch. 15. What are implied powers of the President? a. Restrictions on actions of the President and the government to protect the liberty of citizens b. Powers the President has that are explicitly listed in the Constitution. c. Powers the President has by virtue of the responsibilities vested in the President by the Constitution d. Powers constitutionally vested in Congress, that Congress has granted to the President 16. Which of the following characterizes modern presidents? a. The public expects much from modern presidents, leading modern presidents to seek a larger role in policy-making b. Through less interaction with foreign nations, modern presidents are less able to influence Congress on policy c. Modern presidents typically have fewer staff members than presidents from earlier periods. d. Modern presidents are reluctant to use the media to transmit the president’s message to the public. 17. The president must share foreign policy powers with a. the Joint Chiefs of Staff. b. Congress. c. the states. d. the vice president. 18. What is required for Congress to override a presidential veto? a. a majority of both houses of Congress b. two-thirds of both houses of Congress c. three-fourths of both houses of Congress d. A president’s veto cannot be overridden unless it concerns the budget, in which case it requires a three-fourths majority of both houses. 19. The justices of the U.S. Supreme Court are a. nominated by the president and confirmed by the Senate. b. elected in nonpartisan balloting. c. nominated by the Senate. d. nominated by the president. The House and Senate play no role in the selection process.
30
Arbour, Karras, Lee TLC Paper Appendix 1 20. Appellate courts usually a. hear new testimony in a case if necessary b. correct errors of law and procedure from courts below. c. hear only criminal cases. d. hear only civil cases. 21. Party identification is: a. A voter’s opinion on the current president. b. Voting based on the performance in office of the current incumbent. c. A voter’s assessment of how close or far a party’s policy positions are from their own. d. A psychological attachment to a political party. 22. The strongest influence on an individual’s political views is: a. the values instilled to them as children by their parents and family. b. the economic interests of the company they work for c. the news media, especially television news d. a voter’s reaction to the image a candidate presents 23. Scholars find that individuals: a. Have detailed knowledge of the workings of their government b. Do not know all that much about their government. c. Have a surprising degree of intellectual consistency with regard to their opinions on government. d. Have logical and well-thought out opinions on government. 24. Party identification is important to political participation because: a. Those attached to political parties have less political knowledge because they blindly follow their party leaders. b. Strong partisans are less likely to vote than weak partisans or independents c. Party identification provides a shortcut for the evaluation of issues and candidates d. Individual’s political views are haphazard, with little coherent structure. 25. Retrospective voting is: a. The reliance on the issue positions of candidates as a basis for voting. b. The use of beliefs about what candidates will do once in office as a basis for voting. c. The use of evaluations of the performance of current elected officials as a basis for voting d. Voting based on one’s attachment or identification with a political party.
31
Arbour, Karras, Lee TLC Paper Appendix 2 Appendix 2—Ad Hoc Student Survey
Course Instructor Survey The objective of this survey is to aide in improving teaching effectiveness. Your responses provide valuable feedback to instructors about their course, and how to improve it. Your responses to the questions are extremely important. Instructions: For the following questions, answer on a 7 point scale. 7 means you completely agree, 1 means you completely disagree, and 4 means you are neutral. Disagree This course challenged me as a student. The class atmosphere was conducive to student learning I enjoyed taking this class. I would recommend this instructor to other students I would recommend this course to other students Instructor encouraged active learning Instructor's presentation style facilitated note taking Course developed appropriate writing strategies Exams fairly tested my understanding of course material Instructor encouraged students to think critically The instructor made the subject interesting The instructor was genuinely interested in teaching the course. Displayed a personal interest in students and their learning Instructor had a realistic appraisal of students' background and experience Instructor's relationship with students was good The instructor was accessible to students outside of class. Encouraged student-faculty interaction outside of class Instructor conveyed the importance of teaching and learning
Neutral
Agree
1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2
3 3 3 3 3 3 3 3 3 3 3 3 3 3
4 4 4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5 5 5
6 6 6 6 6 6 6 6 6 6 6 6 6 6
7 7 7 7 7 7 7 7 7 7 7 7 7 7
1 1 1 1
2 2 2 2
3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
7 7 7 7
How often do you attend this class? [ ] Rarely [ ] Somewhat [ ] Frequently Reason for taking this course? [ ] Major [ ] Minor [ ] Gen. Ed. Req. [ ] General Interest Where Did You Sit in the Classroom? [ ] Front [ ] Back What is your approximate GPA? [ ] 3.0 to 4.0 [ ] 2.0 to 2.99 [ ] 1.0 to 1.99 [ ] 0.0 to 0.99 Approximate number of hours each week spent outside of class studying for this course? ______
32
Arbour, Karras, Lee TLC Paper Appendix 2
Comments What do you think you learned from this class that will most remember?
What was the best part of this course?
What part of the class did you enjoy least? What suggestions do you have on how the professor could improve it?
What is your opinion of your instructor? How did he/she do their job?
How engaged or interested were you in the course materials (readings, lectures, assignments) throughout the semester?
How good or bad of a job did your instructor do in engaging you with the course material?
General comments on the course:
33
