Enhancing the Quality of Discussions in Undergraduate Online Courses

Enhancing the Quality of Discussions in Undergraduate Online Courses

Kerstin Hamann [email protected]

Philip H. Pollock [email protected]

Bruce M. Wilson [email protected] Department of Political Science University of Central Florida Orlando, FL 32816-1356 Phone: 407-823-2608

Paper prepared for delivery at the Annual Meeting of the American Political Science Association, Seattle, WA, September 1-4, 2011. This is work in progress and comments and suggestions are welcome. The authors wish to thank Nathalia Martins and Sabrina Stein for research assistance.

1

Enhancing the Quality of Discussions in Undergraduate Online Courses Abstract An established pedagogical literature suggests that student-student interaction in discussion settings promotes cognitive engagement, student satisfaction, and positive learning outcomes— and may enhance critical thinking skills. Furthermore, a growing body of research indicates that these positive processes and outcomes can be effectively recreated in the online environment. Yet we also know that the effectiveness of online discussions can be conditioned—even compromised—by such factors as group size, gender composition, and differences in the prior academic achievement of participants. In this study, we offer a preliminary look at the efficacy of a conditioning intervention: The role of student preceptors in promoting student interaction and elevating discussion quality. Using data from 26 discussion groups and 3 substantive discussions, we measure quantitative indicators of discussion activity under two quasiexperimental conditions: preceptor-as-student and preceptor-as-GTA. In the control condition, no student preceptor was present. In terms of quantitative indicators of discussion activity, we find no emergent differences between the preceptor intervention and the control.

2

Introduction: Student Discussion A long line of research underscores the benefits of student-oriented pedagogy, especially discussion, for achieving desired learning outcomes and student satisfaction (Prince 2004). Naturally, discussions promote communication skills (Dallimore, Hertenstein, and Platt 2008), but they also improve learning (Bender 2003; Davis and Hillman Murrell 1993; Huerta 2007), including cooperative learning and critical thinking (Garside 1996). In her comparison of student learning of material taught in lecture and presented in discussion, Garside (1996: 212) reports “significantly more learning with regard to higher-level items” in the discussion mode. In regard to the overall desiderata of classroom instruction—student interaction, critical thinking, problem solving, cooperation, mastery of content—the salutary effects of student discussion are legion (Beyer 1987; Nicol and Boyle 2003; McCarthy and Anderson 2000; Philips 2005; Gall and Gall 1990; Ellis et al. 2004). To be sure, discussion is no panacea for a roomful of bored, disengaged, academically challenged, or poorly prepared students (Laurillard 2002: 158-159). Yet it seems clear that discussions are beneficial for learning, and especially for developing critical thinking skills. These benefits notwithstanding, the main drawback with discussions is that students don’t participate in them. Although most estimates are anecdotal, they do resonate with typical instructor experience: As many as two-thirds of students never or rarely participate in class (Caspi, Chajut, and Saporta 2008: 718). This problem compounds as class size increases. As Nicol and Boyle (2003: 457) point out, instructors face important difficulties when they attempt to use “methods centered on dialogue and discussion” in large classes. One way to overcome this limitation is to create small groups to generate cooperative team learning (see, e.g., Occhipinti 2003). It is an open question, however, whether requiring student-student cooperative learning

3

can bring about the same positive results that are obtained through voluntary student participation (Rabow et al. 1994). In online instruction, however, discussion groups may be the only way to recreate the immediacy and dynamism of the face-to-face environment. In this context, numerous studies have identified the potential and actual benefits of engaging students in virtual discussions. DeLoach and Greenlaw (2005), for example, report the beneficial effects of online discussions on student critical thinking—a conclusion echoed by Arbaugh (2000), Ellis et al. (2004), Meyer (2003), and Williams and Lahman (2009). Similarly, Thomas (2002: 361) observes that “online discussion forums promoted high levels of cognitive engagement and critical thinking.” Even at the more pedestrian level of factual and conceptual recall, Johnson, Howell, and Code (2005) report distinct comparative advantages of online modalities. From higher-order learning to lower-level skills, the literature has established the benefits of engaging students in discussions in classes conducted partially or entirely online (Clawson, Deen, and Oxley 2005; Hamann, Pollock, and Wilson 2009; Krentler and Willis-Flurry 2005; Pollock, Hamann, and Wilson 2005; Tiene 2000; Wilson, Hamann, and Pollock 2007; Botsch and Botsch 2001). Just as in real-time face-to-face settings, however, some online discussions are better than others. Size matters. Although small-group discussions are more likely to engage a higher number of students than discussions in large classes (Lowry et al. 2006; Pollock, Hamann, and Wilson 2011), which are often driven by just a few individual students (see Occhipinti 2003: 69), “larger” small groups (10 participants) do not perform as well as smaller ones (about 5 participants), which foster more give-and-take (Fay, Garrod, and Carletta 2000; Mayo, Sharma, and Muller 2009). It appears that, when students are placed into smaller discussion groups to make discussions less repetitive, higher-order critical thinking is present across all groups

4

(Wickersham and Dooley 2006). Students’ academic histories matter, too, although perhaps not in the ways one might predict. Lower-GPA students, who are more inclined toward “passive” learning—reading discussion postings by other group members—realize greater overall improvement in learning outcomes than do their higher-GPA cohorts, who participate more actively but return fewer gains across an array of indicators (Hamann, Pollock, and Wilson 2009; Wilson, Pollock, and Hamann 2007). The gender composition of the group makes a difference as well. Left to their own devices, gender-homogeneous groups revert to non-interactive rhetorical styles—independent statements, anemic threads—while gender-balanced groups produce much more interaction between participants (Pollock, Hamann, and Wilson 2005). In theory, several of these discussion-enhancing conditions, such as group size and gender composition, can be benignly manipulated by the instructor. In practice, however, instructors often must compromise these interventions. In very large online classes (of 250 students or more), instructors may need to construct larger-than-desirable discussion groups or settle for groups with a suboptimal gender composition or GPA mix. Scarce resources, such as time or GTA assistance, might limit the degree to which the instructor can guide the discussion topic, modulate flow, head off tangents, defuse monopolization, and correct multi-thread drift. The current study looks at the potential benefit of a low-resource resolution for these issues: student preceptors.

Student Preceptors If online discussions provide a useful foil to gear students towards critical thinking, what can instructors to do maximize this potential effect? DeLoach and Greenlaw (2005) conclude that online discussions have “spillover” effects for critical thinking; that is, if initial postings are of

5

high quality and reflect critical thinking and substantive points, other students will follow suit, model their postings after the high-quality one, and the overall quality of the discussion improves. The question, then, is how to induce higher-order thinking postings that can serve as a model for future postings. This is particularly difficult in very large classes where the professor is not necessarily in a position to constantly monitor and comment on all postings. To assist with student learning in this case, we have turned to the use of an undergraduate preceptor. Undergraduate preceptors have become a widely used resource to enhance the educational experience of other undergraduate students. Nursing programs, for instance, frequently use preceptors “as a valuable link to assist students to make the connection between nursing theory and its application to patient care.” (Blum 2006: 7). Even outside of the medical education fields, preceptors’ talents are increasingly harnessed in non-practicum, traditional classroom settings to improve student learning. For example, in the Political Science Department at the University of Chicago, undergraduate preceptors are used to lead undergraduate colloquia and guide students undertaking research projects; the Chemistry Department at the University of Arizona allows students to earn course credit as a preceptor where they “function as student mentors and guides and as instructional assistants” for faculty members (University of Arizona 2007). The impact of undergraduate preceptors has been recently examined in a study of over 90 introductory-level courses at the University of Hartford (Black and Voelker 2008). This study finds that undergraduate preceptors “were effective role models for good student habits” and that students in courses where preceptors were present experienced a “significantly greater engagement” than students in courses without preceptors.

6

Based on these insights of the benefits of preceptors, our goal was to evaluate the impact of inserting undergraduate preceptors into the discussion groups of a large online version of the GEP Introduction to American National Government class as a role model and discussion facilitator. We employed a single preceptor, a highly motivated undergraduate who had previously taken the online version of the course and performed exceptionally well in all aspects of that class. The preceptor was assigned to half of the course’s discussion groups; the groups he monitored were split in half – in 50 percent, he appeared as a teaching assistant and in the other 50 percent as a regular student where the rest of the students were not aware of his role as a preceptor. His charge was to post contributions to the discussion boards to try and set a high standard for the group to follow. He was to post the same initial comments in each of the discussion groups he was assigned to and to make that post well before the initial deadline, but not necessarily make the first posting to avoid students waiting for his post or using his posting as a foil for their own. His subsequent posts were to respond to the comments of the students in those groups while gently reminding the students to keep the discussions focused and reasoned.

The Study Our study assesses the use of student preceptors in enhancing student discussions in online discussion groups. Throughout the class students were required to read the textbook, take short quizzes on the subject matter of the chapters, engage in a discussion to apply the concepts covered in the textbook chapters to contemporary issues, and finally write short papers. By way of example, in the civil rights and civil liberties section of the course, students read the relevant two chapters, took quizzes, and then discussed the contemporary issue of same sex marriage. Students were given access to two relevant US Supreme Court cases, Romer v. Evans (1996) and

7

Lawrence v. Texas (2003), as well as short newspaper reports on the California Proposition 8 battle and a Newsweek article by Ted Olsen (the conservative constitutional lawyer who successfully argued the Bush vs. Gore case in 2000) on the right of same-sex couples to marry. Students were instructed to discuss the following questions: What are the constitutional arguments that the proponents and opponents of same-sex marriage employ? In what way are the proponents of same-sex marriage employing the language of Civil Rights? Once a group has achieved rights (such as the right to same-sex marriage in California), do majorities then have the power to take away those rights through a simple majority vote? In their discussion, students were asked to refer to the constitutional arguments that were made in this case by both sides. They could refer to the textbook for background information as well as to the websites provided, or additional information if they chose to. Students were reminded that they should avoid statements that began with “I believe that...”. They had to present evidence to support their assertions in their postings. They were also reminded that it did not matter what their personal opinion was on this issue but instead to think about this question in the context of civil rights and civil liberties as constitutional provisions. Each module’s discussion has two deadlines; one for the initial substantive posting, and a second deadline for the subsequent three postings that need to include at least one response to their peers’ postings. 1 The module on healthcare had the students discuss the complexity of the legislative path of President Obama’s health care reform bill; the module on media asked the students to discuss the role of political blogs. Our data were gathered from students enrolled in a large (330 students) online section of American National Government taught by one of the authors during Fall semester of 2010. Each 1

In the final days of the module, students wrote a 550-650 word essay summarizing the main arguments made by both sides of the same sex marriage issue, with conclusions based on the evidence to support one side or the other. Papers were graded not on student opinions, but on the cohesiveness of their arguments and the evidence presented to support their argument. Students were reminded that their paper’s conclusions should be drawn on the constitutionality of the issue and did not need to reflect their own personal values or opinion.

8

student was assigned to one of 26 groups (of 11-13 students) and remained in that group throughout the semester. In 7 groups, the student preceptor represented himself as a fellow student (the student condition), and in 7 he represented himself as the class GTA (the GTA condition). In the remaining 12 groups, the controls, he was not present (the not present condition). Here we report basic quantitative indicators of student discussion activity—length of longest thread, number of discussion threads, thread length, and posting length—arising from three discussion topics: civil rights, the healthcare debate, and the role of online media, which were spaced out throughout the semester. Did the student preceptor have any discernible effects? The short answer: No.

Results Table 1 reports mean values for each quantitative indicator for each preceptor condition, separately for each discussion topic. For example, the entries beneath “length of longest thread” for the civil rights discussion show the average length of the longest thread for the 12 groups in the not present condition, the 7 groups in the student condition, and the 7 in the GTA condition. The “total” row reports the mean for all groups combined. Figure 1 graphically displays the Table 1 data for posting length, probably our most valid quantitative indicator of discussion quality. The data support two general observations. First, preceptor status made no measureable difference in length of longest thread, thread length, or posting length. For each discussion topic, the “total” row is a fair statistical guide to any given entry in the column. For example, all groups averaged 4.3 postings per thread in the civil rights discussion. The means for each preceptor status—4.4 for not present, 3.9 for student, and 4.7 for GTA—do not differ significantly from

9

each other or from the overall mean. Indeed, with the possible exception of differences in the mean length of the longest thread, there appear to be no differences in student behavior across discussion topics. (The civil rights discussion generated somewhat more lengthy longest threads than did the other discussions.) Second, preceptor-as-GTA was associated with a significantly greater number of threads in all three discussions. Statistically, the effect ranges from the noteworthy—on average, two more threads in the civil rights and media discussions—to the robust—three more threads, on average, in the discussion on health care. In view of the fact that a high thread count is not necessarily a desirable pedagogical goal, this perverse outcome requires some explanation. It could be that, in groups where the preceptor appeared as GTA, students started new threads to avoid engaging in a debate with an authority figure. Instead, students preferred to “talk” to each other rather than with the GTA. A cursory examination of the discussion threads confirms that in most cases the GTA posting elicited no response. This could explain the overall higher number of threads when the preceptor was present as a GTA.

Discussion and Conclusion The literature has firmly established that students learn more, or better, when they engage in peer discussion. This is true for face-to-face classes but also for online classes. In very large classes, however, it is often difficult for the professor to monitor all discussion postings. Thus, we were interested in finding out how the quality of online discussions can be assured; in other words, how critical thinking and higher-order thinking can be stimulated in online discussion groups. The literature suggests that undergraduate preceptors can be useful in ensuring student learning outcomes and critical thinking. In online classes, it has been found that early postings that were

10

of high quality—that is, reflected higher-order thinking—served as models for succeeding postings, thus improving overall discussion quality. We therefore decided to see to what extent the intervention of a preceptor was able to affect the discussion in a large, online introductory American Government class. The preceptor posted “model” postings early on in the discussion, with the intent of creating a “spill-over” effect for future postings. In this paper, we have analyzed the relationships between preceptor status and quantitative indicators: number of threads, length of longest thread, thread length, and posting length. All in all, the presence or absence of the preceptor—whether in the role of a peer undergraduate or in the role of a GTA—did not affect these indicators. Our only statistically meaningful finding is that students, by and large, shied away from engaging in a discussion with the GTA, an averse behavior that probably explains the higher number of threads in these groups. Yet, while the preceptor might not have had any impact on these quantitative indicators, these findings are limited. An important limitation of our analysis is that it only considers aggregate group behavior and does not take into account the attributes and behavior of individual students. For example, do students with lower GPAs react differently to the model postings by the preceptor than do those with higher academic credentials? Are first-year students affected differently from seniors (who perhaps in their last semester realize they are still lacking the required GEP course)? Do political science majors respond differently than non-majors? More broadly, quantitative indicators are one way to assess the impact of preceptors, but they do not necessarily gauge the impact of the preceptor on critical thinking. To analyze the effect of model postings on discussion quality, we will as a next step conduct a content analysis of the postings. This will give us better leverage on the question of whether the effect of a preceptor lies in the

11

qualitative realm, an area our current data leave unexplored. In the end, the goal of the study is to improve student learning in online discussion groups through the use of preceptors.

12

Figure 1. Mean Posting Length, by Student Preceptor Status and Discussion Topic

13

Table 1. Preceptor Status and Student Discussion Indicators, by Discussion Topic Discussion topic

Preceptor status

Length of longest thread

Thread length

Number of threads

Posting length

Civil rights

Not present Student GTA Total

9.6 10.7 10.0 10.0

4.4 3.9 4.7 4.3

8.8 10.1 10.9* 9.7

11.3 11.7 11.3 11.4

Healthcare


7.3 7.0 6.9 7.1

4.0 3.7 3.6 3.8

6.6 7.6 9.6** 7.7

9.4 10.3 9.9 9.8

Media


7.3 8.3 8.6 7.9

4.1 4.1 4.4 4.2

8.1 9.3 10.4* 9.0

11.9 11.2 10.3 11.3

Total


8.1 8.7 8.5 8.3

4.2 3.9 4.2 4.1

7.8 9.0* 10.3** 8.8

10.9 11.1 10.5 10.8

Note: For Not present condition, N=12; for Student condition, N=7; for GTA condition, N=7. *Significantly greater than Not present condition @ .05. **Significantly greater than Not present condition @ .01.

14

References Arbaugh, J.B. 2000. “Virtual Classroom versus Physical Classroom: An Exploratory Study of Class Discussion Patterns and Student Learning in an Asynchronous Internet-Based MBA Course.” Journal of Management Education 24(2): 213-233. Bender, Tisha. 2003. Discussion-Based Online Teaching to Enhance Student-Learning. Sterling, VA: Stylus. Black, Katherine A. and Joseph C. Voelker. 2008. “The Role of Preceptors in First-Year Student Engagement in Introductory Courses.” Journal of The First-Year Experience & Students in Transition 20(2):25-44. Bligh, Donald A. 2000. What’s the Use of Lectures? San Francisco: Jossey-Bass. Blum, Cynthia Ann. 2006. “Development of an undergraduate Preceptor model of Practice Education for Nursing Students Using Participatory Action Research.” PhD dissertation, Florida Atlantic University. Botsch, Carol S. and Botsch, Robert E. 2001. “Audiences and Outcomes in Online and Traditional American Government Classes: A Comparative Two-Year Case Study.” PS: Political Science & Politics 34(1): 135-141. Brookfield, Stephen D. and Stephen Preskill. 2005. Discussion as a Way of Teaching: Tools and Techniques for Democratic Classrooms. 2nd ed. San Francisco: Joessey-Bass. Buckley, Geoffrey L.; Nancy R. Bain, April M. Luginbuhl, and Mary L. Dyer. 2004. “Adding an ‘Active Learning’ Component to a Large Lecture Course.” Journal of Geography 103(6): 231237. Burgan, Mary. 2006. “In Defense of Lecturing.” Change 38(6): 30-34. Caspi, Avner; Eran Chajut, and Kelly Saporta. 2008. “Participation in Class and in Online Discussions: Gender Differences.” Computers & Education 50: 718-724. Chang, Ni. 2006. “E-Discussions as a Complement to Traditional Instruction: Did the Students Like Online Communication and Why?” Journal of Early Childhood Teacher Education 27: 249-264. Clawson, Rosalee A; Rebecca E. Deen, and Zoe M. Oxley. 2002. “Online Discussion Across Three Universities: Student Participation and Pedagogy.” PS: Political Science & Politics 35(4): 713-718. Cohen, Elizabeth G. 1994. “Restructuring the Classroom: Conditions for Productive Small Groups.” Review of Educational Research 64(1): 1-35.

15

Dallimore, Elise J.; Julie H. Hertenstein, and Marjorie B. Platt. 2008. “Using Discussion Pedagogy to Enhance Oral and Written Communication Skills.” College Teaching 56(3): 163172. Davis, Todd M. and Patricia Hillman Murrell. 1993. Turning Teaching into Learning: The Role of Student Responsibility in the Collegiate Experience. ASHE-ERIC Higher Education Report No. 8. Washington, D.C.: The George Washington University, School of Education and Human Development. DeLoach, Stephen B. and Steven A. Greenlaw. 2005. “Do Electronic Discussions Create Critical Thinking Spillovers?” Contemporary Economic Policy 23(1): 149-163. Ellis, Robert A., Calvo, Rafael, Levy, David, & Tan, Kelvin. 2004. “Learning through Discussions.” Higher Education Research and Development 23(1): 73-93. Fay, Nicolas, Simon Garrod, and Jean Carletta. 2000. “Group Discussion as Interactive Dialogue or as Serial Monologue: The Influence of Group Size.” Psychological Science 11(6): 481-486. Gall, Joyce P. and Meredith D. Gall. 1990. “Outcomes of the Discussion Method.” In William M. Wilen (ed.), Teaching and Learning Through Discussion: The Theory, Research and Practice of the Discussion Method. Springfield, IL: Charles C. Thomas, pp. 25-44. Garside, Colleen. 1996. “Look Who’s Talking: A Comparison of Lecture and Group Discussion Teaching Strategies in Developing Critical Thinking Skills.” Communication Education 45(3): 212-227. Pollock, Philip; Kerstin Hamann, and Bruce Wilson .Forthcoming 2011. “Learning Through Discussions: Comparing the Benefits of Small-Group and Large-Class Settings.” Journal of Political Science Education 7(1): 1-17. Hamann, Kerstin; Philip H. Pollock, and Bruce M. Wilson. 2009. “Learning from ‘Listening’ to Peers in Online Political Science Classes.” Journal of Political Science Education 5(1): 1-11. Ho, Shirley S. and Douglas M. McLeod. 2008. “Social-Psychological Influences on Opinion Expression in Face-to-Face and Computer-Mediated Communication.” Communication Research 35(2): 190-207. Huerta, Juan Carlos. 2007. “Getting Active in the Large Lecture.” Journal of Political Science Education 3(3): 237-249. Johnson, Genevieve M; Andrew J. Howell, and Jillianne R. Code. 2005. “Online Discussion and College Student Learning: Toward a Model of Influence.” Technology, Pedagogy and Education 14(1): 61-76.

16

Krentler, Kathleen A and Laura A. Willis-Flurry. 2005. “Does Technology Enhance Actual Student Learning? The Case of Online Discussion Boards.” Journal of Education for Business (July/August): 316-321. Laurillard, Diana. 2002. Rethinking University Teaching. 2nd ed. A Conversational Framework for the Effective Use of Learning Technologies. London: Routledge. Lowry, Paul Benjamin; Tom L. Roberts, Nicholas C. Romano, Jr., Paul D. Cheney, Ross T. Hightower. 2006. “The Impact of Group Size and Social Presence on Small-Group Communication Does

Computer-Mediated Communication Make a Difference?” Small Group Research 37(6): 631661. Marks, Michael P. 2008. “Fostering Scholarly Discussion and Critical Thinking in the Political Science Classroom.” Journal of Political Science Education 4: 205-224. Mayo, Ashleigh; Manjula D. Sharma, and Derek A. Muller. 2009. “Qualitative Differences Between Learning Environments Using Videos in Small Groups and Whole Class Discussions: A Preliminary Study in Physics.” Research in Science Education 39(4): 477-493. McCarthy, J. Patrick and Liam Anderson. 2000. “Active Learning Techniques Versus Traditional Teaching Styles: Two Experiments from History and Political Science.” Innovative Higher Education 24(4): 279-294. Meyer, Katrina A. 2003. “Face-To-Face Versus Threaded Discussions: The Role of Time and Higher Order Thinking.” Journal of Asynchronous Learning Networks 7: 55-65. NSSE (National Survey of Student Engagement). 2008. Promoting Engagement for All Students: The Imperative to Look Within. Bloomington, IN: National Survey of Student Engagement. http://nsse.iub.edu/NSSE_2008_Results/docs/withhold/NSSE2008_Results_revised_11-142008.pdf Nicol, David J. and Boyle, James T. 2003. “Peer Instruction versus Class-Wide Discussion in Large Classes: a comparison of two interaction methods in the wired classroom.” Studies in Higher Education 28: 457-473. Occhipinti, John D. 2003. “Active and Accountable: Teaching Comparative Politics Using Cooperative Team Learning.” PS: Political Science & Politics 36 (1): 69-74. Phillips, Rob. 2005. “Challenging the Primacy of Lectures: The Dissonance Between Theory and Practice in University Teaching.” Journal of University Teaching and Learning Practice 2(1): 112. Pollock, Philip H., Kerstin Hamann and Bruce M. Wilson. 2005. “Teaching and Learning Online: Assessing the Effects of Gender Context on Active Learning.” Journal of Political Science Education 1(1):1-16.

17

Prince, Michael. 2004. “Does Active Learning Work? A Review of the Research.” Journal of Engineering Education 93(3): 223-232. Rabow, Jerome; Michelle A. Charness, Johanna Kipperman, Susan Radcliffe-Vasile. 1994. William Fawcett Hill’s Learning Through Discussion. 3rd ed. Thousand Oaks: Sage. Thomas, M. J. W. 2002. “Learning with Incoherent Structures: The Space of Online Discussion Forums.” Journal of Computer Assisted Learning 18(3): 351-366. Tiene, Drew. 2004. “Online Discussions: A Survey of Advantages and Disadvantages Compared to Face-to-Face Discussions.” Journal of Educational Multimedia and Hypermedia 9: 371. U.S. Department of Education. 2009. Evaluation of Evidence-Based Practices in Online Learning. A Meta-Analysis and Review of Online Learning Studies. Washington, D.C. Available at http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf. Wickersham, Leah E. and Kim E. Dooley. 2006. “A Content Analysis of Critical Thinking Skills as an Indicator of Quality of Online Discussion in Virtual Learning Environments.” The Quarterly Review of Distance Education 7(2): 185-193. Wilen, William M. 1990. “Forms and Phases of Discussion.” In William M. Wilen (ed.), Teaching and Learning Through Discussion: The Theory, Research and Practice of the Discussion Method. Springfield, IL: Charles C. Thomas, pp. 3-24. Wilson, Bruce M., Philip H. Pollock and Kerstin Hamann. 2007. “Does Active Learning Enhance Learner Outcomes? Evidence from Discussion Participation in Online Classes.” Journal of Political Science Education 3(2): 131-142. Wilson, Bruce M., Philip H. Pollock; and Kerstin Hamann. 2006. “Partial Online Instruction and Gender-Based Differences in Learning: A Quasi-Experimental Study of American Government.” PS: Political Science & Politics 39(2): 335-339. University of Arizona – Department Of Chemistry Undergraduate Preceptorship. 2007.Guidelines & Application. http://www.chem.arizona.edu/ug_in_chem/forms/Precep_Cntrct_APR07.pdf Windschitl, Mark. 1999. “Using Small-Group Discussions in Science Lectures.” College Teaching 47(1): 23-28.