Adv Physiol Educ 39: 63–66, 2015; doi:10.1152/advan.00064.2014.
How We Teach: Generalizable Education Research
Online quizzes promote inconsistent improvements on in-class test performance in introductory anatomy and physiology Gregory A. Brown,1 Matthew R. Bice,1 Brandon S. Shaw,2 and Ina Shaw2,3 1
Physical Activity and Wellness Laboratory, Department of Kinesiology and Sport Sciences, University of Nebraska, Kearney, Nebraska; 2Department of Sport and Movement Studies, University of Johannesburg, Auckland Park, Republic of South Africa; and 3Office of the Deputy Pro Vice-Chancellor: Research, Monash South Africa, Ruimsig, Republic of South Africa Submitted 21 May 2014; accepted in final form 30 January 2015
Brown GA, Bice MR, Shaw BS, Shaw I. Online quizzes promote inconsistent improvements on in-class test performance in introductory anatomy and physiology. Adv Physiol Educ 39: 63– 66, 2015; doi:10.1152/advan.00064.2014.—Review quizzes can provide students with feedback and assist in the preparation for in-class tests, but students often do not voluntarily use self-testing resources. The purpose of the present study was to evaluate if taking a mandatory online review quiz alters performance on subsequent in-class tests. During two semesters of a single-semester introductory anatomy and physiology course, students were required to complete brief online quizzes after each textbook chapter had been covered during lecture as well as the day before an in-class test. During the next two semesters, students were not required to take the online review quizzes. Overall scores on chapter specific in-class tests were higher (P ⬍ 0.05) during the semesters in which students took online review quizzes (82.9 ⫾ 14.3%) compared with when they did not (78.7 ⫾ 15.5%), but all in-class tests were not improved. Scores on comprehensive midterm examinations were higher (83.0 ⫾ 12.9% vs. 78.9 ⫾ 13.7%, P ⬍ 0.05) but not on final examinations (72.4 ⫾ 13.8% vs. 71.8 ⫾ 14.0%) between those with online review quizzes and those without, respectively. Overall scores on in-class tests and comprehensive examinations were higher (P ⬍ 0.05) during the semesters in which students took online review quizzes (83.4 ⫾ 16.8%) compared with when they did not (80.3 ⫾ 17.6%). These data suggest that an online review quiz taken the day before an in-class test increases performance on some in-class tests. However, online review quizzes taken after completion of each chapter do not consistently enhance performance on comprehensive examinations. formative assessment; summative assessment; undergraduate; exercise science; internet testing ONE COMPONENT of effective learning is to obtain new information through lecture, course notes, reading, or discussion and then recall the information by quizzing oneself or others (10, 17). Quizzes can be formative, in which the quiz is used to obtain information on concepts students may be struggling with to help direct course progression and students’ studying endeavors (6, 14), or quizzes can be summative, in which the quiz is used to help evaluate student performance in the course and determine grades (3). Formative quizzes as a study tool are an established component of an effective learning strategy (4, 9). To facilitate effective learning, many textbooks include self-testing quizzes, and these quizzes are frequently part of the textbook online ancillaries. Despite the well-known effectiveness of using practice quizzes as study aids, students often do not voluntarily
Address for reprint requests and other correspondence: G. A. Brown, Dept. of Kinesiology and Sport Sciences, Univ. of Nebraska at Kearney, 1410 W. 26th St., Kearney, NE 68849 (e-mail:
[email protected]).
take advantage of self-testing resources provided with their textbooks or other course materials (9, 17). Thus, making the review quizzes a required course component not only enhances student use of these learning resources but also appears to increase student learning (8, 9). Review quizzes have been effectively used to evaluate student knowledge and encourage student preparation for subsequent tests in a wide range of courses, including introductory biology (5, 13), exercise physiology (4), medical physiology (8), and social psychology (7). For example, Dobson (4) administered online formative quizzes in an undergraduate exercise physiology course during the weeks in which summative exams were not administered and reported that the overall score on summative examinations was improved through the use of online quizzes. Kibble (8) administered online quizzes in medical physiology 2– 4 wk before summative exams and observed improved performance on exams if students used online quizzes. Based on the notion that online quizzes should enhance student learning and performance on in-class tests, brief mandatory online review quizzes before higher point value in-class tests were included as part of a single semester introductory anatomy and physiology class (PE 310). However, based on student comments on both end-of-semester course evaluations and casual comments during informal conversations with students, it was discovered that mandatory online quizzes were very unpopular with students and so the use of mandatory online quizzes was discontinued. The purpose of the present study was to evaluate whether these mandatory online review quizzes were beneficial to students despite the lack of student enthusiasm for these learning tools. Another purpose was to evaluate if the use of online quizzes provided benefits on all subsequent in-class tests. METHODS
Overview. For two consecutive semesters, college students in a single semester introductory anatomy and physiology class (PE 310) were required to take online review quizzes consisting of 10 questions the day before taking larger in-class chapter tests consisting of 25– 40 questions. During the next two semesters, online quizzes were not used in the class but the same in-class testing format was. Student performance on chapter tests, midterm exams, and final exams for the semesters with and without online quizzes was compared to evaluate if online quizzes enhanced student performance. Course description. PE 310 is titled Introductory Human Physiology of Exercise and is taught as a single-semester human anatomy and physiology course required for students with majors in Physical Education, Exercise Science, and Athletic Training as well as for students with minors in Physical Education or Interscholastic Coaching or as an elective course for any student on campus. There are no prerequisites for enrollment in PE 310, but PE 310 is a prerequisite for
1043-4046/15 Copyright © 2015 The American Physiological Society
63
How We Teach: Generalizable Education Research 64
ONLINE QUIZZES INCREASE TEST SCORES
enrollment in subsequent courses in Anatomical Kinesiology and Biomechanics as well as Exercise Physiology, which are then perquisites for other courses, so PE 310 is the first in what is typically a four-semester sequence of exercise science classes. During the semesters used for this investigation, lectures for PE 310 met from 8:00 to 8:50 AM on Mondays, Wednesdays, and Fridays. Subjects. Online quizzes were used as part of the course during the spring and fall 2011 semesters, with a total of 111 students. During the next spring and fall 2012 semesters, 123 students were enrolled in PE 310 and online quizzes were not used as part of the class. Distributions of students by academic major as well as academic year classification are shown in Table 1 (academic minor information is not provided as part of typical enrollment information available to faculty members). Only data for students that completed the entire course were used in the present analysis. Online quizzes. Online quizzes consisted of eight multiple-choice and two fill in the blank questions worth 1 point each, had a 15-min time limit (1), were delivered through the course Blackboard interface (Blackboard, Washington, DC), and were available to students anywhere they could access the internet. Questions and answers for online quizzes were presented in random order to students in hopes that this would discourage cheating on the quizzes, but questions and answers were all presented at once to avoid difficulties that had been previously reported to campus Information Technology services when the questions appeared one at a time. Multiple-choice questions on all online quizzes consisted of a question and four possible answers drawn from the test bank provided with the course textbook (4a) or developed by the course instructor with ⬃15% of the questions coming from the textbook test bank. Fill in the blank questions for online quizzes were drawn from the key words listed at the beginning of each chapter of the textbook, and students were given the textbook definition of the word and then were required to fill in the correct word. Due to software limitations for the online quizzes, words had to be spelled fully correctly to receive credit with no partial credit possible for misspelled answers. Online quizzes were available beginning at noon immediately after completion of a textbook chapter in lecture and were available to students for 24 h (e.g., noon on Monday until noon on Tuesday). There were 13 online quizzes worth a total of ⬃11.6% of the overall course grade. In-class tests. In the next course lecture session after completion of the textbook chapter in lecture, students then took an in-class test (e.g., if the chapter was finished on Monday, the test was held on Wednesday). In-class tests consisted of 80% multiple-choice and 20% fill in the blank questions consisting of 25– 40 questions with a time limit of 30 –50 min depending on the number of questions on the test (⬃1.25 min/question). Questions for the in-class tests were developed
Table 1. Distribution of students by academic major and academic classification in an undergraduate introductory anatomy and physiology course during which mandatory 10-point online topic review quizzes were or were not used in the 24 h before an in-class test was taken OnLine Quizzes
Academic Major Athletic Training Exercise Science Physical Education Other Majors Academic Classification Freshman Sophomore Junior Senior Total
No Online Quizzes
Spring 2011
Fall 2011
Spring 2012
Fall 2012
14 20 12 10
1 36 6 12
12 23 14 10
4 34 17 9
13 18 15 10 56
2 23 22 8 55
8 22 22 7 59
1 22 36 5 64
in the same manner as for the online quizzes. For the in-class quizzes, minor spelling errors on the fill in the blank questions resulted in earning half credit for the answer. There were 13 in-class chapter specific tests worth a total of ⬃21.7% of the overall course grade for the class with online quizzes. In-class tests accounted for one-third of the course grade in the courses without online quizzes. Overall, for all class sections, chapter-specific testing was worth one-third of the overall course grade. Midterm and final exams. The midterm exam consisted of 30 multiple-choice and 20 fill in the blank questions worth 2 points each and was taken during the regular course meeting time at the end of the eighth week of class and had a 50-min time limit. The midterm exam was a cumulative exam covering the material on concepts in chemistry, cells, cellular bioenergetics, anatomic terminology and tissues, skeletal system, and the muscular system. The final exam consisted of 60 multiple-choice and 40 fill in the blank questions worth 2 points each and was taken during finals week and had a 2-h time limit. The final exam was cumulative, covering all topics in the course. Questions for the midterm and final exam were developed in the same manner as for the quizzes. Collectively, the midterm and final exams were worth one-third of the overall course grade. Calculations and statistics. Because of the point value differences between the online quizzes, in-class chapter tests, midterm exam, and final exam, individual student scores on all quizzes, tests, and exams were converted to percentages for statistical analysis. Data were then analyzed using two-way (factor 1: online quiz or not; factor 2: test topic) repeated-measures ANOVA with a P value of 0.05. Where significant effects were found, a Newman-Keuls post hoc test was performed. The relationship between scores for online quizzes and scores for in-class tests were evaluated using Pearson’s correlation. Data are presented as means ⫾ SD. RESULTS
In-class chapter tests. Overall scores on in-class chapter tests were higher (P ⬍ 0.05) during semesters in which mandatory online review quizzes were used versus those without online quizzes (Table 2). Whereas scores on every in-class chapter test were not different between classes with and without online quizzes, scores on in-class tests on cells, the heart and blood vessels, the nervous system, the endocrine system, and the digestive and urinary systems were higher (P ⬍ 0.05) in classes with online quizzes (Table 2). There was a significant (P ⬍ 0.05) but weak positive relationship between scores for online quizzes and in-class chapter tests (r2 ⫽ 0.30). Overall scores for online quizzes were lower (P ⬍ 0.05) than scores for in-class tests. Student participation in the online quizzes was very high (95–98% of students took each online quiz). Midterm and final exam. Midterm exam scores were higher (P ⬍ 0.05) during the semesters in which mandatory online review quizzes were used versus those without online quizzes (Table 2). There were no differences (P ⫽ 0.74) for final exam scores between semesters in which online review quizzes were used and those without (Table 2). If midterm and final exam scores were pooled to represent cumulative exams, there were no differences (P ⫽ 0.08) in combined scores between semesters in which online review quizzes were used (77.7 ⫾ 14.3%) and those without (75.4 ⫾ 14.2%). DISCUSSION
One of the main findings of the present investigation is that brief online review quizzes given the day before an in-class chapter test improved performance on some, but not all, of the
Advances in Physiology Education • doi:10.1152/advan.00064.2014 • http://advan.physiology.org
How We Teach: Generalizable Education Research ONLINE QUIZZES INCREASE TEST SCORES
65
Table 2. Topics and percent earned for online quizzes, in-class tests, and comprehensive midterm and final exams in an undergraduate introductory anatomy and physiology course during which mandatory 10-point online topic review quizzes were or were not used in the 24 h before an in-class test was taken Topic
Concepts in chemistry Cells Cellular bioenergetics Anatomic terminology and tissues Skeletal system Muscular system Blood Heart and blood vessels Lymphatic system Respiratory system Nervous system Endocrine system Digestive and urinary systems Overall scores for online quizzes or topic test Midterm exam Final exam
Online Quiz Scores
In-Class Test Scores When Online Quizzes Were Used
In-Class Test Scores When Online Quizzes Were Not Used
85.9 ⫾ 17.8 85.6 ⫾ 18.8 70.5 ⫾ 24.5 89.2 ⫾ 21.6 82.8 ⫾ 26.0 72.9 ⫾ 26.7 80.4 ⫾ 23.7 59.8 ⫾ 22.0 82.2 ⫾ 22.8 72.2 ⫾ 21.5 82.4 ⫾ 28.4 75.8 ⫾ 29.3 77.8 ⫾ 26.2 78.3 ⫾ 25.0
81.3 ⫾ 10.0 79.8 ⫾ 16.9* 86.6 ⫾ 10.3 88.1 ⫾ 9.0 87.5 ⫾ 8.0 82.4 ⫾ 7.6 80.1 ⫾ 17.2 85.5 ⫾ 14.9* 76.0 ⫾ 17.5 81.0 ⫾ 9.6 84.4 ⫾ 12.2* 75.7 ⫾ 15.7* 82.3 ⫾ 8.3* 82.9 ⫾ 14.3† 83.0 ⫾ 12.9* 72.4 ⫾ 13.8
79.7 ⫾ 15.2 72.1 ⫾ 17.8 83.1 ⫾ 8.8 87.5 ⫾ 16.5 84.7 ⫾ 8.0 83.2 ⫾ 8.6 77.5 ⫾ 14.0 73.6 ⫾ 17.0 75.0 ⫾ 15.2 79.5 ⫾ 12.6 75.8 ⫾ 16.9 69.5 ⫾ 17.0 80.1 ⫾ 16.0 78.7 ⫾ 15.5 78.9 ⫾ 15.3 71.8 ⫾ 14.0
Data are means ⫾ SD. *Interaction effect (P ⬍ 0.05) between online and no online quizzes; †main effect (P ⬍ 0.05) between online and no online quizzes.
in-class chapter tests. However, overall performance on the in-class chapter tests was improved by the use of online quizzes. Furthermore, online review quizzes given on the day before an in-class chapter test enhanced performance on the cumulative midterm examination taken 2 days after the previous in-class test, but the online review quizzes did not influence performance on the cumulative final examination taken 1 wk after the previous in-class test. These results can be used by instructors to help support the use of online review quizzes even when such quizzes are not popular with students. In the present investigation, the online quizzes were meant to help students prepare for an upcoming higher point value in-class test by helping them to identify weaknesses in their content specific knowledge, and, thus, the quizzes were somewhat formative in nature. Scores for the online quizzes in the present investigation were also used to evaluate student learning and determine student grades, so the online quizzes were also summative. Unfortunately, taking quizzes or tests and having assignments due add to students’ feelings of stress (16). Thus, it is not surprising that the online quizzes in the present investigation were unpopular with students as these online quizzes added yet one more item to students’ “to-do lists.” However, students often complain about the amount of studying and preparation they must do for a class and yet there typically is considerable discrepancy between the self-reported class preparation time and actual time spent in course preparation (2). So, although the online quizzes in the present investigation were not popular with the students, in agreement with Orr and Foster (12) and others (4, 8, 9), it can be concluded that online quizzing before in-class testing can effectively increase student performance on exams. Furthermore, having students engaged in the course by taking online quizzes can improve students’ perceived course satisfaction (6). Dobson (4) observed that student performance on online formative assessments was a valid predictor (r2 ⫽ 0.25) of subsequent performance on summative tests in an undergraduate exercise physiology course. In the present investigation, performance on the online quizzes was similarly related to
performance on tests (r2 ⫽ 0.30). It is worth noting that although scores on all in-class tests were not improved through the use of online review quizzes, none of the scores on in-class tests decreased when online quizzes were used. Although the present data do not provide insights into why some in-class test scores were improved while others were not, the present data along with previous observations (4, 8, 9, 12) indicate that online review quizzes are a useful part of an instructor’s inventory to educational tools. It is also interesting to note that scores on the final exam were not improved by the online quizzes, but scores on the midterm exam were higher. Many factors may influence performance on exams, including course attendance, effective studying, difficulty of the material, and student interest in the course material (15), none of which were measured in the present investigation. It is also possible that since the midterm was given during a regular course meeting with no break from routine and the final exam was given during finals week in which the regular class routine was disrupted and there were several days between the previous class meeting and the date of the final exam, the timing of the online quizzes relative to the examination may have influenced performance. However, O’Conner (11) observed that in-semester breaks do not impact exam performance. Thus, it is unclear why the online quizzes improved performance on the midterm exam but not the final in the present investigation. It is important to note that the online quizzes did not impair performance on the final exam, so while the benefit of the online quizzes may be limited to certain tests and exams, there is no drawback for students in terms of course performance. Kibble (8) noted that while the use of course points for online formative assessment increased students’ use of the quizzes, there was also an increase in students that apparently cheated on the online quizzes. Given the unsupervised nature of online quizzes, it is practically impossible to prevent all possible forms of cheating on the quizzes. In the present investigation, students were not instructed to use, or not use, their textbooks and course notes while taking the online quizzes. However, students were encouraged to be prepared for the
Advances in Physiology Education • doi:10.1152/advan.00064.2014 • http://advan.physiology.org
How We Teach: Generalizable Education Research 66
ONLINE QUIZZES INCREASE TEST SCORES
online quiz through the use of a 15-min time limit (1, 4) and randomization of the questions was incorporated to discourage students from cheating on the quizzes. The potential for students to use books, notes, and each other as resources when taking online quizzes represents a possible drawback for using online quizzes in summative assessment. Thus, instructors need to decide if such behavior would be ethical or not, and how to police such behavior. In conclusion, although mandatory brief online review quizzes before higher-value in-class tests may not be popular with students in an introductory anatomy and physiology class, these online quizzes enhance, or at least do not impair, student performance on the in-class tests. Online quizzes may also enhance performance on comprehensive examinations. DISCLOSURES No conflicts of interest, financial or otherwise, are declared by the author(s). AUTHOR CONTRIBUTIONS Author contributions: G.A.B., B.S.S., and I.S. conception and design of research; G.A.B. performed experiments; G.A.B., M.R.B., B.S.S., and I.S. analyzed data; G.A.B., M.R.B., B.S.S., and I.S. interpreted results of experiments; G.A.B. prepared figures; G.A.B., M.R.B., B.S.S., and I.S. drafted manuscript; G.A.B., M.R.B., B.S.S., and I.S. edited and revised manuscript; G.A.B., M.R.B., B.S.S., and I.S. approved final version of manuscript. REFERENCES 1. Brothen T, Wambach C. The value of time limits on internet quizzes. Teach Psych 31: 62– 64, 2004. 2. Campbell JM, Kinion ES. College students’ management of time. Imprint 39: 86 – 88, 1992. 3. Cilliers FJ, Schuwirth LW, van der Vleuten CP. A model of the pre-assessment learning effects of assessment is operational in an undergraduate clinical context. BMC Med Educ 12: 9, 2012.
4. Dobson JL. The use of formative online quizzes to enhance class preparation and scores on summative exams. Adv Physiol Educ 32: 297–302, 2008. 4a.Gunstream SE. Anatomy and Physiology with Integrated Study Guide (4th ed.). Columbus, OH: McGraw Hill Higher Education, 2009. 5. Haberyan KA. Do weekly quizzes improve student performance on general biology exams? Am Biol Teach 65: 110 –114, 2003. 6. Hope SA, Polwart A. Engagement with online pre-exam formative tests improves exam performance and feedback satisfaction. Biosci Educ 20: 37–52, 2012. 7. Johnson BC, Kiviniemi MT. The effect of online chapter quizzes on exam performance in an undergraduate social psychology course. Teach Psychol 36: 33–37, 2009. 8. Kibble J. Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance. Adv Physiol Educ 31: 253–260, 2007. 9. Kibble JD, Johnson TR, Khalil MK, Nelson LD, Riggs GH, Borrero JL, Payer AF. Insights gained from the analysis of performance and participation in online formative assessment. Teach Learn Med 23: 125– 129, 2011. 10. Marden NY, Ulman LG, Wilson FS, Velan GM. Online feedback assessments in physiology: effects on students’ learning experiences and outcomes. Adv Physiol Educ 37: 192–200, 2013. 11. O’Conner KJ. Should I give the exam before or after the break? Teach Psychol 41: 63– 65, 2014. 12. Orr R, Foster S. Increasing student success using online quizzing in introductory (majors) biology. Life Sci Educ 12: 509 –214, 2013. 13. Peteroy-Kelly M. Online pre-laboratory modules enhance introductory biology students’ preparedness and performance in the laboratory. J Microbiol Biol Educ 11: 5–13, 2010. 14. Rolfe I, McPherson J. Formative assessment: how am I doing? Lancet 345: 837– 839, 1995. 15. Smith G. How does student performance on formative assessments relate to learning assessed by exams? J College Sci Teach 36: 28 –34, 2007. 16. Staats S, Cosmar D, Kaffenberger J. Sources of happiness and stress for college students: a replication and comparison over 20 years. Psychol Rep 101: 685– 696, 2007. 17. Van Blerkom ML, Van Blerkom DL. Self-monitoring strategies used by developmental and non-developmental college students. J Coll Read Learn 34: 45– 60, 2004.
Advances in Physiology Education • doi:10.1152/advan.00064.2014 • http://advan.physiology.org
