Indian J Pediatr (August 2011) 78(8):983–986 DOI 10.1007/s12098-010-0361-1
ORIGINAL ARTICLE
Improving Learning during Pediatric Lectures with Multiple Choice Questions Tiroumourougane V. Serane & Thirunavukkarasu Arun Babu & Roshni Menon & Vijaya Devagaran & Bhuvaneswari Kothendaraman
Received: 13 September 2010 / Accepted: 31 December 2010 / Published online: 14 February 2011 # Dr. K C Chaudhuri Foundation 2011
Abstract Objective To evaluate the role of multiple choice questions (MCQs) administered at the conclusion of the lecture in improving medical student learning. Methods This prospective interventional study was conducted in a Medical College. In ten randomly chosen pediatric lectures delivered to prefinal year medical students, MCQs were administered at the conclusion of the lecture and the answers were discussed with the students. After the series of lectures, students were evaluated by hundred ‘single best response’ MCQs with half from the lectures with MCQs selfassessment (MSA group) and the rest were from the other set of lectures (WMSA group). Scores from both groups were compared using paired ‘t’ test. Results When the scores from MSA and WMSA groups were compared for the study population, no significant difference was noted. A significant difference (p=0.003) was found between the groups, when the mean (± SD) T. V. Serane (*) : T. Arun Babu : V. Devagaran Department of Pediatrics, Sri Lakshmi Narayana Institute of Medical Sciences (SLIMS), Osudu, Agaram Village, Villianur Commune, Kodapakkam Post, Pondicherry 605 502, India e-mail:
[email protected] R. Menon Department of Dermatology, Sri Lakshmi Narayana Institute of Medical Sciences (SLIMS), Osudu, Agaram Village, Pondicherry 605 502, India B. Kothendaraman Department of General Medicine, Jawaharlal Institute of Post Graduate Medical Education and Research (JIPMER), Pondicherry 605006, India
scores of the men (25.2±4.8 in the MSA group and 28.6±7.3 in the WMSA group) were compared. No difference was seen among women, whose mean (± SD) scores in the MSA and the WMSA group were 26.6±6.9 and 26.3±6.9 respectively. When the scores of the first half of the session was analyzed, there was significant difference between the two groups among men (p=0.0001) implying that lectures with MCQs self-assessment improved the long-term retention. Conclusions MCQs’ based self-assessment at the conclusion of the lecture does not improve the learning among prefinal year medical students. However, this study has shown the usefulness of MCQs based self-assessment among male medical students. Keywords Lecture . Multiple choice questions . Education
Introduction Among all the teaching methods, lecturing continues to be one of the major modes of delivering information, knowledge and concepts to a large group of medical students [1, 2]. It is an effective tool for efficient and organized delivery of a large volume of information, when used relevantly. However, successful utilization of this educational technique is not as simple as it might appear. Lectures are delivered to students with varied individual ability to assimilate and recall information. Students can recall up to 62% of the information soon after a lecture, which drops down to 24% when retested after 8 weeks [3]. Lack of active participation and opportunity for self-assessment are some of the other disadvantages of lectures. Promoting active participation by providing opportunities for self-assessment and reflection
984
can encourage students to become better learners and increase their motivation. The objective of this study is to evaluate the role of multiple choice questions administered at the conclusion of the lecture in improving medical student learning
Material and Methods This prospective study was carried out as part of the Pediatric teaching programme of prefinal year undergraduate students of the authors’ medical college over 6 months period from June 2009 through December 2009. In the study period, a series of 20 lectures were delivered by three different teachers. Out of these, ten lectures were randomly selected for selfassessment with multiple choice questions (MCQs) at the conclusion of the lecture. Students were informed about the MCQs self-assessment only at the beginning of the session. A set of ten MCQs from the discussed topic were administered and the answers were discussed with the students, who selfcorrected their responses. The learners were encouraged to discuss and clarify any doubts about the MCQs. After the series of lectures, formative evaluation was done using 100 ‘single best response’ MCQs. Half of the questions were from the lectures with MCQs self-assessment and the rest were from the other set of lectures. MCQs and suitable distracters were framed with the following criteria: (1) Each question had a choice of four answers with only one correct answer (2) Options with partially correct answers or multiple answers were avoided (3) The distractors listed were plausible and were related to the correct answer. Prevalidation of the MCQs were done by three independent teachers, who analyzed the relevance of the questions with respect to the learning outcome, clarity and plausibility. MCQs that were used for self-assessment were not included in the final formative evaluation. The duration of the test was 75 min. Answers were marked on a coding sheet and there was no negative marking. The Institute’s ethics committee had approved the study design. Informed consent was obtained from students at the start of the term and they had the option of withdrawing at any point during the study.
Indian J Pediatr (August 2011) 78(8):983–986
Results Out of 100 and ten students, 73 attended all the lectures and formed the study population. Forty eight of them were women and the rest were men. The mean (± SD) score was 53.2±12.2. When the scores from MSA and WMSA groups were compared, no significant difference was noted. The mean (± SD) scores of the men was 25.2±4.8 in the MSA group and 28.6±7.3 in the WMSA groups and a significant difference (p=0.003) was found between the groups. To find out if the significant result among the men was because of the lecture modification technique under study, standard error of the means for the lectures without MCQs between men and women was calculated. The standard error of the means was not significant, implying the difference was due to the method under discussion. No difference was seen among women, whose mean (± SD) scores in the MSA and the WMSA group was 26.6±6.9 and 26.3±6.9, respectively (Table 1). To study the effect of time on this lecture modification technique, scores from the lectures taken during the first and the second half of the study period were analyzed independently. During the first half, the average (± SD) scores were 13.6 ± 3.0 and 14.9 ± 4.8 and there was significant difference between scores obtained by the two groups (p=0.018). When subgroup analysis was done, the effect was seen only in the men (Mean (± SD) 12.8±16.3 in the MSA group and 16.3±5.3 the WMSA group, p=0.0001) and not in women. When the scores from the second half were compared, there was no difference suggesting that lectures with MCQs self-assessment improved the long-term retention. The effect of this lecture modification technique among the high and low achievers by using 50th centile as the separator was also studied. Among the students who ranked above 50th percentile, there was no significant difference between the marks obtained in the MSA and the WMSA group. However, subgroup analysis showed a significant effect for the lecture topics covered during the first 3 months of the study (p=0.006).
Discussion Statistical Analysis Data from the final evaluation was entered in a database for analysis using SPSS software. Data from students who did not attend all the lectures was excluded from the analysis. Scores obtained on topics with MCQs self-assessment (MSA group) were compared with the ones without multiple choice questions (WMSA group) using Paired ‘t’ test. The relationship of gender, time, and "achievement" level of the learner was also studied.
Medical teachers often deliver complex information to medical students, who are adult learners in a manner which does not promote active learning [4]. Efforts towards improving the quality of medical teaching have concentrated on improving existing teaching techniques and promoting active learning of the students. Most published articles are descriptive accounts rather than rigorous research, though this anomaly is being addressed by different authors recently [5, 6].
Indian J Pediatr (August 2011) 78(8):983–986
985
Table 1 Mean, Standard Deviation and p value of the scores Number (N)
Total Men Women First Half
Second Half
Top 50% of students
Bottom 50% of students
Mean (± Standard Deviation)
P value
MSA group
WMSA group
Total Men
73 25 48 73 25
26.1±6.2 25.2±4.8 26.6±6.9 13.6±3.0 12.8±16.3
27.1±7.1 28.6±7.3 26.3±6.9 14.9±4.8 16.3±5.3
Not Significant 0.003 Not Significant 0.018 0.0001
Women Total Men Women Total First Half Second Half Total First Half Second Half
48 73 25 48 37 37 37 36 36 36
13.6±3.6 12.9±3.0 12.4±2.3 13.2±3.4 30.4±4.8 15.5±2.5 15.0±2.4 21.6±4.1 10.9±2.0 10.7±1.9
13.5±4.7 12.6±3.4 12.2±3.3 12.7±3.7 32.2±4.9 17.7±3.7 14.5±2.9 21.6±4.4 10.9±3.8 10.5±2.5
Not Significant Not Significant Not Significant Not Significant Not Significant 0.006 Not Significant Not Significant Not Significant Not Significant
(MSA group—‘Lectures with MCQs self-assessment’ group, WMSA group—‘Lectures without MCQs self-assessment’ group)
Lectures in medical education are often used as a way of passive transfer of knowledge rather than as a means of promoting active learning in students. This form of formal didactic teaching has many limitations [7]. Some of the methods tried for improving the lectures were behavior modification of the lecturer, brainstorming, buzz groups, response systems, pyramiding, summaries, ‘instant’ questions, slide shows, integrated teaching module and discussions during the lecture [8–10]. MCQs have been used extensively as a tool for evaluation. The authors have studied the role of MCQs self-assessment in improving learning during medical lectures. According to Maslow’s humanistic theory, self-actualization is the main goal of adult learning and learning occurs chiefly through intrinsic motivation and reflection on personal experience [11]. Rogers has stated that the essential characteristics of adult learning are personal involvement, self-initiation and self-evaluation [12]. Giving the learners more responsibility could foster self-esteem and influence them into greater depths of learning [13]. Self-assessment in the classroom provide feedback to the learners on how to improve their learning. MCQs can be used as a competent assessment technique both for summative and formative assessment [14]. It has been shown to be objective and dependable even with larger numbers of items on a test [15]. MCQ based tests require less time to administer and can evaluate the student’s knowledge comprehensively than tests requiring written responses. There is a lack of high quality evidence about the use of MCQs, despite their widespread use [16].
In this study, the answers were discussed after the MCQs were administered. This enables the students to self-assess without the fear of rebuke. Administering the MCQs at the conclusion of the lecture, helps them to recollect and reflect on the topic discussed. An added advantage is that it does not distract the learners, which can occur with open questions during the lecture. The announcement about the MCQs at the beginning can increase the attention of the students during lectures and their involvement in the educational process. Gender preferences of assessment methods have generated extensive debate among researchers. Some have suggested that men tend to be more deductive and are better at deciding quickly giving them natural advantage in tests with multiple choice questions [17, 18]. A large study looking at MCQs and gender bias proposed that answer construction rather than answer selection removes gender bias [19]. The present study has shown that men learn better when MCQs were administered at the conclusion of the lecture and this effect is not because of innate gender differences in approaching the MCQs. The ‘difficulty index’ of the MCQs was calculated to find out if the questions were too easy, too difficult or just right. Almost all the questions had a difficulty index between 30% and 70% enabling the critical analysis of the learner’s responses. A key advantage of this lecture modification technique is that increased involvement and active learning of the learners does not demand additional independent study on their part. One of the limitation of this study is the skewed ratio of men and women. A major barrier for employing this method universally is the need
986
for development of skills of setting MCQs by the faculty. Considerable time is needed for preparing MCQs, which the faculty might prefer to grant to other educational activities. This technique chiefly uses the cognitive domain of learning and is not useful for the psychomotor and affective domains. MCQs based self-assessment at the conclusion of the lecture does not improve the learning among prefinal year medical students. However, this study has shown the usefulness of MCQs based self-assessment among male medical students. In view of its potential in improving the effectiveness of lectures, large scale multicentric studies are needed.
Contributions TSV designed the study, analyzed and critically evaluated the data and revised the manuscript and he will act as the guarantor of the paper. ABT collected the data, analyzed and interpreted the data and drafted the article. RM, VD, BK were involved in designing the study, collecting the data and drafting the manuscript.
Conflict of Interest None.
Role of Funding Source None.
References 1. Held S, McKimm J. Improve your lecturing. Br J Hosp Med. 2009;70:466–9. 2. Kacker SK, Adkoli BV. Need-based undergraduate medical curriculum. Indian J Pediatr. 1993;60:751–7. 3. Bonwell CC, Eison JA. Active learning: creating excitement in the classroom. Washington DC: George Washington University; 1991. p. 63.
Indian J Pediatr (August 2011) 78(8):983–986 4. Graffam B. Active learning in medical education: strategies for beginning implementation. Med Teach. 2007;29:38–42. 5. Palmer EJ, Devitt PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? BMC Med Educ. 2007;7:49. 6. Diesel E, Alley M, Schreiber M, Borrego M. Improving student learning in large classes by incorporating active learning with a new design of teaching slides. Front Educ Conf. 2006;27:11–6. 7. Rawat MS, Kamal S. Education for primary pediatric care. Indian J Pediatr. 1997;64:369–72. 8. Gibbs G, Habeshaw S, Habeshaw T. Improving student learning during lectures. Med Teach. 1987;9:11–20. 9. Soudarssanane MB, Singh MC. Teaching epidemiology of acute diarrheal diseases to medical undergraduates—a new approach. Indian J Pediatr. 1994;61:277–80. 10. Verma K, Monte BD, Adkoli BV, Nayer U, Kacker SK. Inquirydriven strategies for innovation in medical education: experiences in India. Indian J Pediatr. 1993;60:739–49. 11. Kramlinger T, Huberty T. Behaviorism versus humanism. Train and Dev J. 1990;44:41–5. 12. Rogers CR, Freiberg HJ. Freedom to learn. Columbus, Ohio: Merrill; 1994. p. 266. 13. Hart D. Opening assessment to our students. Soc Educ. 1999;65: 343–5. 14. Abdel-Hameed AA, Al-Faris EA, Alorainy IA, Al-Rukban MO. The criteria and analysis of good multiple choice questions in a health professional setting. Saudi Med J. 2005;26:1505–10. 15. Downing SM. Reliability: on the reproducibility of assessment data. Med Educ. 2004;38:1006–12. 16. Considine J, Botti M, Thomas S. Design, format, validity and reliability of multiple choice questions for use in nursing research and education. Collegian. 2005;12:19–24. 17. DeMars CE. Gender differences in mathematics and science on a high school proficiency exam: the role of response format. Appl Meas Educ. 1998;11:279–99. 18. Ryan KE, Fan M. Examining gender DIF on a multiple-choice test of mathematics: a confirmatory approach. Educ Meas. 1996;15:15– 20. 19. Cole N. The ETS gender study. How females and males perform in educational settings. ETS Technical Report 1997. Available from URL: http://www.edge.org/3rd_culture/debate05/debate05_index. html
