and information technology, teaching this course is particularly challenging for several reasons. .... come from all disciplines at CSU with a wide range of.
Using the iClicker Classroom Response System to Enhance Student Involvement and Learning Christopher WHITEHEAD Computer Science, Columbus State University Columbus, GA 31909, United States of America and Lydia RAY Computer Science, Columbus State University Columbus, GA 31909, United States of America ABSTRACT Classroom response systems are emerging as a novel technology to enhance involvement and learning of a new generation of students. In this paper, we describe our experiment of using such a classroom response technology in an introductory computer science course offered to students of all majors. The results of the experiment provide evidence that the use of classroom response systems positively impacts student learning. Keywords: Classroom Response System, Student Involvement, Student Learning, Computer Literacy, iClicker.
1.
INTRODUCTION
Modern society is becoming increasingly dependent on information technology. As a consequence, the demand for skilled workers who can understand and take advantage of the new methods of communication and information management is also growing [6], [9]. This trend greatly impacts the mode of higher education in the United States by requiring that a computer literacy course to be an integral part of liberal arts education [7], [8]. Although this computer literacy course is typically a very basic course that offers simple fundamentals of computer and information technology, teaching this course is particularly challenging for several reasons. The group of students taking this course is typically very diverse with students coming from various disciplines and various backgrounds. While some students attending this course may already know a lot about computers, some students can be absolutely illiterate about computers. Therefore,
presenting course material in a learning style that is optimal for such a wide range of students and fully engaging students during class periods pose a great challenge. At Columbus State University, the authors have been experiencing the difficulties mentioned in the previous paragraph while teaching the introductory computer literacy course (CPSC 1105) for last few years. Since the course is relatively easy and not related to any major, students often appear uninterested and tend to skip lecture periods. Even if students are forced to attend lectures by enforcing attendance rules, they do not pay attention to the lectures and engage in other activities instead. The authors have tried various quiz schemes. Too many graded quizzes in addition to regular assignments and examinations created additional pressure on students and additional administrative activities for the faculty. On the other hand, students did not take quizzes that were not graded seriously. In either case, quiz schemes did not succeed in enhancing student involvement or learning for this particular course. As a result, the authors sought a different solution and came upon an emerging classroom response system called iClicker. In order to study the effect of this new technology on student engagement and learning, the authors implemented the use of the iClicker devices in two sections of CPSC 1105, while using two other sections as a baseline. At the end of each class, students were required to complete a survey which was carefully designed by us to extract the views of students on the use of this technology and on the course. The authors analyzed the results of the survey as well as students’ performance at the end of the corresponding courses. In
this paper, the authors describe our experiment, the results of the survey and the analysis of the impact the devices had on student learning in detail.
The paper is organized as follows. The next section contains a brief description of iClicker system including usage and costs. Section 3 documents outlines of previous research related to education and technology. In sections 4 and 5 we describe our experiment and the results correspondingly. The paper is concluded in section 6.
2.2 Benefits and Issues The iClicker classroom response system provides the following benefits: It enables an instructor to test for student understanding instantly in the classroom, initiate class discussion and enhance the quality of interactive teaching. The instructor can integrate iClicker with his/her course management plan and save student responses for later analysis. The instructor can make a variety of statistical analyses on student performance data collected using iClicker. This system enables students to anonymously respond in the classroom so that they are more encouraged to participate in classroom activity without any fear of being embarrassed for making mistakes. While this is an advantage from students’ perspective, it prevents the instructor from knowing individual performance of the students. Therefore, this classroom response system is not a replacement for quizzes, assignments or examinations.
2.
CLICKER CLASSROOM RESPONSE SYSTEM
2.1 The Technology iClicker is a flexible and easy-to-use classroom response system, invented in 2005 by a team of physicists from University of Illinois [5], that enables students to instantly provide feedback and answer questions posed by their instructors. It is used in the following way:
The instructor and all students each possess a portable handheld device called a “clicker”. The clicker device contains several buttons (A, B, C, D, E), which when pressed, transmit a wireless signal to a wireless receiver, which is connected via a USB (universal serial bus) connection to a computer An easy-to-use iClicker application running on the computer allows the data received by the receiver to be managed. By registering his/her clicker with the receiving device, the instructor is able to use his/her clicker to manage the iClicker application and the data transmitted by the student clickers. To solicit student responses, the instructor first presents a multiple choice or true/false type question on a Microsoft PowerPoint slide, then starts a timer which allows student responses to be recorded. While the timer is going, the students can select an answer by "clicking" on the appropriate button (A, B, C, D, E) on his/her clicker. When a button on a student’s clicker is clicked, a wireless signal is sent immediately from the clicker to the receiver. The receiver, through iClicker’s application software, logs and stores the data of each individual student. If a student clicks on multiple buttons, the last button clicked is recorded. Using his/her clicker, the instructor can stop the timer which then discontinues the recording of any further responses until the timer is started again. The instructor can then use his/her clicker to display the correct answer and the response results including individual percentages in a bar graph. If the instructor chooses, he/she can also display the response results dynamically in a bar graph while the timer is still going.
The response results are also available for later analysis, grading, and exporting to any gradebook software or course management systems.
3.
RELATED RESEARCH
As an emerging classroom response system, the use of iClicker technology is in its infancy and is not yet widespread. A few reports have been published in 2009 Clickers in the Classroom conference [3]. Studies have been done on freshmen courses such as Introductory Geology, Communication, Psychology, Management, and Statistics in various universities across the United States. Issues such as student participation, anonymous response on sensitive issues such as gender bias or racial issues, teaching in a very large classroom were discussed in these papers. Dr. Terry Hancock, from College of Business at University of Louisville, documents his experiment with the iClicker system for managing large classroom and testing students’ understanding of complex material in [4]. In [1], Dr. Leilani describes her experience with using the classroom response technology in Environmental Geology in order to enhance student participation in class. In [2], the author documents use of iClicker to enhance critical thinking in an atypical class on “Communication and Gender”. All these experiments show that this novel technology positively impacts student learning.
4.
ICLICKER EXPERIMENT AT CSU
4.1 Overview of the Course Introduction to Information Technology (CPSC 1105) is an introductory, interdisciplinary course designed to
enhance students’ knowledge of information technology. The course covers the nature of information, computer hardware, software, communications technology, and computer-based information systems. The theory is complemented by practical work aimed at gaining basic proficiency with different types of widely used application software including Microsoft Office (Word, Excel, PowerPoint, and Access). 4.2 Motivation for Using iClicker The primary motivation for using the iClicker student response system was to enhance student participation in the classroom and actively engage students in the learning process thereby increasing student learning. As explained in the introduction, students who enroll for this course come from all disciplines at CSU with a wide range of background in information technology. Therefore, meeting demands of all students and engaging them in classroom activities is a great challenge. Schemes such as quizzes and classroom discussions have not yielded satisfactory results. Therefore, the authors sought a new solution with this novel technology. 4.3 Implementation The study involved the two authors who taught four different sections of CPSC 1105 during the Spring 2009 semester. The course content, exams, assignment grading and other course material in the four sections were identical. The lectures delivered throughout the course were one hour and fifteen minutes in length and were based on the use of PowerPoint presentations provided by the textbook publisher. Except for individual instructor styles, the lectures were delivered in as much the same way as possible. Unfortunately, the iClicker devices were not acquired until the second month of the semester. As a result, the devices were only used in the seven lectures following the midterm exam. Each instructor taught two sections of the same course – one in which the iClicker devices were used and one in which the iClicker devices were not used. Each instructor selected the weaker section with worse average student performance in the midterm for iClicker use. The authors independently developed their own set of iClicker questions, ranging from 5 to 14 questions per lecture with a median of 8 questions.
5.
RESULTS
5.1 The Survey for Students Appendix A contains the survey (questions 1 - 20) that was administered to students in the two different sections of the course in which the iClicker devices were used. 43 students participated in this survey; 28% males, 73% females. Appendix A also contains the survey (questions
11 - 20) administered to students in the two different sections of the course in which the iClicker devices were not used. 50 students participated in this survey; 32% males, 68% females. To aid in comparing the results from the sections using the iClicker devices and the sections not using the iClicker devices, questions 11 through 20 of the survey administered to sections using the iClicker devices were identical to the questions 1 through 10 administered to the sections in which the iClicker devices were not used. 5.2 The iClicker Survey Results Table 1 lists the mean responses for the survey questions administered to the two sections in which the iClicker devices were used based on a scale of 1 to 5. The questions are grouped according to the categories: Pedagogical Impact, Device Functionality, Delivery Methodology and Administrative Utility. The results from questions 2, 3 and 6 indicate the use of the iClicker devices had a significant impact on student learning, while the results from questions 8 and 9 indicate somewhat less of an impact though still positive.
Category Pedagogical Impact
Delivery Methodology Device Functionality Device Utility
Question #
Mean Response
2
4.5
3 6
4.2 4.1
8 9
2.6 3.5
10 4
4.4 4.0
5 1
2.7 4.7
7
4.0
Table 1. Survey responses in sections in which iClickers were used. As previously mentioned, the iClicker devices were used over a series of seven lectures. During the seven lectures, the number of iClickers questions asked ranged from 5 to 14 with a median of 8. According to the results from question 4, the students felt this number was appropriate. In general, the type of questions asked was recall type questions. For example, “When a computer is turned on, the first step in the boot process is:”. Although the results of the survey indicate the students felt this type of questions was relatively easy, these results need further investigating to evaluate their impact on student learning. The results from question 1 clearly indicate the iClicker devices were easy to use. This information, however, would be more valuable in comparison to the use of other similar devices.
The final question, question 7, was designed to ascertain the student’s willingness to accept the iClicker device as a method of taking attendance and earning grades. The results of the survey indicate the students would be willing to accept the use of the devices in this capacity. 5.3 Comparison Between Classes Using iClickers and Not Using iClickers Table 2 provides the mean responses for the survey questions that were asked in all four sections of the course. As can be seen, except for the questions, “The course lectures were informative and helped me better understand the course material.” and “I have developed a better understanding of the subject by taking this course.”, the mean response in the sections in which the iClickers were used appear to be higher than the mean response in the sections in which the iClickers were not used. However, t-tests assuming equal variances showed that none of the paired values were significantly different at the 0.05 level of significance. The authors believe that the small amount of data and the assumption of equal variance contributed to this conclusion.
in which the iClickers were used. Likewise, the percentage of students who received a non-passing grade was lower initially but increased in the sections in which the iClickers were not used, while the percentage of students who received a non-passing grade was initially higher in the sections in which the iClickers were used, but which decreased between the midterm exam and the final course grade. Without iClicker
With iClicker
Midterm
Final
Midterm
Final
A
11.5
40.7
16.7
14.0
B
50.7
31.5
35.4
42.0
C Total Passed
24.4
11.1
29.2
32.7
86.7
83.3
81.3
88.7
D
3.8
7.4
6.3
6.5
F Total Not Passed
9.5
9.3
12.5
4.8
13.3
16.7
18.8
11.3
Table 3. Results (values in percentage) Question # 1/11
Mean Response in Sections Not Using iClicker 3.73
Mean Response in Sections Using iClicker 3.90
2/12
3.53
3.30
3/13 4/14
3.55 3.12
3.78 3.28
5/15 6/16
3.38 3.34
3.68 3.63
7/17 8/18
3.09 3.66
3.28 3.90
Table 2. Comparison of survey responses. 5.4 Impact on Student Grades Table 3 lists the percentages of grade distributions in the two sections in which the iClickers were not used compared to the two sections in which the iClickers were used. The results are also displayed graphically in Figure 1. As previously mentioned, the sections with worse average midterm performances were selected for iClicker use. Table 3 and Figure 1 show that while the percentage of students who received passing scores just after the midterm exam was higher in the sections in which iClickers were not used, the percentage of students who received a passing score at the end of the course was significantly higher in the two sections in which the iClickers were used. In addition, the number of students who received a passing grade decreased between the midterm exam and the final grade in the sections in which the iClickers were not used, while the percentage of students who passed the course increased in the sections
Figure 1: Performance comparison between classes with and without iClicker
6.
CONCLUSION
The results of this study provide evidence that the use of iClickers positively impacted the students’ learning during the course. These results are supported by the increased number of students who received passing
grades in the course in the sections in which the iClickers were used. However, these results are based on the limited sample size of four sections of the course, and are subject to numerous outside factors including possible instructor bias. Additional research is needed to substantiate these results and to provide clear evidence that the use of devices such as iClicker do indeed positively impact student learning. In addition, the survey results indicate that students preferred approximately eight questions per one hour and fifteen minute lectures. More data is also needed to support this finding as well as the optimal number of questions for maximizing student learning. Lastly, the type of questions asked during this study was general recall as previously mentioned. Additional research is also needed to support the impact of the use of different types of questions, including those involving teams.
7.
REFERENCES
[1]
Arthurs, D. L. (2008). Expanding the Use of Classroom Response Systems (CRS): CRS Integration with In-Class Group Activities in Large Classrooms. Inaugural Conference on Classroom Response Systems 2008. Louisville.
[2]
Ashlock, D. M. (2008). Communication and Gender: Using the i>clicker to Generate Critical Thinking. Inaugural Conference on Classroom Response Systems 2008. Louisville.
[3]
Conference Papers. (n.d.). Retrieved October 10, 2009, from iClicker: http://www.iclicker.com/dnn/UserCommuni ty/ConferencePapers/tabid/171/Default.asp x
[4]
Hancock, D. T. (2008). Use of Classroom Response Systems (CRS) in Formal Testing. Inaugural Conference on Classroom Response Systems 2008. Louisville.
[5]
Inventors. (n.d.). Retrieved February 15, 2009, from iClicker Web Site: http://iclicker.com/dnn/Abouticlicker/Inven tors/tabid/146/Default.aspx
[6]
Kai S. Koong, L. C. (2009, June). A Study of the Demand for Information Technology Professionals in Selected Internet Job Portals. Journal of Information Systems Education , Vol 13(1).
[7]
Mohan
Rao,
J.
K.
(2006).
Computer
Literacy: Implications for Teaching a College-Level Course. Southwest Decision Sciences Institute Annual Conference. Oklahoma.
[8]
Nitham M. Hindi, D. M. (January1, 2002). Computer Literacy: Implications for Teaching a College-Level Course. Journal of Information Systems Education , Vol 13.
[9]
Richard Ellis, B. Lindsay Lowell. (1999). Assessing the Demand forInformation Technology Workers. http://www.cpst.org/IT-4.pdf.
APPENDIX A SURVEY ADMINISTERED TO STUDENTS Questions 1 - 20 were administered to “iClicker” students; questions 11 - 20 were administered to “non-iClicker” students: Question # 1 2 3 4 5 6 7
Question The iClicker handset was easy to use. The iClicker questions encouraged me to actively participate in lectures. Answering iClicker questions during the lecture helped me better understand the concepts in this course. The number of iClicker questions asked in class was: The iClicker questions asked in class were, on average: The iClicker questions helped me focus and pay more attention in lectures. Grades and/or participation points should be allocated for answering iClicker questions.
Possible Responses 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 3 = Too few | 4 = Just right | 5 = Too many 1 = Very easy | 2 = Easy | 3 = Just right | 4 = Difficult | 5 = Very difficult 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA
8
The iClicker questions would have been just as useful if they had been presented in class without the use of the iClickers.
1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA
9 10
The use of iClickers encouraged me to come to class. Overall, how would you rate your iClicker experience in this class?
11
The course lectures were informative and helped me better understand the course material.
1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 1 = Very negative | 2 = Negative | 3 = Neutral | 4 = Positive | 5 = Very positive 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA
12 13
The textbook helped me better understand the course material. Assignments in this course were appropriate, reflected the course content and helped me to learn course material.
1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA
14
The assignments in this course were:
15 16
MyITLab is an effective learning environment: Splitting the course into a lecture period and a lab period helped me to learn course material. The overall difficulty of this course was:
1 = Too easy | 2 = Easy | 3 = Appropriate | 4 = Difficult | 5 = Too difficult 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA
17 18 19 20
1 = Too easy | 2 = Easy | 3 = Appropriate | 4 = Difficult | 5 = Too difficult 1 = SD | 2 = D | 3 = N | 4 = A | 5 = SA
I have developed a better understanding of the subject by taking this course. My gender is: 4 = Female | 5 = Male Please provide any additional comments not covered above including suggestions for improving the course:
Note: SD = Strongly disagree; D = Disagree; N = Neutral; A = Agree; SA = Strongly agree
