and universities use laptop computers, while 62% of those students use a smartphone ..... the mini-lecture, subjects are asked whether the interaction felt natural ...
Classroom Response Systems in Higher Education: Meeting User Needs With NetClick Darren Abramson, Krzysztof Pietroszek, Leila Chinaei, Edward Lank and Michael Terry David R. Cheriton School of Computer Science University of Waterloo, Waterloo, Ontario, Canada e-mail: {dabramso, kmpietro, lchinaei, lank, michael.terry}@uwaterloo.ca Abstract—Classroom response systems (CRS) have been shown to dramatically improve a variety of learning outcomes in science and engineering education. Their adoption in research university classrooms has been very slow. This paper postulates a contributing factor to the lack of widespread adoption of CRS in that context and introduces a new CRS designed to mitigate the identified factor. The CRS, NetClick, is built on the premise that the requirement of authoring new content prevents many Professors from using existing CRS. After describing the operation and advantages of NetClick to other CRS, we describe initial feedback from Computer Science Professors on its use for converting existing teaching slides into interactive content.
I. I NTRODUCTION There is a broad consensus in experimental research on education that interactive learning results in better outcomes, where these are defined not only to include higher academic achievement, but also student retention in academic programs, perception by students of social support in learning, and quality of social interaction in the learning environment [1, 4]. Therefore, to the extent that educators in engineeringrelated disciplines are interested in improving this variety of outcomes among their student population, there is significance incentive for them to introduce interaction into the classroom. Computerized technology has numerous advantages as a medium for interactive learning, but is not widely used for this purpose. We offer both an account of why this is, and present a novel classroom response system. We show that our constructed system can overcome barriers to adoption by other such systems by presenting the results of a study of its use. We show that the technological impediment to widespread adoption of computer-based interactivity in the classroom is not intrinsic to such technology, but the interaction of that technology with existing work practices. II. I NTERACTIVITY I N THE C LASSROOM : C OSTS , B ENEFITS , AND W ORKFLOW A. Standard methods and challenges Interaction in the classroom can be as simple as forming groups that present material to the rest of the class. Systems that use computer communication to respond to queries in a classroom system are called ‘classroom response systems’ (CRS) or ‘audience response systems’. Classroom response systems provide significant advantages over ‘low tech’ approaches to interaction. For example, if interaction is mediated through individual student use of computers, and aggregated
by the instructor, then feedback can be collected quickly and anonymously. Qualitative studies of users of CRS show that they value the anonymity they provide, claiming that it leads to both higher amounts of interaction and greater depth of interaction than a non-anonymous alternative [2, 13]. The most well known example of a CRS is the commercial product ‘i>clicker,’ manufactured by Macmillan (see www.iclicker.com). The i>clicker system includes specialized devices that are distributed to students, and an interactive content creation system. While this CRS provides significant advantages in the classroom compared to either no interaction or low tech interaction, it also introduces notable disadvantages. One study of the use of i>clickers in a large statistics classroom environment, despite finding significant quantitative evidence for the advantages just mentioned, singled out the following disadvantages for educators [3, 5]: • • • • •
integration of i>clickers with course material was time consuming; integration of i>clickers with course material exceeded estimates of required time; individual instances of i>clicker use took 1-2 minutes, perceived by educators as too long; management of i>clickers involved dealing with lost and defective hardware units; students expected educators to resolve issues in i>clicker management.
While we have not yet found reliable industry statistics on the use of i>clickers in the post-secondary classroom environment, our own anecdotal experience working as professors and graduate students across 5 major universities suggests that the penetration is very low. The above deficiencies are suggestive of why this is the case despite the clear incentive for professors to use a CRS. There is evidence that the deficiencies noted above, which we will argue are associated with hardware- and software-specific CRS, are in fact responsible for low adoption rates. A significant proportion of higher education in North America takes place at so-called ‘research institutions’: these are universities at which full time faculty are hired and promoted largely on the basis of their production of original research, and securing grant support for that research. Again, speaking anecdotally, research institutions often pay lip service to other forms of professional accomplishment such as success in
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 840
teaching and service to an academic discipline when making hiring and promotion decisions. There is clear evidence, though, that faculty members at research institutions perceive teaching as having significantly lower value than research among decision makers; also, faculty who place great investment in improving teaching outcomes are in the minority, and report difficulty in justifying this investment [4]. Therefore, CRS such as i>clickers have the potential to dramatically improve the learning experience of undergraduates at research institutions across a variety of dimensions; but, since their professors have little incentive to invest significant time into teaching, and i>clickers require such an investment, it is unlikely that professors at research institutions will choose to use them. This incentive structure and time investment required provide a plausible explanation for the apparent lack of deployment of CRS in classrooms at research institutions. B. CRS in the brave new world The last twenty years have seen revolutionary changes in the cost, adoption, and development of consumer electronic devices. Many students now arrive to the post-secondary classroom with a variety of Internet-enabled computers: smartphones, tablets, and notebook computers. The EDUCAUSE Center for Applied Research (ECAR; EDUCAUSE is a not for profit institution that promotes the use of technology in higher education) refers to this as the ‘bring-your-own-device era’ [5]. They have followed adoption of various forms of computer technology over the past 8 years; present data shows that 86% of undergraduate students at American colleges and universities use laptop computers, while 62% of those students use a smartphone (defined to be a cellphone that has rich access to the Internet through a web browser) [5]. Unfortunately, that study did not identify the percentage of students who owned either a smartphone or a laptop, nor what percentage of students had those devices with them in the classroom. Nevertheless, there is evidence that the vast majority of university students bring an Internet-enabled portable computer to the classroom. A recent wave of CRS that leverage the bringyour-own-device phenomenon include such systems as www.coursemodo.com and www.tophatmonocle.com. In this paper we argue that these CRS address some of the deficiencies of hardware-based CRS, but fail to address the most important preventing their adoption in research institutions. Each of these systems shares a common element of workflow: in order to create an interactive presentation, the user of the system must create a new presentation using a web interface unique to the CRS. There is some indication that the developers of new CRS recognize the disincentive associated with the creation of new content. For example, www.tophatmonocle.com indicates that their system can be used ‘with any presentation system, such as PowerPoint’.’ After completing a guided tutorial of the Top Hat Monocle CRS, it is apparent what this means: a dedicated program can be run in minimized form with a floating icon over a PowerPoint presentation in full screen
mode. When the educator chooses to pose a question, the CRS window is maximized, obscuring the slide presentation. The CRS window provides a set of pre-programmed questions that must be organized in such a fashion as to match the order of content displayed in the third party software such as PowerPoint. We have not described the detailed operations of other CRS because, after investigation, all the ones we have discovered follow the same model: dedicated interface, either through the Web or software tied to the CRS is used to create content. Interaction between students and the CRS is mediated by a dedicated system, and only involves responses to queries created through the dedicated interface. As described above, one of the main complaints professors have about traditional hardware-based CRS is the initial time investment associated with their adoption. Although the systems just mentioned address issues of hardware maintenance (since students use their own smartphones, laptops, and tablets), they do not address the combination of initial resources required, combined with a lack of compelling incentives to justify the investment. III. S OLVING CRS CHALLENGES WITH A NEW APPROACH A. NetClick.mobi: a CRS for existing content Some of the authors have created a novel CRS that aims at addressing this central issue with existing systems, available for public use at www.NetClick.mobi. NetClick.mobi was explicitly designed to address the fact that most educators in post-secondary education have a significant amount of teaching presentation material already created. The system is premised on its ability to be used without authoring new content, or even specifying interaction modes. Existing computer slides, in .pdf format (which major presentation software suites can natively output to) can be dragged and dropped onto NetClick.mobi through a browser interface: this initiates the upload and conversion process to interactive format. Once completed, the slides are now available for interactive presentation. The operation of NetClick requires the initiation of a presentation within the system, and then an interaction within that presentation. In the following, we assume that the user (Professor) of NetClick has already taught a class at least once using a static (non-interactive) computer slide presentation, such as PowerPoint, or PDF output using the Beamer package for LATEX. An initiation of a presentation within NetClick denotes the following set of events: 1) Prepare lecture content by converting slides to PDF format if not already in that format, and by locating the icon of the PDF slides. 2) Register at www.netclick.mobi, or log in if one has already registered. 3) Drag-and-drop the icon corresponding to the existing lecture notes from the file system onto the ‘drop zone’ in the browser. 4) Once the file has been uploaded and converted, click the ‘play’ button.
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 841
5) Enter the key command to display the browser in full screen mode (indicated, and differing depending on browser and operating system). 6) Instruct students to open the browser on their smartphone, tablet, or laptop. 7) Instruct students to enter the 6-letter code in the field under the ‘Slideshow’ tab. In order for the initiation sequence to be successful, students must be running a browser that supports HTML5. This includes all Android and iOS devices sold within the last year, and all recent desktop browsers, with the exception of Internet Explorer. If a student or professor attempts to use NetClick with an unsupported browser, a message is displayed explaining that a different browser must be used, with instructions on obtaining one. Once connected to a presentation, each student sees the currently displayed slide on their device. The Professor may advance to the next slide by using either the arrow key or the space bar. As the Professor advances slides, the slide displayed in students’ browsers is advanced to match it. Using navigation arrow buttons, students may go back and then forward within the presented slides. Students however cannot advance forward past the currently displayed slide. The 6-letter code and the control buttons for the Professor are displayed on a toolbar that, by default, is visible for the first three slides of their presentation. After the third slide it auto-hides, but reappears if the Professor hovers the mouse pointer near the top of the browser. ‘Interaction’ denotes a specific set of events within the NetClick system. The sequence of events for an interaction within the NetClick system is as follows: 1) The Professor is displaying a slide with the ‘enable interaction’ button depressed. This button is available on the toolbar, and is depressed by default. 2) The Professor invites students to interact, by posing a question concerning the currently displayed slide content. 3) Students click (if using a mouse controlled pointer) or press (if using a touch interface) to indicate the region of the slide they intend as the answer to the Professor’s question. 4) The Professor depresses the ‘display interaction’ button. This button is available on the toolbar, and is left undepressed by default. 5) An overlay corresponding to the aggregated responses of students is displayed over the current slide. The displayed overlay is a heat map of the students’ responses: colour represents number of click or press events at a given location on the slide. Primary colours are used in conventional fashion to indicate number of overlapping touch/click regions from least to most, as a proportion of total touch/click events. From least to most overlapping interaction regions, the colours used are: blue, green, yellow, red. Following a NetClick interaction, the Professor may either hide the interaction overlay by clicking the ‘display interaction’ toggle and return to the current slide, or advance to
the next slide. If the interaction is hidden, then the Professor can solicit additional interactions for that slide. Students can change their selection by simply clicking or touching on a new region of the presented slide. There is no limit to the number of interactions per slide. The NetClick system has been tested with over 150 concurrent students, and is able to aggregate that data in real time for near-instant presentation of a heat-map overlay. An assumption of the system is that slides that have been designed and authored for use without technological mediation with students are well suited to interactive presentation using technology. This assumption is justified by the observation that most educators interact with students even in the absence of technology introduced for this purpose. The most simple example of such interaction is querying a classroom to ask if there are any questions, or posing a specific question whose answer is implied by the displayed content. B. NetClick and other CRS While designed to address the issue of reusability of content, there are other prima facie advantages of NetClick compared to other CRS. There is evidence that users of i>clickers perceive the process of formulating questions with set answers before lecture as lacking spontaneity; one user describes the experience of seeing blank faces in a classroom and wishing to apply interactive technology, but being unable to do so since no prepared questions for the CRS are available [6]. The interactive model of NetClick presumes only that the Professor have some set of slides containing the content they wish to teach from; it does not presuppose that they will, in advance, know which material will present unique difficulty to students. It might be argued that if Professors did generally know that, then CRS would not be necessary for generating awareness of where students face difficulty. Another prima facie advantage of NetClick addresses a problem recently identified in studies of i>clickers; Chinaei and Lank show the following. Subjects who use CRS and those who study the use of CRS agree on what makes up a good interactive question: it is one that separates the class into distinct groups, based on their understanding of the material. Quoting a subject, “the clicker pedagogy says that you should design your question such that it’s not the case that all students got the right answer the first time. . . that question is in some sense useless” [6, 3]. Similarly, a question that all of the students get wrong the first time is also likely to be useless, since it does not discriminate those students that are paying attention and understand presented material. Let us call a question that tends to produce significant numbers of both correct and incorrect answers a ‘good discriminator’. Good discriminators provide the pedagogical advantage of helping to redirect Professors’ energy towards unclear material. In order to create good discriminators, though, the Professor must know beforehand where some, but not all students will face difficulty. A lack of spontaneity is therefore a symptom of a deeper issue with existing CRS. To be effective,
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 842
the Professor must not need them to know where students need help. To be fair, there is evidence that CRS provide advantages beyond informing Professors of student difficulty. They provide additional benefits: posing good discriminators not only helps to measure and respond to student difficulties, but also generates significantly greater amount of student discussion, itself a valuable form of interaction. Therefore, using a CRS can be helpful, even if they do not provide otherwise unavailable information concerning student knowledge. NetClick has the advantage that Professors who are less able to anticipate precise sources of student difficulty can spontaneously pose questions, thereby taking advantage of blank faces, and discovering good discriminators as they go. A final advantage that NetClick offers over existing CRS is its ability to make spatial queries concerning graphical content. One example is the presentation of the deduction of a theorem in a logic class. Even if each line is labeled, so that students can identify them in questions, individual lines in the deduction may be lengthy. Enabling students to click or touch portions they don’t understand allows random access for indicating lack of comprehension across the entire deduction. Queries concerning mechanical diagrams are another vivid case where pointing or clicking can be useful. A Professor can ask ‘are there any parts of this diagram that you don’t understand’ and instantly get feedback from an entire class. Unpublished evidence from the authors’ own experience with NetClick has shown significant advantages in teaching Unified Modeling Langage (UML). Using NetClick, students can be queried on many concepts with a single slide containing a complicated UML diagram, as follows: 1) Which class has an association with this class? (Professor points at a class in a UML diagram) 2) Display interaction. (Professor explains accordingly.) Hide interaction. 3) Which class realizes this class? (Professor points at another class in a UML diagram) 4) Display interaction. (Professor explains accordingly.) Hide interaction. 5) Which class generalizes this class? (Professor points at yet another class in a UML diagram) 6) Display interaction. (Professor explains accordingly.) Hide interaction. Notice that the choice of questions in a complicated UML diagram will be quite vast giving the Professor many choices in real time. Also, notice the difficulty both of constructing multiple choice questions in advance for such a use case, and anticipating precisely where students will have difficulty. IV. A PILOT STUDY OF N ET C LICK ’ S CLAIMED
2) Is the removal of this disincentive likely to lead to the use of a CRS? 3) Does NetClick.mobi technology have other barriers to use that make it unattractive in terms of time required for use? 4) Is existing presentation content suitable for NetClick.mobi use in the classroom? The last question is answered by having subjects drag and drop existing content into the NetClick, and then walking through a mini-lecture, with the stipulation that they elicit interaction from the interviewer at least once per slide. After the mini-lecture, subjects are asked whether the interaction felt natural, contributed to the presentation of the material or distracted from it, and could be applied at a similar rate to other lecture material. A. Methods Subjects were instructed to prepare lecture slides for use in the mini-lecture portion of the interview. These were specified to be slides that had been used in a past course, but did not have an interactive component. We requested that subjects convert slides to PDF before the interview if they were not already in that form. The interview consisted of three parts: 1) an interview before the mini-lecture which included a primer on NetClick, describing the steps for initiating a presentation and soliciting interaction, and questions concerning the subject’s teaching outlook and experience; 2) a mini-lecture in which the subject presented 15-20 minutes of course material from pre-existing slides, but using the NetClick system; 3) and, an interview after the mini-lecture intended to produce feedback concerning the suitability of NetClick for their teaching. The interviews were semi-structured, although the following questions were used as starting points for further discussion The following questions were posed before the mini-lecture: • • • • •
•
ADVANTAGES
We present the results of a qualitative study of computer science and engineering professors at a research institution in order to answer the following questions: 1) Is their disincentive associated with creating new content enough to prevent the use of a CRS?
How long have you been teaching? How many times have you taught this particular course? How frequently do you update the lecture notes for this course? Is it a minor or major update? Would you describe your teaching as interactive? If so, how do you typically generate interaction? Please start your mini-lecture at any point in the lecture notes, so long as you have material to sustain 15-20 minutes of presentation. Please, with the interaction type understood, use NetClick for as many slides as possible.
The following questions were posed after the mini-lecture: • •
Do you think aggregated feedback of the type generated would be useful for you? How? Do you think aggregated feedback of the type generated would be useful for the student? How?
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 843
• •
•
• • •
Was it difficult or easy to come up with appropriate questions for the interaction model? Would using NetClick be appropriate for the in-class time constraints? Not enough time? Too much time taken from class? Is the technology suited for use in class? Are there any hurdles that would make this projection technology a poor choice compared to what you are using now? Do you have any other concerns about the use of NetClick in the classroom? Do you anticipate using NetClick, or would it take up too much extra time/effort? Was it difficult or burdensome to convert lecture notes to PDF format?
B. Subjects Four subjects provided us with a 1 hour interview in which they conducted a mini-lecture. All of the subjects are either tenured or tenure-track faculty members in Computer Science with research appointments at a large research university. The interviewer played the role of a student, and provided feedback through the NetClick system. Subjects understood that the heat map overlay would be similar to the one perceived in the interview, with signified regions corresponding to many students’ clicks/touches. Of the four subjects, two used PDF slides already in teaching. The third normally used PowerPoint, but said that they faced no difficulty in converting the slides to PDF. Subject 1 (S1) has been teaching for less than 10 years. The subject matter S1 taught was from an upper-level undergraduate graphics course. S1 had taught the course twice before. S1 described that they frequently revised their courses each time they taught them, but hoped to reduce the amount of time that they spent preparing classes going forward. According to S1, preparation for some courses took as much as “8 to 1,” which they explained to mean that it took 8 hours to prepare for 1 hour of class. Subject 2 (S2) has been teaching for more than 20 years as a faculty member. For their mini-lecture S2 used slides from a software engineering course they have taught approximately over 10 times. Typically, S2 makes only minor updates to course material, with major revisions to only one course from their teaching load per year, at most. S2 does not use a CRS in any classes, and describes the current levels of interaction in classes they teach as ranging from “dead” to “alive.” Subject 3 (S3) has less than 10 years of teaching experience. The course that S3 used for the mini-lecture was an upper-level undergraduate course on topics in applied computer science that they have taught 6 times before. They rarely do any major updates of course material, but occasionally make minor updates. In some courses S3 generates interaction by calling on students to ask questions. In other courses, S3 uses i>clickers for generating interaction. Subject 4 (S4) has 10 years of teaching experience. The course that S4 presented material from for the mini-lecture is an upper-level undergraduate architecture course they have
taught 5 times. Typically, they make minor updates to the course material each time they teach it. When teaching this course, S4 attempts to generate interaction by asking the class specific questions. C. Basic use of the NetClick CRS Each of the subjects was given a short tutorial in the basic operation of NetClick. S1 described having had very little difficulty in converting slides from PowerPoint to PDF. S3 had configured their browser to avoid accepting cookies, which is used by NetClick for maintaining authenticated status. In this case, the interviewer lent the subject their laptop, and they alternated browser windows for the Professor and Student role. In each case, the conversion and upload process was uneventful. S4 had some confusion over the behaviour of the two buttons that govern NetClick functionality. The system was designed to require as little use of the interface by the Professor as possible during teaching. The tutorial prior to the mini-lecture did not make clear the following behaviours of NetClick: • on the initiation of a slideshow, interaction is enabled, but hidden; • when the slide is advanced, interaction remains enabled, but is hidden whether or not it was shown on the previous slide. D. Spontaneously creating questions There were varying levels of ability to repurpose existing content for interaction using NetClick. Subjects S1 and S4 were able to pose questions concerning displayed content with little apparent difficulty. S1 was able to pose three questions concerning slide content. The first question concerned a slide with two mathematical descriptions of shader models for determining the brightness of a given pixel. S1 simply posed the question ‘click on an area that is unclear’ with respect to that slide. The interviewer chose an area at random to simulate student interaction. Two subsequent slides contained images produced by different shader models. On the first of these slides S1 asked ‘please point out artifacts that are the result of using the shader model.’ The interviewer clicked a region and S1 explained that the choice was incorrect, and indicated the correct region of the image. With the second set of images, S1 queried the ‘class’ to point out artifacts that persisted with an improved shader model. In this case, the interviewer indicated a region that contained an artifact, but one that did not indicate a deficiency of the model in question. A discussion of types of artifacts ensued. S4 queried the interviewer as to what forms of communications processors have with one another. S4 pointed out, in a brief tangent from the mini-lecture, that the lecture would typically involve soliciting example from the class. S4 went on to ask, on the next slide containing a list of processor communication types, which the students thought was the most important one.
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 844
S3 improvised a question that did not actually involve any text or images contained on the slide used to pose the query. In asking the class how old the given area of applied computer science was, they simply denoted regions of the slide along the horizontal axis as corresponding to a timeline, and roughly indicated, using the mouse pointer, epochs of 500 years. The query posed was to guess how old cryptography is and then indicate one’s answer by clicking on the correspond region. The interviewer guessed about 2000 years old through a NetClick interaction. S2 displayed the interaction and explained the first known instances of cryptography, approximately 6000 years old. S2 was unable to formulate a question using the NetClick interaction model, but did elicit a number of interactions with the interviewer, posing questions about the material during the mini-lecture. E. Feedback on the NetClick CRS S2 and S3 had similar feedback about the NetClick system. Both explained in asides during the mini-lecture, and the discussion after the mini-lecture that they experienced great difficulty coming up with questions that used the NetClick interaction model. Both subjects also explained their perceived source of difficulty in coming up with questions. S2 explained that in order to display slides that are amenable to the NetClick interaction model, “you have to know about [NetClick] as you’re designing slides.” S3 explained that “if you made a slide to work with your system – a very visual question – this would be really good.” S4 had similar observations to S3 concerning the presentation of visual content, and expressed an interest in modifying slide content to contain more opportunities for queries involving S1, although able to pose a number of NetClick-suitable questions, expressed reservations about their existing slide content. For example, S1 pointed out that a ‘really good question’ would involve removing the text labels from the images created using different shader models. They expressed an interest in modifying slides in this way, claiming that it would be ‘really easy’ to do so, since it would involve a small edit in PowerPoint. S1 had some reservations, though, about using a new slide projection tool. PowerPoint has the ability to display different content to students than is displayed to the Professor, allowing the Professor to view upcoming slides. S1 had reservations over giving up that ability in order to generate interaction. S4 also used PowerPoint and described recently introducing animations into lecture content. S4 was reluctant to give up the ability to display animations, which NetClick does not currently support. S3 is the only subject with prior experience using clickers in class. S3 recognized that NetClick was designed to be used for spontaneous interaction. Surprisingly, despite research cited to the contrary, S3 described using clickers for spontaneous purposes. S3 described coming up with a multiple choice question while teaching, writing the various answers to that question on the board, and then soliciting feedback in class. Therefore, expert users of a traditional CRS have been able to
adapt it to serve a similar purpose (spontaneous interactive questions) to NetClick. S3 did point out, though, that the majority of interactions they used with traditional clickers were for pre-planned questions, and not spontaneous questions. F. Discussion There were two distinct attitudes to the interaction model present among the subjects, represented by two subjects each. We call the first attitude the analytic one: during the minilecture, subjects S2 and S3 expressed the view that it was very difficult to formulate questions that used NetClick’s interaction model. Furthermore, both subjects S2 and S3 expressed the source of their difficulty: not having designed the slides to take advantage of intended interaction for the mini-lecture. We use the term ‘analytic’ to refer to a conscious, planned composition of displayed slide content so as to apply available technology during teaching. We call the second attitude the synthetic one, as expressed by S1 and S4. These subjects, despite finding advantages that could be gained by NetClick through minor adjustments to slide content, more actively improvised in creating NetClick interactions. We use the term ‘synthetic’ to refer to a predisposition to applying whatever interactive tools are available during teaching so as to engage with students. It is important to note that the attitude types just described are totally independent from level of interaction. All subjects engaged with the interviewer, posing many verbal questions. For example, S2 described a scenario in which a co-worker at a software company had failed to provide any specifications for software written as part of a group project. “How would you feel if that happened?” S2 asked the interviewer, effectively communicating, through non-technological interaction, the importance of software specifications. The subjects interviewed for this study suggest that there is a specific type among the studied population that would be able to take advantage of NetClick for adopting a CRS with minimal effort. The ideal subject has not invested too much in alternative display technologies (such as PowerPoint-specific features); and, they are willing to make minor adjustments to their slides in advance of teaching, OR they have a strongly synthetic attitude towards teaching. V. C ONCLUSION AND F UTURE W ORK On the basis of the findings from this study, we suggest that NetClick.mobi solves a central barrier to adoption of CRS in classrooms at research institutions, and is well suited to adding an interactive layer to existing content. There are some important caveats, however. Many Professors will not adopt any CRS unless they are aware that it can be used without introducing significant time burdens. Engineering and computer science content created without planning for the use of interactive technologies such as a CRS is appropriate for being made interactive by NetClick, but with minor adjustments. As S1 pointed out, since the same tools used to create existing content are used to adjust content before importing into NetClick, there is little additional
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 845
learning required. As S2 and S3 emphasized, though, many Professors will not be willing to make those adjustments until they have a clear understanding of the interaction model that a particular CRS has. As a result of this study, we conclude that removing the barrier of creating new content will for many Professors be necessary, but not sufficient motivation for adopting a CRS. Effective communication of the abilities of a CRS such as NetClick is also required. The same technologies that make NetClick and other modern CRS possible present a threat to the learning experience. The adoption rate by university students of smartphones is accelerating. The vast majority of university students already bring at least one type of Internet-connected computer to class, and it is not unreasonable to expect that soon nearly all students will have such a device in class. While Internetconnected computers can be invaluable for research and learning, students often use them for other purposes while in class. There is considerable evidence that students who are permitted to use their laptops for search, social networking, or self-directed browsing suffer significant reductions in learning compared to peers who do not use laptops [7], [8]. Some researchers have responded to the challenge of ‘digital distraction’ in their classroom by banning the use laptops in the classroom [9]. This researcher, who bans laptops from his classroom, also claims that “. . . learning in a classroom should be a social process in which the student interacts with the instructor and other students. I try to avoid straight lecturing in favor of engaging my students in interactive questions and answers” (ibid.). It is therefore an open question whether computers and smartphones can be leveraged against the digital distraction they can create, while also enhancing learning through an increase in interactivity. Future work will study whether students, when presented with a synchronized, interactive copy of a Professor’s lecture slides can mitigate the distracting effects of an Internet-connected computer. A useful quantitative metric might be the comparison of three conditions: students who are permitted to use their smartphones and laptops however they wish during class, students who are asked to interact using NetClick (but not given instructions to avoid multitasking), and students who are instructed to put away Internet-connect computers during class. An ideal outcome would be that students who use
NetClick to view lecture material outperform the other two groups. Notice, though, that a valuable result would have the students who use NetClick perform at the same level as the students without computers. This is plausible: the advantages of interaction and the focusing effect of having slides change on one’s computer might counter-balance the distractions of available Internet content. In this case, the use of NetClick might be preferable to a policy of banning laptops and other computing devices: learning outcomes could be preserved without antagonizing students. The unpublished experiences of one of the authors is that policies that ask students to put away personal computing devices such as laptops and smartphones, when not under the aegis of a research study on in-class research into the use of technology, causes significant resentment – itself an impediment to learning outcomes. Laptops, tablets and smartphones aren’t going anywhere. Classroom response systems have a proven record in improving learning outcomes. Post-secondary technical education will have to meet the challenges of ubiquitous computing. As demonstrated here, NetClick has the potential to make interactive teaching easier, more widespread, and may counterbalance the negative effects of technology in the classroom. R EFERENCES [1] M. Prince, “Does active learning work? a review of the research,” J. Engr. Education, pp. 223–231, 2004. [2] J. Freeman and A. Dobbie, “Use of an audience response system to augment interactive learning,” Fam Med, vol. 37, no. 1, pp. 12–4, 2005. [3] Z. Mateo, “Creating active learning in a large introductory statistics class using clicker technology,” in Data and context in statistics education: Towards an evidence-based society. Proceedings of the 8th International Conference on Teaching Statistics, Ljubljana, Slovenia. Retrieved from http://www. stat. auckland. ac. nz/˜ iase/publications/icots8/ICOTS8 9E4 MATEO. pdf, 2010. [4] R. Serow, “Research and teaching at a research university,” Higher Education, vol. 40, no. 4, pp. 449–463, 2000. [5] E. Dahlstrom, “The ECAR study of undergraduate students and information technology, 2012,” ECAR, 2012. [6] L. Chinaei and E. Lank, “A study on the use of clickers in classroom,” Under review, 2012. [7] H. Hembrooke and G. Gay, “The laptop and the lecture: The effects of multitasking in learning environments,” Journal of Computing in Higher Education, vol. 15, pp. 46–64, 2003. [Online]. Available: http://dx.doi.org/10.1007/BF02940852 [8] C. B. Fried, “In-class laptop use and its effects on student learning,” Comput. Educ., vol. 50, no. 3, pp. 906–914, Apr. 2008. [Online]. Available: http://dx.doi.org/10.1016/j.compedu.2006.09.006 [9] P. Thagard, “Banning laptops in classrooms: Can students multitask effectively?” in Psychology Today, July 2010 ed.
978-1-4673-6110-1/13/$31.00 ©2013 IEEE Technische Universität Berlin, Berlin, Germany, March 13-15, 2013 2013 IEEE Global Engineering Education Conference (EDUCON) Page 846
