Nicolai Scheele. 1. , Anja Wessels. 2. , Wolfgang Effelsberg. 1. ,. Manfred Hofer. 2. & Stefan Fries. 2. 1 Department of Computer Science, University of Mannheim.
Experiences with Interactive Lectures – Considerations from the Perspective of Educational Psychology and Computer Science Nicolai Scheele1, Anja Wessels2, Wolfgang Effelsberg1, Manfred Hofer2 & Stefan Fries2 1
Department of Computer Science, University of Mannheim {scheele,effelsberg}@informatik.uni-mannheim.de 2 Department of Educational Science II (Pedagogical Psychology), University of Mannheim, {wessels,hofer,fries}@uni-mannheim.de
Abstract. The conventional lecture scenario implicates fundamental didactic problems due to a lack of interactivity and opportunity for feedback. In an interactive lecture, each student is equipped with a lightweight, mobile device that can be used to wirelessly interact with the lecturer during the lesson. This creates an additional channel of communication. In this paper, we present our experiences with this new scenario over the last three years. After discussing the benefits of interactive lectures, similar projects, and possible mobile devices, we introduce the softwaretoolkit used in our scenario and present a selection of results from over our six major studies. Keywords: interactive lectures, ubiquitous computing, wireless communication, blended learning
INTRODUCTION In Germany, governmental and scientific institutions (e.g. Bundesministerium für Bildung und Forschung, 2001; Fraunhofer-Gesellschaft, 2002) have launched several programmes to introduce the use of new media into the system of higher education. These aim to improve the quality and effectiveness of teaching in universities through the use of multimedia elements. Appropriate new teaching media and learning procedures should help to achieve a better adjustment to the individual learning needs, learning rates, and time budgets of students, as well as afford the instructors more flexibility in their teaching. But despite the various multimedia projects and the efforts on the part of dedicated instructors, the introduction of educational media has led to an only partial modernisation of the universities. This is particularly evident in the classical university teaching-learning scenario: the lecture. Lectures in universities have profited from many technical advances over the last few years. Blackboards were replaced by overhead projectors, which, in turn were substituted by video projectors and electronic whiteboards (Geyer & Effelsberg, 1998). Most lecture halls nowadays are equipped with computers, as well as video and audio systems, thus allowing the integration of every possible type of media into the lecture. Nonetheless, the basic teaching paradigm continues to remain largely unchanged; one of the few exceptions is the scenario of the telelecture (e.g. Datta & Ottmann, 2001) or digitally recorded lectures (Zupancic & Horz, 2002). The main disadvantage of lectures is their lack of interactivity: Conventional lectures can be characterized as situations in which a teacher presents new information to the learners without guiding their learning processes. The limited opportunities for interaction in lectures engender a set of problems regarding students’ attention and motivation, as well as the adaptivity of the lecturer’s instruction. Lecturers often attempt to overcome such problems by asking questions, to trigger feedback on how well the students have understood the presented material, as well as to provoke their active participation in the meetings. This is problematic in lectures with a large audience, since only a few students are able to interact with the lecturer in this way. The overwhelming majority will not benefit from this form of interactivity. Further problems arise if the lecturer wants to get feedback on how the students accept the lecture or their suggestions for improvements. In lectures with a small audience, the teacher can typically deduce this information from the students’ reactions, e.g. bored expressions. In meetings with large audiences, however, this information is usually gathered by passing out feedback questionnaires to the students at the end of the lecture period. Unfortunately, this approach is rather imprecise and does not allow any assessment of individual lecture elements. Nor can the lecturer react quickly to problems. Other forms of interactivity are spontaneous questions by the students. These can often be difficult in large lectures. First of all, due to time constraints, not all students
547
are able to ask questions. Secondly, many do not dare to ask questions in front of a large audience. Finally, if students can pose questions only at certain times, these will be out of context when finally put. Given all these problems, most students do not interact at all during the lecture. Thus, despite the possible use of different media to illustrate lecture topics, interaction is hardly possible in mass meetings. This unidirectional communication leads to several motivational and cognitive problems: From a pedagogic-psychological view, learning (in lectures) has to be reconstructed as an active process (e.g. Ernest, 1995; Jonassen, 1994; Honebein, 1996; Wilson & Cole, 1991). Interactivity represents an opportunity for the learner to take a hand in shaping the informational, communicational and learning processes, rather than remaining a passive recipient; thus, the active involvement of the learners has a great impact upon successful learning (Ramsden, 1992). In respect of the learning success in lectures, empirical results state that while lectures are not generally ineffective, they still are not suitable for a global knowledge transfer (see for an overview Gage & Berliner, 1996; Peterson, 1979). Directly connected to the problem of the low interactivity in this method of teaching is the lack of adaptivity of the teacher’s behaviour: During the lecture, the instructor can only adapt a limited portion of the contents or topics of his lecture to the learners’ state of knowledge. On the other hand, in the instructional-psychological context, adaptivity is an essential tool to improve the learning process. The underlying rationale is to adapt explanations or curricula to the learners’ current state of knowledge, thereby achieving greater efficiency and efficacy of instruction. Empirical findings reveal the effects of diverse learning-centred measures upon learning success (Sass, 1989; Cronbach & Snow, 1977; Bligh, 1971). Finally, an essential problem in lectures is the continuous attention required of the learner, usually over 90 minutes. This requirement is not realistic: Usually, the attention span is only about 20 minutes (Smith, 2001). Subsequently, an activity change must follow if students are to maintain their attention (e.g. a change from the lecture to the discussion phase). Studies show that a decreasing mental performance is responsible for the inferior knowledge acquisition (e.g. Siegel, Siegel, Capretta, Jones & Berkovitz, 1963; Bloom, 1953). However, such activity changes are not foreseen in the classic scenario, and if so, they depend exclusively on the ability of the lecturer (Ramsden, 1992). In dominant instruction models (e.g. Glaser, 1976; Rosenshine, 1979), the diagnosis of the learners’ knowledge status by the teacher is a central element of the educational process: Each instruction cycle contains two diagnostic elements: the diagnosis of relevant learners’ characteristics and the diagnosis of the learning achievement, for the planning of the further educational procedure and/or the examination of success in learning. Thus, learner feedback enables an adaptive teaching behaviour, which can lead to an improved learning process. Lecturers, for example, can adapt explanations or contents to the learners’ knowledge status, in order to heighten the efficiency and effectiveness of their instruction. Directed interactivity can strengthen learner-centred instruction. This is further strengthened if the learners feel they can communicate with the lecturer and give him feedback, thus have a real opportunity to affect the learning process - despite the rather passive role as receptive learners. Since successful learning represents an active process, active involvement of the learners should have a large effect on learning. Effectively increased interactivity should promote students’ attention and motivation and, finally, support their acquisition of knowledge. The pre-condition is that lecturers receive more exact information, to enable a micro adaptation of their presentation. Hence, there are evident theoretical and practical reasons to improve this type of learning scenario or to create a new (more interactive) scenario as a replacement. An innovative approach to improve interactivity and to realize bi-directional, synchronous communication in lectures is to equip the students with small electronic devices, such as handheld computers. These devices communicate with the lecturer’s computer, thus allowing the exchange of information with the lecturer at any time, without disturbing the lecture. The type of information exchanged can be arbitrarily complex, ranging from a simple "virtual hand-raising" over detailed feedback to quizzes that may even be counted towards the grades of the students. To avoid costintensive modifications of the lecture hall, the handheld PCs and the server are connected by a wireless LAN. Two departments at the University of Mannheim (Computer Science and Educational Science) have initiated the LectureLab project (http://www.lecturelab.de) to create a new form of multimedia-enhanced teaching: the interactive lecture. We have designed and implemented a full-featured software system and carried out several major field studies to evaluate this concept. In this paper we will: x describe the possible use of mobile devices in large learning environments; x present the scenario of the Interactive Lecture, as well as our own technology at the University of Mannheim, WIL/MA (Wireless Interactive Lectures in MAnnheim); x discuss the results of two detailed case studies conducted among graduate students, focusing on the comparison between the use of PDAs and notebook PCs; and
548
x
give an overview of the results of six experimental field studies we carried out in computer science and educational science lectures in order to investigate the motivational and cognitive effects of this new teaching-learning-method.
MOBILE DEVICES IN LECTURES Numerous projects focusing on the use of mobile devices in lectures in order to enhance learning and teaching have evolved over the last few years. Most of them have very unique ideas about what aspect of the lecture they intend to improve, and about how to cope with eventual problems. The following is a short list of past and ongoing projects, along with a short description of the basic ideas behind them. Classtalk (Abrahamson, 1999, 1998; Webking, 1998; Dufresne, Gerace, Leonard, Mestre & Wenk, 1996) is a well-known Classroom Communication System by Better Education Inc.. For the better involvement of every single student, the teacher “beams” three to four Classtalk tasks per lesson to the students’ devices; these can be calculators, organizers or personal computers, and the students often own them. A "task" can be anything from a simple question to a midterm exam, from a group exercise to a survey of class opinions. The results are displayed immediately on the teacher’s notebook PC; the teacher can either keep them confidential or show them to the class. The class sessions can be archived for review, and can be analyzed and compared to other sessions. Additional features include feedback (from the teacher), tests and grading. Classtalk can also be used for the so-called „peer-instruction“-method, a kind of collaborative learning in which answers to several questions („ConcepTests“) are to be discussed in small learner groups (Mazur, 1997). A questionnaire study by Hake (1998) with more than 6 000 American physics-students showed that interactive learning environments (Classtalk vs. traditional instruction) enhanced the students’ capability to solve problems. Abrahamson (1999) could also assess an increase of the knowledge gain if interactive elements were integrated into instruction. Another study, in the Netherlands (Massen, Poulis, Robens, & Gilbert, 1998), showed that with physics students, the integration of the so-called Audience Paced Feedback (APF), a system comparable to the Classtalk system, leads to an enhancement of learning success. Finally, Hartline (1997) stated that using Classtalk improved the reading comprehension of elementary school pupils. ClassInHand, from Wake Forest University, turns a PDA equipped with a wireless adapter into a presentation controller and a quizzing-and-feedback device for the lecturer (http://classinhand.wfu.edu/, last checked 11/10/04). Its major components are a presentation control application and a web server for the PocketPC of the teacher; the clients only need a web browser to participate. The Presentation Control allows remote control of the Powerpoint slides on the lecturer’s PC. It also gives him the possibility to forward the quiz results to the class. The Web Server enables concept tests (quizzes), textual feedback, a feedback meter, and easy document posting. The quiz feature can be used to present a question with up to four answers, and to view the results immediately on the PDA or to forward them to the students’ devices. The textual feedback component allows students to send their questions directly to the teacher's PDA. Finally, the feedback meter enables students to submit numeric responses (range: -10 to 10). ConcertStudeo, a project of the Fraunhofer Institute IPSI, uses an electronic blackboard combined with handheld devices (Dawabi, Dietz, Fernandez & Wessner, 2003). It features exercises and interactions, such as multiple-choice quizzes, brainstorming sessions, queries, or role-plays. During a lecture, the teacher introduces the exercise, and the learners enter their answers into their handheld devices. The software does the collection, analysis, and presentation. Specifically designed for online feedback is CFS (the Classroom Feedback System), from the University of Washington (Anderson, Vandegrift, Wolfman & Yasuhara, 2003). It allows students to post annotations directly on lecture slides. The lecturer sees the annotations in real-time. The students use their notebook PCs to generate their feedback by clicking a location on a slide and selecting a category from a fixed menu (such as “more explanation”, “got it”, “example”). The teacher’s screen shows the number of feedback requests for each slide, and displays the aggregated feedback with a shaded dot for each annotation at the actual presentation slide. The slides depict categorical information by colour (e.g., red for “more explanation”), and the slide context by location.
THE WIL/MA SOFTWARE As shown, there are many different settings and ways to take advantage of mobile devices to improve interactivity in the lecture hall. Most of the earlier work has focused on specific issues, such as quiz only, online feedback only, or annotations only. Furthermore, the software is often designed to run only on a particular hardware device or, in some cases, only on very proprietary hardware. Finally, none of the existing projects features synchronous, bi-directional communication for large classrooms: in most cases, it is only the students who can send data to the teacher at any time, while the teacher cannot send personalized responses.
549
Our software tools attempt to solve these problems: the same basic software architecture accommodates many different interactivity services. The system is written in Java and is portable to almost all modern mobile devices. System Architecture The WIL/MA system is designed as a classical client/server application (see figure 1). As the central part of the architecture, the server provides all the fundamental functionality: management of the connections, users, and services. Connection management establishes connections to the clients upon request, processes incoming and outgoing data, and monitors the registered connections for broken links. User management identifies individual users via password and stores personal information for internal and external use. Service management dynamically loads a requested number of plug-in service modules, informs clients about the availability of certain services, and controls the data flow between the services within the server structure itself and between clients.
Figure 1: The WIL/MA architecture All functionality that is visible to the users is bundled into services. Services are built as independent modules to be loaded by the server and the clients at start-up time; for each service, there is a server-module, a teacher-module and a student-module. The server-modules are the central part of a service. They aggregate all incoming data, analyze the information, and broadcast trimmed data packets in various ways back to the teacher and each individual student. The server software provides a sophisticated messaging system for this purpose. All other modules are loaded into the clients of the students and the teacher. While the teacher-module focuses more on editing various aspects of the service, as well as on the display of analyzed data, it is more important for the student-module to display prepared material appealingly and to provide an intuitive user interface. The client for the lecturer runs on a machine typically connected to the server via a wired network; all other clients use the wireless LAN to connect to the server. By means of interface utilities, multiple servers can be connected to extend the range of an interactive lecture to other lecture halls easily, without overloading the network in-between. Interfaces to other similar software systems can be created to share data of common services. Besides the already discussed functionality, the server software also provides several tools to easily manage a larger number of students' devices. The two most important features are a DHCP server, allowing the central configuration of all network-related parameters, and a Java class server that offers all required Java classes for download. Thus, only a very small footage of classes has to be installed on the students' computers, while all other classes are loaded automatically at start-up. The class server can also be used to update all mobile devices whenever the software is changed (e.g. new releases, updates). Implemented Services Three services have been implemented so far: a quiz tool, an online-feedback tool, and a call-in tool. The quiz tool allows the teacher to pose questions (that possibly include graphics or animations) about actual lecture contents and “beam” them via wireless LAN to the audience. The students work on them and send their answers back to the lecturer’s computer. After a timeout, the cumulated results are presented graphically on the projector. In this way, the lecturer and the students gain representative feedback on the newly acquired knowledge. Apart from two different multiple-choice question styles (only one correct answer, multiple correct
550
answers), we integrated other optional question types into this service, which can be automatically analyzed. To give some examples: Clickable images can be used to ask the student to point into a certain area of a picture as an answer (for example: "point at the location of Moscow on a map of Russia"). Fill-in questions make it impossible for the student to accidentally guess the right answer to a mathematical exercise. The feedback tool delivers direct and systematic feedback about different aspects of the lecture from all students to the lecturer, who can then instantly adapt his/her presentation style to the new situation. An aspect or category - could be the speed or the level of the lecture; so students can ask the teacher during the lecture to progress more slowly or to discuss a certain topic in more detail. Technical issues also can be used as a feedback category; for example, video or audio distortions in telepresence scenarios can be discovered much sooner, or the students are able to complain, when their learning environment is suboptimal (because other students in the back rows are too loud ,or bright sunlight makes it impossible to read the projected lecture slides).
Figures 2a, 2b, and 2c: Screenshot of the students' client, showing a quiz, feedback, and call-in
Figures 2d, 2e: Screenshot of the administrator client and a typical quiz result Finally, the call-in tool forwards spontaneous text questions to the teacher at any time during the lecture. The questions are stored in a list and can be dealt with in three ways: using the software, they can either be answered individually, or the answer can be sent to all students if the question is of general interest (of course, the anonymity of the original student is maintained). In these cases, FAQ lists can be created, which are then put on the Web for the next generations of students. The third way is to integrate questions or remarks from students into the lecture. A selection of screenshots from the teacher's client, as well as from the students' client, can be seen in figures 2a-e. Group Support From the software-engineering perspective, there are three types of group support to be considered in an interactive lecture: device sharing, working groups and distributed working groups. Device sharing is particularly interesting in lectures, where many students want to participate but only a limited number of devices are available. In this case, the software could allow multiple students to log in and then select their name before accessing a certain service. This way, each student in a group still acts as an individual, from the server perspective. In quiz rounds, for example, the students could work out their answers
551
on a piece of paper and then use the device only to send them to the server. The ConcertStudeo software is one of the few projects that support device sharing. Most projects - including WIL/MA - do not because of some severe problems with this feature (for example, the second student in a row can easily copy the answers of his predecessor). Furthermore, the second type of group support is an easy, but feasible, alternative: A group of students collaborates using a single device. The software has not to support this explicitly because there is little difference between a working group and an individual student when using only one login account. The third type of group support is much more interesting because it offers a wide range of possibilities. In this scenario, students are able to form groups or are put into groups, but still have their own individual device. This way, they can still act individually in some services (feedback or call-in). In other services (quiz, or online brainstorming, for example), the server specifically aggregates the individual input, to form a homogenous group input. This technique allows the formation of groups over wide distances, connecting students who don't know each other, or in crowded lecture halls, where group members often cannot sit next to each other. The groupbuilding process is also much more interesting: students can advertise their skills in a list and can be invited by a group that lacks these skills. Or groups can be formed automatically using various heuristics, thus bringing together students of equal or complementary knowledge. Obviously, this kind of group support places high demands on the software system. First of all, the students in a group have to be able to communicate. The communication must be easy to handle, must not disturb other students, and should be blended into the standard screen of the service as seamlessly as possible. During a quiz, for example, the students would see little coloured dots next to the answers that their fellow group members think to be correct. The brighter the dots are, the more confident is that group member of his or her selection. Whenever there is a disagreement, the students can switch back to a VoIP or chat screen to discuss the final answer. Handling unresolved disagreements is a second demand on the software. The single analysis step of a system that only supports individual input has to forego a pre-analysis step, where heuristics decide the final input of a group in case of discrepancies. Using the Tools within an Interactive Lecture To dispatch an interactive lecture, only three devices are needed: a single access point is usually sufficient to handle the connections from more than 100 students. The server software can be run on any computer running Java; if it is installed in the lecture hall, a standard notebook is sufficient. This computer is usually used for the teacher software, as well. Finally, a projector is needed to display the aggregated results of several services (primarily the quiz service) to the audience. Since most of the lecture halls are already equipped with projectors or large monitors, all equipment needed fits into one notebook travel bag and is installed and started in usually less than 5 minutes before the lecture starts, including the time needed to start the appropriate software. Of course, it is quite time-consuming to hand out several dozens of PocketPCs to students who do not own a mobile device. This requires some assistance; but in our experience, the students are quite disciplined, and all the devices are usually treated very carefully. Furthermore, more and more students own a PocketPC or Palm or would like to use their notebook in the lecture anyway. Hence, it is most likely that having to provide large pools of mobile devices will no longer be an issue in the very near future. Once the software is started, the students will begin to log in. In our case, the services: feedback and call-in are started right at the beginning and are thus accessible the whole time. Quizzes are scheduled approximately every 30 minutes. In our interactive lectures, the students thus had two breaks for the quiz rounds in each lesson of about three to four questions, which proved to be very effective. Also, a good practice may be to start with a short quiz, to see what the students have learned in the last lecture. The questions are prepared before the lecture and submitted to the students at an appropriate time during the lecture. Depending on the difficulty of the questions, the students are given three to five minutes to answer them; the discussion of the results usually takes another five minutes. The results of all services are stored on the machine running the teacher’s client in a portable XML-based format, so that the teacher can analyze the information at any time later.
EXPERIMENTAL FIELD STUDIES AND EXPERIENCES Altogether, we have conducted six experimental studies to date in order to investigate the motivational and cognitive impacts of this scenario (assessed in the form of questionnaires with respect to acceptance, and through the application of tests with respect to the learning success). Four of the studies were carried out in
552
computer science lectures, and two in education science meetings. A seventh study (in education science) is still running. In the following chapter, we will give a brief overview of our studies’ results, separated by the different faculties where the interactive lectures took place. Interactive Lectures in Computer Science First, a test trial (winter semester 2001/2002) of the interactive lecture scenario was carried out (Wessels, Fries, Horz, Scheele & Effelsberg, submitted). In an experimental study (2x2 design), a first prototype of the WIL/MA tools was technically and empirically tested in a computer science lecture by comparing two wireless LAN-supported sessions with two conventional lectures on the same topic. The 44 randomly assigned students at this lecture each participated in both an interactive and in a conventional lecture session, then the groups were compared with respect to acceptance of the teaching method (questionnaire based on 13 items) and success in learning (pre-post measures). Regarding the acceptance, the interactive condition was evaluated significantly better than the conditional one (p
