Emergent Patterns of Teaching/Learning in ... - Semantic Scholar

Emergent Patterns of Teaching/Learning in Electronic Classrooms Draft: July 5, 1998 Ben Shneiderman ([email protected], 301-405-2680) Department of Computer Science, Human-Computer Interaction Laboratory, and Institute for Systems Research Ellen Yu Borkowski ([email protected], 301-405-2922) Academic Information Technology Services Maryam Alavi ([email protected], 301-405-2226) College of Business and Management Kent Norman ([email protected], 301-405-5924) Department of Psychology University of Maryland College Park, Maryland 20742 Short title: Emergent Patterns of Teaching/Learning Abstract Novel patterns of teaching/learning have emerged from faculty and students who use our three Teaching/Learning Theaters at the University of Maryland, College Park. These fullyequipped electronic classrooms have been used by 74 faculty in 264 semester-long courses since the Fall of 1991 with largely enthusiastic reception by both faculty and students. The designers of the Teaching/Learning Theaters sought to provide a technologically rich environment and a support staff so that faculty could concentrate on changing the traditional lecture from its unidirectional information flow to a more collaborative activity. As faculty evolved their personal styles in using the electronic classrooms, novel patterns of teaching/learning have emerged. In addition to enhanced lectures, we identified three common patterns: active individual learning, small-group collaborative learning, and entire-class collaborative learning.

- 1 -

Many questions concerning the use of instructional technology may be resolved as researchers and instructors begin to provide feedback from innovative pedagogical explorations. Proponents of computers in every classroom do sometimes blindly promote technology without adequate vision. Similarly, opponents of technology sometimes raise unrealistic fears of alienated students, displaced faculty, and corporate sell-outs (Noble, 1998). While instructional technology professionals may have more realistic understandings, inflated expectations and unrealistic fears are still widespread. The turbulence is likely to subside as all stakeholders become engaged in the process of choosing appropriate technologies, designing effective courses and curricula, training teachers, and setting new goals. We can only hope that wishful thinking and fear-provoking rhetoric will give way to clearer goals with well-conceived field trials accompanied by thoughtful evaluations. We believe the confusion is brought on by the large number of provocative new technologies and appealing philosophies that are being promoted (Hofstetter, 1995; Jonassen, 1994; Laurillard, 1993; NAS/NRC, 1996; PCAST, 1997). Some of the technologies are: - Email for individual discussions among students and instructors and email reflectors for entire classes - Bulletin boards, newsgroups, and listservs for archived asynchronous discussions, often with threaded topics - Chat rooms, MOOs, and MUDs for real-time interactions - Websites with digital libraries, learning resources, and interactive course materials - CD-ROMs or websites with simulation games and dynamic models - Specialized educational or general purpose software for home, lab, and classroom use - Video/audio conferencing for synchronous remote access - Electronic classrooms for lectures and face-to-face collaborations The plethora of technologies is matched by the diversity of pedagogical philosophies, including: - Distance education, or tele-learning, by which students reduce their need to travel and can participate synchronously or asynchronously with other students and professors (Alavi, Wheeler, & Valacich, 1995; Harasim, Hiltz, Teles, & Turoff, 1995; Hiltz; 1994; Neal, 1997) - Active learning and inquiry-based education, in which students investigate issues or solve problems with varying levels of human and computer guidance (NAS/NRC, 1996) - Collaborative and cooperative learning, in which short- or long-term teamwork supports the social construction of knowledge (Davidson & Worsham, 1992; Edelson, Pea, & Gomez, 1996; Sebrechts, Silverman, Boehm-Davis, & Norman, 1995; Slavin, 1990)

- 2 -

- Service learning, in which students work on projects on campus or in their communities (Jacoby, 1996) - Individualized, or self-paced instruction, in which students work on their own using computer software that guides their progress and gives feedback. The old terms such as "computer-assisted instruction" and "intelligent tutoring systems" have given way to "interactive learning environments" based on "learner-centered design" (Soloway, Jackson, Klein, Quintana, Reed, Spitulnik, Stratford, Studer, Jul, Eng, & Scala, 1996). Motivations for change are equally complex, including the need to keep up with livelier media such as television and to ensure that students are sufficiently proficient with computers to be employable. These motivators are broad, but when administrators sit down to plan, they split into two camps: those who argue for improved educational quality and those who see an opportunity for lower costs and larger markets (Gilbert, 1996). Advocates of quality often emphasize active learning and collaborative methods to promote greater student engagement with higher retention rates. Administrators who worry more about budgets are often attracted to self-paced instruction and distance learning, but unfortunately these are often euphemisms for computerized education and higher student-to-faculty ratios. It is remarkable that many decision makers are lured into the fantasy that teachers can be replaced by technology. Neither books, television, or videotapes have replaced faculty, but the seduction of "intelligent tutoring systems" has lured some commentators to believe that this technology is different. While the technology can be wonderfully empowering for teachers and students, the relationship between human beings is still the heart of the educational process. The central premise is that "knowledgeable teachers provide challenge, guidance, and evaluation. They build a motivating and supportive environment while attending to the diverse needs of each student. The successful teacher conveys enthusiasm for and competence in the subject and the process of instruction, earning trust by presenting the right level of challenges for each individual and team" (Shneiderman, 1998a). The best teachers create relationships in which students eagerly seek greater challenges and accept responsibility for their own education. Some students learn what they need by reading books, but a key function of a university or school setting is to enable the relationship between teachers and students to flourish. Teachers can employ technology to support the relationship directly by email and indirectly by providing students with access to remarkable resources (e.g., digital libraries or simulation models) and tools (e.g., word processor or music composition packages). Technology can support and strengthen relationships, but never create or replace them.

- 3 -

In addition to the controversy about the role of technology, another troubling aspect is the allor-none attitude of many discussants and administrators. They seem to promote a single approach and seek a single answer. Imagine if in 1910, you had to choose among trains, cars, or planes as the single mode of transportation for the future. Cars were riskier than trains because no system of highways was in place and airplanes were just too fragile to carry large numbers of people. Similarly, no one pedagogical formula will dominate. It is likely that instructors and administrators will mix and match technologies and philosophies as each is refined over time. Email is likely to be the personal car of the future. Most people will have it and use it daily. Listservs and websites might be the trains, with large numbers of regular commuters. Electronic classrooms might be the airplanes for special occasions because of their higher cost and complexity. These metaphors are playfully suggestive, but they help sort out some of the possibilities. Email is certain to have profound influence on education, commerce, and many aspects of daily life in the coming century. Listservs and websites seem likely to be widely used in creative ways. Electronic classrooms are more of a mystery (Bruce, Peyton, & Batson, 1993). Proponents believe that in the future every classroom desk will have a computer or at least a plug for the student's personal laptop. They envision classes in which students use the computers for much of the time in class to cooperate, send messages, or browse websites. Electronic classroom builders may create showcase environments that impress visitors or donors, but their pedagogical vision is often lacking and implementation flawed. Skeptics are troubled by the high costs and are likely to prefer traditional lecturing with student questions or group discussions. Given these concerns, it seems beneficial to conduct evaluations of electronic classrooms and the way that they are used by teachers and students.

Electronic Classroom Goals

The University of Maryland, College Park, has made a long-term commitment to develop electronic classrooms to explore how they can influence teaching and learning in a range of disciplines. Our major goal has been to provide faculty with an environment in which technology and a support staff can be used to enhance and transform teaching, from its traditional unidirectional information flow to a more collaborative teaching/learning process. Our focus is not on the technology but rather on its use as a tool for promoting effective learning (Shneiderman, Alavi, Norman, & Borkowski, 1995).

- 4 -

These collaborative goals were only vaguely incorporated into our early expectations of electronic classrooms, but experience has gradually refocused our vision. As in many research projects, new opportunities and ideas emerged as we learned what worked and what didn’t. Our goals were ambitious in that we were seeking to change a large community of faculty in diverse disciplines. To this end we prepared announcements and brochures, sent email notices, held seminars, and organized day-long Teaching with Technology conferences, to tell the story to our colleagues on campus and off. To date, 74 faculty have taught in these collaborative electronic classrooms, and most of them have rethought their philosophies of education. Some have gained new insights that inform their teaching even when in traditional classrooms, while a few passionately refuse to teach in traditional classrooms. Not surprisingly, most faculty have evolved personal styles in using the classrooms. While these styles reflect individual subject matters and personalities, they have in common a shift to a more collaborative approach. New patterns of teaching/learning have also emerged from the way these faculty have used the classrooms. Faculty have become guides and coaches, while students have taken a more active role in the classroom. In addition to enhanced lectures, we identified three common teaching/learning patterns: active individual learning, small-group collaborative learning, and entire-class collaborative learning. Some faculty emphasized active individual learning and reporting through individual students use of software tools to write, draw, simulate, search, etc., and then share their products by displaying them on the large screens in the rooms. Some faculty encouraged small-group collaborative learning by organizing small teams (usually 2- 5 students), while others emphasized entire-class collaborative learning by creating whole-class experiences based on brainstorming and groupwork. After overcoming initial problems with the technology and taking at least a semester to develop their personal styles, faculty report tackling more ambitious projects, giving their students more authentic experiences, and creating a higher level of engagement (Alavi, Wheeler, and Valacich, 1995; Norman, 1994a, 1994b; Shneiderman, 1993a). Many of the lessons learned are in harmony with the research on computing in education in elementary and secondary schools. Becker’s analysis of K-12 teachers found that exemplary computer-using teachers were located in a context of “collegiality among users, school support for use of computers in consequential activities, resources allocated to staff development and computer coordination" (Becker, 1994). He also found, as we did, that major positive curriculum changes accompanied the shift to computer use.

Electronic Classroom Infrastructure

- 5 -

The AT&T Teaching/Learning Theater, the first of our electronic classrooms opened in the Fall semester of 1991. It was made possible by a grant from AT&T to investigate how technology could improve the teaching process. The IBM-TQ (Total Quality) Teaching/Learning Theater opened in the Fall semester of 1993. This room was made possible by a grant from IBM as part of a TQM (Total Quality Management) in education project. The third classroom, the AITS (Academic Information Technology Services) Teaching/Learning Theater, was built in 1996 with existing building funds for the new building in which this classroom is located. Two more electronic classrooms are being built as part of a major construction project. The electronic classrooms are supervised and supported by the Teaching Technologies group of the University's Academic Information Technology Services. The mission of the Teaching Technologies group is to seek out new ways to use technology to improve the quality of education across the University campus (see also http://www.inform.umd.edu/TeachTech/). Academic units that have used the Teaching/Learning Theaters include American Studies, Anthropology, Art History, Business and Management, Chinese, Civil Engineering, Computer Science, Education, Electrical Engineering, English, Government and Politics, History, Housing and Design, Library and Information Services, Mathematics, Mechanical Engineering, and Psychology. Specifications The AT&T Teaching/Learning Theater has four tiered rows of five custom-designed desks, each equipped with a computer (see Figure 1). Each desk can comfortably seat two students for a total capacity of 40 students. The desks are cantilevered so that there are no supports to restrict chair movement; the front row of desks is wheelchair accessible. The computer monitors are recessed into the desk tops to permit clear sight lines with the instructor and other students. Five-caster swivel chairs promote interaction among students. The instructor’s console is designed to encourage experimentation with the layout of equipment and controls. A similar arrangement was used in the IBM-TQ Teaching/Learning Theater, which used a tiered double Ushape instead of rows (see Figure 2) (Allen, Bowen, Clabaugh, DeWitt, Francis, Kerstetter, & Rieck, 1996). Feedback from faculty indicated a preference for the double U-shaped configuration, therefore, the AITS Teaching/Learning Theater is in the same configuration as the IBM-TQ Teaching/Learning Theater. The original student computers in the AT&T Teaching/Learning Theater were 386-based AT&T 6386 WGS; these computers were upgraded in 1995 to Pentium-based AT&T Globalyst 620’s with 16 MB of RAM, 570 MB hard disks and 17” monitors (1024 x 768 pixels). The

- 6 -

instructor has two similarly configured computers at the front of the room. The original student computers in the IBM-TQ Teaching/Learning Theater were 486-based IBM Ultimedia computers with 16 MB RAM, 200 MB hard disks, and 14” color monitors; these computers were upgraded in 1997 to IBM Pentium-based 32 MB RAM computers. The instructor also has two similarly configured computers at the front of the room. The aITs Teaching/Learning Theater has Pentium-based Gateway 2000 computers with 32 MB of RAM, 2 GB hard disks, and 17” color monitors. The computers in each electronic classroom use Microsoft Windows, are networked on a local Novell network, and are connected to the campus optical-fiber network. Each desk in our electronic classrooms contains a monitor, keyboard, and mouse. The CPU’s in the AT&T and IBM-TQ Teaching/Learning Theaters are housed in a separate room -- an arrangement which reduces the noise level in the classroom, allows for better control of temperature for people and computers, improves security, and reduces equipment in the classroom. This setup also allows quicker and easier access for repair: a CPU can be replaced quickly, even while a class is in session. The monitor, keyboard, and mouse are run over long cables (up to 45’) strung under the floors and up through the desks. This setup does not allow students to have access to the floppy drives of the computers. Our network environment addresses this issue: each student who is enrolled in a class held in an electronic classroom is given an account and some storage space. These accounts can be accessed from any of the 30 open labs on campus. Printing support is not available in the electronic classrooms, but that capability is also available in the open labs. A single printer in the support room is available for faculty use. Audio/Visual Support A touch-panel control box for the audio/visual equipment and lights controls the equipment itself (e.g., starting/stopping the VCR) as well as what is displayed on the projectors. Available audio/visual equipment in each Teaching/Learning Theater include: 2 high-resolution rear screen projectors S-VHS video tape player U-Matic (3/4”) videotape player laser disc player broadcast television antenna connection to the campus video cable 35mm slide projector (video image) visual presenter compact disc (CD) player audiotape player stereo speaker system closed-captioning decoders A video recording system with three cameras, three ceiling-mounted microphones, and a wireless microphone allows instructors to videotape their classes and allows the teaching technician to monitor the classroom from outside the room. This system is linked to the

- 7 -

University’s campus cable system, allowing down-linked satellite programs to be viewed as well as allowing two rooms to be linked visually. A critical control box, called the LINK box (Applied Computer Systems, Inc., of Johnstown, Ohio), allows for shared viewing of computer monitors by providing the following capabilities: - The instructor’s screen can be broadcast to any or all of the student monitors. - Any screen on a student workstation can be displayed on the instructor’s monitor and broadcast to all the students’ monitors. - The keyboard and mouse on any student workstation can be “taken over” by the instructor’s workstation. Effectively, the instructor can take over control of any workstation in the room. - Any monitor’s display can be shown on the rear projection screens. - All student screens can be “blanked” by the instructor. Courseware When the AT&T Teaching/Learning Theater opened in 1991, little collaborative software for educational support was available. An on-line collaborative software product, a commercial group decision support system called VisionQuest (Intellect Corporation of Dallas, Texas) was adopted to enable faculty to devise innovative ways of using the software’s tools in support of collaborative classroom exercises. Later, another collaborative software product called GroupSystems (Ventana Corporation of Tuscon, Arizona) was installed. Over the years, the Teaching Technologies staff have developed “lectureware” tools as faculty helped the staff to discover what worked and didn’t work in the classrooms. These “lectureware” tools include the One Minute Paper, Feedback Meter, MultiChat, Class Directory, and Caprina. This software is also usable from the open labs on campus. • One Minute Paper enables the instructor to get anonymous contributions from the students. In response to a question, students can write a paragraph that they submit to the instructor who can display them to the class. In an International Business course, students learning commercial Spanish had to compose a definition of marketing using the recently taught vocabulary. The instructor could review the 20 submissions and show good and bad examples, anonymously. • Feedback Meter enables students to click on buttons to indicate whether they are following the lecture or are confused. Instructors get a summary of the number of students in each category. This has proven to be more valuable at appropriate stopping points rather than continuously during the class. • MultiChat allows students to “chat” with each other, anonymously or identified in small groups. An art history teacher integrates verbal and textual chat discussions of images that students find on web sites.

- 8 -

• •

•

Class Directory combines each student’s picture, name, and a personal biographic sketch into a format searchable by picture or by name. Caprina makes 15,000 high-quality color images easily accessible through a multislide program or a quiz slide program. Students can individually explore these images and then present them to the whole class as a slide show. Two faculty have developed their own courseware. HyperCourseware (Norman, 1994a, 1994b) is a system of interlocking modules that serves as a complete electronic infrastructure for classroom learning by providing templates for lectures, exams, grading, and interactive exercises (see http://www.hypercourseware.com). The TQ Classroom Software (Alavi & Yoo, 1995) consists of a number of integrated and easy-to-use student and faculty tools, including electronic information display and note taking, an electronic seating chart, and a set of classroom evaluation and feedback tools.

Proposal Process for Classroom Allocations To take advantage of the special capabilities of the rooms, all three electronic classrooms are scheduled through a proposal process rather than by the registrar. Courses that are scheduled into the electronic classrooms for the full semester must go through a proposal process that ensures they are used as designed -- that is, to take full advantage of an interactive, collaborative, multimedia environment to support teaching and learning. A Teaching/Learning Theaters Steering Committee reviews the proposals to ensure the goals of the classrooms are being addressed. Composed of a dozen faculty from the various colleges and administrative units on campus, the Steering Committee is chaired by the head of the campus Center for Teaching Excellence. A call for proposals to use the electronic classrooms is distributed to the entire campus at least one semester in advance (usually a year in advance). Acceptance is based on how the faculty plan to use the facilities available in the room to foster a collaborative learning environment. Faculty with proposals that show promise, but are weak in some area, are contacted for further discussion and refinement. Where possible, a faculty mentor is assigned to work with the proposer and the Coordinator of Instructional Technology & Support provides additional support and training. The Coordinator also resolves scheduling conflicts, supports short-term usage, and prepares demonstrations for the steady stream of visitors. Once the courses are selected by the Teaching/Learning Theaters Steering Committee, the faculty are contacted to arrange for training and/or software development. Support Structure

- 9 -

Clearly one of the keys to the success of the Teaching/Learning Theaters has been the support provided to the faculty and their teaching assistants (TAs) who use the rooms. Support has been an integral part of the electronic classrooms since the beginning: the guiding principle has always been that faculty should be able to come to a high-tech environment and focus on teaching the material for the day, not on fixing a “locked” computer (Yu & Gilbert, 1992). Instructional support is provided before and during the semester through a general orientation for all faculty who are selected to teach in the electronic classrooms and several individual orientation sessions. The Coordinator of Instructional Technology & Support provides as many individual orientation sessions as needed. Technical support is provided in every class throughout the semester. A student technician, assigned to each class, provides dayto-day support and is always present during the class. In addition, weekly “faculty prep” sessions are scheduled to provide time for faculty and their TAs to come to the electronic classrooms to work on materials or try software. Other Classrooms Many universities have been experimenting with electronic classrooms and lecture halls, language laboratories, and distance learning environments. Hiltz (1994) pioneered the use and research of virtual electronic classrooms supported by a computer conferencing system. Her focus is on asynchronous usage in a distance learning environment; her extensive survey, interview, and data collection methods are a model for all researchers concerned with electronic classrooms. The University of Arizona in Tucson has an electronic classroom equipped with 29 student workstations and three workstations for the faculty. The workstations are IBM 486-based computers interconnected via a Novell local area network. The Arizona classroom is furnished with a wide array of audio/visual equipment, including three 10-foot diagonal high-resolution rear projection screens (Alavi, Yoo, & Vogel, 1997). Other electronic classroom projects include Northwestern University (Guo, 1995), the University of Notre Dame (Stuebing, 1994), the University of Delaware (Hofstetter, 1995), and Gallaudet University (Bruce, Peyton & Batson, 1993).

Courses Taught in the Electronic Classrooms

Summary of use

- 10 -

A total of 264 courses (Table 1) [a current list is maintained at http://www.inform.umd.edu/TT/Schedules/Classes.html] have been taught by 74 faculty representing 26 different departments over their first seven years (Fall 1991 - Fall 1997). In Spring 1998, 23 faculty representing 14 different departments will teach 30 courses in the electronic classrooms. A total of 7,514 students have enrolled in classes taught in the electronic classrooms (Fall 1991 - Fall 1997). Average class size was 28 students. ---------------------------------------------------------------Insert Table 1 about here

---------------------------------------------------------------Faculty The 74 faculty and graduate students who taught in the electronic classrooms from Fall 1991 through Fall 1997, included 16 non-tenured faculty, 31 tenured faculty, 9 graduate teaching assistants, 8 staff, and 10 instructors. They represent the following 26 academic units on campus: American Studies History Anthropology Honors Art History Human Development (Education) Arts & Humanities Journalism Behavioral and Social Sciences Library and Information Services Business & Management Maryland English Institute Chinese Mathematics Civil Engineering Physics Computer Science Psychology Curriculum and Instruction (Education) Spanish English Speech Communication French Telecommunications Government & Politics Urban Studies and Planning The initial set of faculty who taught in the classrooms were very experienced computer users. Over the years, a mix of experienced and novice computer users have taught in the electronic classrooms. It is also interesting to reflect on those who have not used these electronic classrooms. A majority of those faculty who do use the Teaching/Learning Theaters come from the Colleges of Arts & Humanities, Behavioral & Social Sciences, and Business. In spite of our publicity, there are still many faculty who don't know about the electronic classrooms or believe that such

- 11 -

facilities are meant for software and computing-oriented courses. However, computer science faculty have been limited users, wrongly believing that it would take substantial effort to prepare programs. Resistance to innovation is an old theme, and many faculty continue to be skeptical or negatively pre-disposed to electronic classrooms.

Use of the Electronic Classrooms

Patterns of Teaching/Learning As faculty have evolved their teaching styles in the electronic classrooms, we recognized four emerging patterns of teaching/learning: - enhanced lecture style plus discussion - active individual learning and reporting - small-group collaborative learning and reporting, and - entire-class collaborative learning. These patterns were gathered from instructor comments, and then they were used to describe courses in our internal symposia and papers. Feedback from instructors and administrators gave us confidence that this was an adequate and useful list. The enhanced lecture style can be a faculty presentation enhanced by technology (for example Microsoft PowerPoint slides or an instructor-controlled animation) with discussion (questions or feedback from the students). By contrast, active individual learning and reporting involves student-initiated participation. Each student uses a piece of software to create his or her own materials. With the technology available in the electronic classrooms, each student’s work can then be shared with the rest of the class. During small-group collaborative learning and reporting, groups of 2-5 students solve a problem collaboratively or create a work product. As with active individual learning, the group’s product can be shared with the rest of the class. Entire-class collaborative learning is possible with certain software that allows the teacher to engage the whole class in an exercise based on brainstorming and teamwork. Examples of the enhanced lecture style range from using a word processor to using a presentation software package. Several faculty who were novice computer users chose to lecture in a presentation-only style using a word processor. As these faculty became more comfortable with the technology, they provided their lecture notes to the students electronically. A professor teaching Chinese poetry started in a presentation-only style but soon moved to a presentationwith-discussion style. He used PowerPoint for his presentations in each class, going through 5060 slides each period. PowerPoint provided him the capability to use color to emphasize

- 12 -

different aspects of poems and to show different translations side-by-side. The instructor provided the same slides to his students to follow during class and for review outside of class. As this instructor became more familiar with the technology, he added discussion to his lecture, integrating the use of the One Minute Paper tool to provide feedback to him on students' comprehension. In the past, he asked for verbal feedback, but he reported that only a few students would respond, while most sat with blank stares. In the electronic classroom, every student would compose a response, creating a much more active learning environment. The instructor could much more accurately gauge student comprehension across the class. With a computer at every desk, students have access to a suite of software for creating their own interpretations of the material being discussed. The LINK box provides the capability to share these interpretations with the entire class for discussion. These abilities fostered the emergence of the active individual learning and reporting style. Instructors from the Maryland English Institute -- which teaches courses in English as a Second Language (ESL), used this style frequently in class. A typical exercise consisted of showing a clip from a video tape or showing a picture on the electronic overhead and asking the students to write a paragraph in a word processor describing what they saw. The instructor used the LINK box to share what each student wrote anonymously with the class and to discuss grammar and writing style. This allowed students to learn from each other’s creative leaps and mistakes. This active learning with reporting pattern was also successful in the other courses. Similar to what was done in the ESL exercise with grammar and writing, a business instructor asked, students to revise a business letter, which was then shared with the class for discussion. Sciences and engineering faculty would have students use software specific to their disciplines to create products in response to a problem. These products were then shared with the class for peer critiquing. For example, Visual Basic was used to create novel user interfaces that were critiqued for layout, color, terminology, and completeness. Small-group collaborative learning and reporting were enabled through the use of software such as MultiChat. In a computer science course, in-class group programming was made possible by the use of the network in the electronic classroom. The professor composed a 20-line main program that invoked three procedures. The students then worked in small groups to generate versions of the procedures, and these were examined by the instructor using the LINK box. The instructor was able to copy the version he liked best, make a few slight changes, and then copy it into his main program on his machine. The program was compiled, and it ran on the first try. The instructor commented that “while he was enthusiastic, his students seemed to think that this was the natural way to do things.” Case-study exercises in business courses provide another example of small-group collaborative learning. Groups of 5-6 students were assigned business-oriented roles and given

- 13 -

individual case studies to analyze. VisionQuest was used to support these exercises, enabling the group members to propose solutions based on their role. Then the solutions were refined, rated, and ranked before each group shared its decision with the entire class for discussion. The collaborative software available in the electronic classrooms, especially VisionQuest and GroupSystems, has allowed for entire-class collaborative learning. VisionQuest’s "brainwriting" and "comment cards" tools were used successfully in many classes. A business school professor frequently starts her classes by having students use the brainwriting tool to identify concepts and issues from the assigned readings that, in their view, need further discussion or clarification in the classroom. Once the issues are identified, she asks each student to pick the two or three issues that are central to understanding the topic. She then focuses her lecture and class discussion on the issues most frequently picked by the class as well as on those that, in her judgment, are necessary to complete and enhance student comprehension. This process of issue identification and prioritization takes about 10 minutes. In a similar manner, a Spanish professor used VisionQuest to have students list troubling vocabulary words and then elicit feedback about how troubling the words were by asking the entire class to rank them electronically. Students seem to like knowing how well or poorly their classmates are doing. As another means of beginning the class discussion, a speech communication professor asked students to list metaphors for the concept of conflict in VisionQuest which she then used as illustrations during the class. A Spanish instructor used MultiChat to foster a combination of small-group collaborative learning and a form of entire-class collaborative learning. Groups of students had been assigned countries in South America to study throughout the semester. At the end of the semester, they used the MultiChat program to support a United Nations trade negotiation exercise: a UN room was set up and each country had its own chat rooms. Students could chat with the students in their group and make decisions on negotiation trade-offs and then go to the UN room (where the entire class participated) to negotiate with the other countries. Empirical Assessments Empirical analyses of courses taught in the Teaching/Learning Theaters are limited and exploratory. However, many faculty are eager to understand how the Teaching/Learning Theaters affect their courses and they are eager to improve their teaching methods. Although such studies fall outside the approach taken in rigorous studies of instructional design, they are helpful to faculty and they provide provocative insights into uses of electronic classrooms. One exploratory study involved a semester-long graduate business (MBA) course on management information systems (MIS) (Alavi & Yoo, 1995). The 20 students met once a week in the electronic classroom for two hours and forty minutes. Prior to each class, the instructor

- 14 -

used the note taking tool to create and distribute an electronic version of her lecture notes to the students. During the class, each student was able to retrieve and annotate the instructor’s lecture notes on his/her workstation. The student was then able to save the annotated notes on his/her private disk space for future reference. The classroom evaluation tools were used regularly throughout the semester to obtain anonymous student feedback on the classroom for the purpose of continuously improving the course content and delivery mechanism. VisionQuest software was frequently used by the class for brainstorming, evaluating alternatives, and collaboratively analyzing business cases. At the conclusion of the course, the students were asked to respond to a questionnaire consisting of 25 structured items and two open-ended questions. The results indicated that the students rated their learning effectiveness in the electronic classroom significantly higher (significant at .05 level) than in the traditional classroom on all the items. Furthermore, the students were highly satisfied with their experience in the electronic classroom, indicating that they would take another course there. They reported that the computer activities in the classroom were well-planned, well-organized and enjoyable. The students identified electronic note taking as the most popular feature. They responded favorably to the room's allowance for interactivity, idea sharing, brainwriting, and multimedia. Students also appreciated the computer/technology taking care of such mechanical aspects as saving notes, displaying the syllabus, etc. Alavi (1994) conducted another study that compared the electronic classroom and a traditional classroom according to the learning outcomes and student evaluation of the learning process. The study involved 127 graduate business students, 79 of whom attended class in the electronic classroom (2 classes) and 48 of whom attended class in a traditional classroom setting. All three classes were taught by the same instructor. The course met once a week for a period of two hours and forty minutes. The instructor followed an identical set of classroom protocols and procedures was in each of the three classes. The primary difference between the courses was the use of VisionQuest in the electronic classroom to support student interactions during the collaborative learning group exercises. A post-course questionnaire with 28 items was used to measure students’ perceptions of their learning and classroom experience. Five-point Likert-type scales were used to measure all items. The learning and evaluation items were subjected to separate principal component analysis, followed by varimax rotations. Results indicated the presence of five factors (70.9 percent variance explained) for the learning items and three factors (67.1 percent variance explained) for the evaluation items. Means, standard deviations, and correlations of the five coherent scales with acceptable alpha reliabilities (3 from learning and 2 from evaluation) are presented separately by experimental condition in Table 2. Alpha reliability coefficients for each scale are

- 15 -

provided in the diagonal. Means, standard deviations, and intercorrelations with other scales for the expected grade measure are also presented in Table 2. ---------------------------------------------------------------Insert Table 2 about here -----------------------------------------------------------------------------

The findings of the study indicated that technology-mediated collaborative learning in the electronic classroom can lead to statistically significantly higher levels of perceived skill development, self-reported learning, and evaluation of classroom experience in comparison to collaborative learning in a traditional classroom. Furthermore, the final test scores of the group of students who were in the electronic classroom were statistically significantly higher (electronic classroom M=88.23, traditional classroom M=83.97, t(125)=-3.92, p