Key words: collaborative learning, computer-mediated communication, ..... judge had an advanced degree in the information systems field and previous ...
Group Decision and Negotiation 8: 427–440, 1999 ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION 427 © 1999 Kluwer Academic Publishers. Printed in the Netherlands
Asynchronous Computer-mediated Communication versus Face-to-face Collaboration: Results on Student Learning, Quality and Satisfaction ROSALIE J. OCKER AND GAYLE J. YAVERBAUM School of Business Administration, Pennsylvania State University at Harrisburg, 777 W. Harrisburg Pikem Middletown, PA 17057-4898
Abstract Although there has been more than a decade of literature on computer-mediated communication in education, the research has been unclear as to whether it is an effective replacement for face-to-face (FtF) collaboration. This study sought to add to this body of research by exploring the effects of two modes of collaboration on student groups. Following a repeated-measures experimental design, each student group collaborated on two case studies, one using face-to-face collaboration and the other using asynchronous computer conferencing technology as a means of collaboration. Empirical findings indicate that asynchronous collaboration is as effective as faceto-face collaboration in terms of learning, quality of solution, solution content, and satisfaction with the solution quality. However, students were significantly less satisfied with the asynchronous learning experience, both in terms of the group interaction process and the quality of group discussions. Key words: collaborative learning, computer-mediated communication, computer conferencing
1. Introduction With ever-increasing amounts of information for students to process, it is no longer acceptable for colleges and universities to only teach facts. These institutions are being challenged to equip students with the skills necessary to be life-long learners. The focus now is to develop students’ reasoning skills so that they can synthesize, analyze, and integrate material in the process of learning (Elder 1991). To meet these challenges, many faculties have moved away from a teacher-centered, lecture format for dispensing knowledge to a collaborative, student-centered environment for creating knowledge. Collaborative learning involves small groups of students working together to actively solve assignments. Students typically collaborate via a series of face-to-face meetings. Generally, students are given a certain amount of time during scheduled classes to collaborate with group members, and must meet with one another outside the confines of the classroom in order to complete collaborative assignments. To date, some 500 studies of collaborative learning have found that students working in a collaborative environment experience a higher degree of learning accompanied by increased levels of satisfaction with the learning process and the learning outcomes (Johnson and Johnson 1989).
428
OCKER AND YAVERBAUM
As methods of learning are changing, so are students. The profile of today’s college student is significantly different compared to that of the past (U.S. Department of Education 1996, 1997; Levine & Cureton 1998). As of 1993, nearly one-quarter of all college students had full-time jobs. As of 1995, 44% of students were at least 25 years old, 54% were working either full or part-time, and 43% were attending school on a part-time basis. Today, less than one out of six undergraduates can be considered a traditional college student – one who is between the ages of 18 and 22, attends school on a full-time basis, and lives on-campus. School is not central to the lives of these nontraditional students, but one of many things in which they are engaged. These students are busy juggling family and job responsibilities with school. To complicate matters more, students often do not live in close proximity to their schools. Thus, in a learning environment consisting of many nontraditional students, collaboration outside of the classroom – and thus a collaborative learning environment – is impractical and difficult to achieve, if not impossible. Given these circumstances, what modifications can be made to curricula to accommodate nontraditional students? How might instruction methods be changed to assist these students with the task of learning? Some researchers are addressing these questions by exploring how new technologies can serve the needs of the nontraditional student in a collaborative learning environment. In this paper, we report on a study conducted to compare two different modes of collaborative learning. Students collaborated on two team assignments using traditional face-to-face (FtF) communication for one assignment and asynchronous computer conferencing (CC) for the other assignment. The following research question was explored: Is group collaboration via asynchronous CC a good substitute for FtF collaboration? The effectiveness of both modes of collaboration is measured in terms of learning, solution quality, solution content, and student perceptions regarding satisfaction with the learning experience.
2. The Collaborative Model of Learning With the traditional lecture model of instruction1, instructors dominate classroom activity and only a small portion of most classes elicit student interaction (Karp and Yoels 1983). On the other hand, collaborative learning is based on students actively participating to achieve learning. Students work in small groups to accomplish collaborative assignments such as solving complex problems, composing essays, and debating and discussing issues (Slavin 1987; Johnson and Johnson 1975). Although they collaborate to deepen their knowledge of the material, collaboration is inherently social (Golub 1988). In addition to social interaction, there are several key attributes of the collaborative model. Collaborative learning focuses on the role of peer relationships as a key component of educational success (Johnson 1981). Rather than knowledge being transferred from expert to learner, it is actively created in the learning community (Whipple 1987; Alavi 1994). The learner participates in the construction of knowledge by formulating ideas into words, and these ideas are built upon through reactions and responses of others (Bouton and Garth 1983). Thus, interaction among students is an important principle of the collaborative approach to learning (Benbunan-Fich 1997). Small groups of students
ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION
429
cooperate in a team atmosphere that provides an opportunity for clarification and feedback from peers as well as exposure to alternative views (Glasser and Bassok 1989).
3. Computer Conferencing in a Collaborative Context Face-to-face learning techniques have been the norm for collaborative experiences for many years. However, the integration of technology within the academic community, especially through computer conferencing, has expanded the possibilities for collaboration. Using a computer conferencing system, text-based discussion and interaction is facilitated by sophisticated software residing on a host computer connected to a network(s). In its asynchronous form, computer conferencing is an online communications medium that allows participants to exchange ideas and information regardless of the time of day or the distance between users (Tooey and Wester 1989). Computer conferencing is similar in concept to electronic mail, however, special software structures exist to support collaborative group work. Various benefits are associated with computer conferencing. The debate of issues and clarification of concepts is facilitated by conferencing (Hiltz 1986; Morrison 1992), as time and space barriers of the traditional classroom are removed (Bonk et al. 1996). The asynchronous nature of computer conferencing gives group members time to reflect before contributing (Aiken 1993; Eastmond 1995). Due to reduced social presence (Short et al. 1976), students concentrate on the message rather than the presenter (Harasim 1987). Conferencing provides a forum for students who ordinarily refrain from classroom discussion (Morrison 1992), so that there is more equal participation of students using computer conferencing (Berge and Collins 1993). Computer conferencing also has its limitations. Due to asynchronicity, coordination can be difficult as group members must agree on how often and when communication should occur (Abrami and Bures 1996). The relative anonymity of conferencing can result in some group members exerting less effort (Abrami and Bures 1996). Lurking can occur, whereby group members read the electronic discussions, but do not contribute (Hiltz 1994). The lack of non-verbal cues may diminish “social presence” (Short et al. 1976), resulting in a sense of depersonalization (Hiltz 1986). Additionally, some students dislike the text-based nature of conferencing because of the increased time it takes to type messages and read other people’s messages (Eastmond 1995). As is apparent from the literature, asynchronous communication has strengths and weaknesses. When considering asynchronous CC as a substitute for FtF collaboration, the question is whether the benefits of computer-mediated communication (CMC) outweigh the limitations.
4. Hypotheses Over the last two decades of research, the use of asynchronous technologies has not been studied to the extent of synchronous technologies (Fjermestad and Hiltz 1999). Furthermore, concerning the impact of computer-mediated communication in the
430
OCKER AND YAVERBAUM
educational domain, “rigorous experimental studies . . . are a rarity” (Wells 1990, p. 3). Most researchers report only anecdotal experiences concerning instructors’ and students’ perceptions of the learning experience (see Alavi 1994 and Hiltz 1986, 1994 for notable exceptions). While this information can shed some light on the impact of~cmC, experimental results based on strong measures of objective as well as subjective variables are necessary. Below, based on the results of the few existing experimental studies comparing asynchronous collaboration to FtF collaboration, hypotheses are developed concerning the effect of communication mode on learning, quality of task performance, and student perceptions regarding satisfaction with the outcome and the process.
4.1. Learning During the last decade, Hiltz (Hiltz 1986, 1988, 1994; Hiltz and Wellman 1997) has gathered experimental results concerning the use of asynchronous computer conferencing in conjunction with video-taped lectures to deliver courses over a distance. No significant differences of learning as measured by student grades were found between traditional (FtF) courses and distance courses using asynchronous computer conferencing. In a field study comparing asynchronous groups communicating via computer conferencing to unsupported FtF groups, researchers found that there was no difference in students’ perceptions of collaborative learning (Benbunan-Fich and Hiltz 1999). Additionally, there was also no significant difference in actual learning, as measured by grades on the final exam (Benbunan-Fich 1997). In an experiment comparing FtF teams to teams consisting of students from different universities using CMC to complete a written case analysis, learning outcomes were measured in terms of student perceptions only. It was found that CMC groups were not significantly different from FtF groups (Scrifes 1998). In a study examining whether supplementing FtF discussion with CMC discussion enhances the academic performance of students in large lecture courses, Althaus et al. (1997) found that students in CMC discussion groups reported learning more and also earned higher grades than students using FtF discussion only. These results are in contrast with the previously cited studies. However, this study was a quasi-experiment where students were not randomly assigned to conditions. The researcher states that, due to students self-selecting to use CMC, it is impossible to tell whether students earned higher grades because of using it, or whether high-achieving students joined the CMC groups. Therefore, given this evidence, we hypothesize that: H1.
There will be no difference, in terms of learning, between asynchronous and FtF groups.
4.2. Quality Generally speaking, there are inconsistent experimental results concerning a communication mode’s impact on the quality of group work (Fjermestad and Hiltz 1999)2.
ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION
431
In the educational domain, Benbunan-Fich and Hiltz (1999) found that groups and individuals using CMC submitted higher quality solutions than groups and individuals working in the traditional unsupported FtF manner. However, Benbunan-Fich (1997) found no significant difference between unsupported FtF groups and CMC groups in terms of quality. Given these conflicting and inconclusive findings, we hypothesize that: H2.
There will be no difference, in terms of the quality of written case analyses, between asynchronous and FtF groups.
An objective measure of quality was developed based on the content of the groups’ written report. We know of no other experiments that measure the actual content of group output (other than counting unique comments in electronic brainstorming experiments, which does not apply to our task). Based on evidence for the previous hypothesis, we hypothesize that: H3.
There will be no difference, in terms of content of the written case analyses, between asynchronous and FtF groups.
4.3. Process Satisfaction In general, research suggests that users communicating via computer will be less satisfied with the process of group interaction than when working face-to-face (Fjermestad and Hiltz 1999). Straus and McGrath (1994) explain that Feelings of depersonalization and a corresponding lack of focus on expressive issues may account for lower levels of satisfactionþother factors that might contribute include greater time pressure because of the effort required to communicate by typing and reading [Arunachalam, 1991; Daly 1993] and difficulties in understanding others’ contributions and in being understood by others. Frustrations are expected to be particularly high when the work requires reaching consensus and when such consensus involves resolving different viewpoints or interests (p. 90).
Within the educational realm, Benbunan-Fich (1997), as well as Warkentin et al. (1997), measured this variable and found that FtF groups were significantly more satisfied with the group interaction process than were CMC groups. Therefore, we hypothesize that: H4.
FtF groups will be more satisfied with the group interaction process than asynchronous groups.
Another measure of the group interaction process is students’ perception of the quality of discussions that occurred using the communication medium. The Benbunan-Fich and Hiltz (1999) study is the only one to measure this variable. They found that the FtF groups had significantly better perceptions of discussion quality compared to~cmC groups. Therefore, we hypothesize: H5.
FtF groups will be more satisfied with the discussion quality than asynchronous groups.
432
OCKER AND YAVERBAUM
4.4. Solution satisfaction Unlike process satisfaction, in general, research comparing CMC to FtF communication suggests that little or no difference exists concerning subjects’ satisfaction with the outcome produced (e.g., project, report, problem solution) (Fjermestad and Hiltz 1999). However, this finding appears to have mixed results in the educational environment. Although Benbunan-Fich (1997) found no significant differences in solution satisfaction, Warkentim et al. (1997) found that FtF groups were more satisfied than CMC groups. However, in light of the general research findings, we hypothesize that: H6.
There will be no difference, in terms of solution satisfaction, between asynchronous and FtF groups.
5. Research Methods Design. This experiment uses a single factor, counter-balanced, repeated measures design (see Table 1). The factor, communication mode, has two treatments: (1) face-to-face (FtF) with no technology support and (2) distributed asynchronous computer conferencing (CC). Experimental groups completed two back-to-back collaborative assignments; each assignment lasted for two weeks. One assignment was completed using only face-to-face collaboration. For the other assignment, groups met face-to-face for an initial meeting and then collaborated asynchronously using only the computer conferencing system. Subjects. A total of 43 graduate students participated in the experiment. The vast majority (40) were part-time students with full-time employment (38). All students were enrolled in the core information systems class required of all MBA and MS/IS students. There were two participating sections of this course, taught by the same instructor. Subjects received course credit for their participation in this study. There were 10 groups in this study, with each group ranging from four to five students. Subjects were assigned to groups near the beginning of each semester and all groups were in existence for the entire semester. Thus, group members had some experience working together prior to the experiment and the expectation that they would continue to work with their group after the completion of the experiment. Efforts were made to balance groups based on subjects’ educational background, work experience, and computer experience. Therefore, within each team, there was a variety of backgrounds and experience, both educationally and skill-wise.
Table 1. Experimental design Time 1
Time 2
Section
# Groups
Case 1 (Kodak)
# Groups
Case 2 (Chemical Bank)
1 2
5 5
FtF Asynch.
5 5
Asynch. FtF
433
ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION
Collaborative Assignments. Each group analyzed two Harvard Business School case studies3 and produced a report for each case that contained an analysis of the case along with recommendations. General guidelines for the case write-up were provided. An attempt was made to control for the difficulty of the cases. Students from the same course, but in a previous semester, rated seven Harvard case studies for the degree of difficulty. The two cases which were rated most comparably were chosen for inclusion in this experiment. Pilot study. The case study assignments, as well as all experimental procedures, were pilot tested in the semester prior to the start of the experiment. Technology and facilitation. The CC groups communicated using the FirstClass computer conferencing system. Each CC group communicated in its own conference set up on FirstClass. The conferences were minimally facilitated. The conference facilitator’s role was that of a technical assistant, helping groups with equipment problems and answering questions of a technical nature. Training. All groups received training. Collaborative learning concepts were presented to each class using the same presentation and script. Similarly, all groups received handson training on FirstClass using the same training materials. Additionally, FirstClass was used as an electronic bulletin-board and to distribute class lecture notes in each class. Thus, prior to the experiment, all subjects had used the FirstClass system beyond the training session. Procedures. Prior to the start of the experiment, all subjects attended an “information” lecture which described aspects of the experiment including collaborative learning concepts, the use of computer conferencing to accomplish group work, and the experimental tasks. No mention was made of the research questions or dependent variables. All teams in the face-to-face treatment met twice during class time. The first meeting occurred at the start of the experiment; the second meeting occurred at the midpoint of the experimental period, exactly one week after the initial meeting. Each meeting was completed within 1 ¼ hours. Face-to-face groups were permitted to meet outside of these two scheduled meetings, and were required to keep a log of their meeting times and lengths (see Table 2). Each group submitted a case analysis report two weeks after the initial face-to-face meeting. Similar to the face-to-face treatment, computer conferencing teams had an initial meeting in class which lasted up to 1 ¼ hours. However, this was the only face-to-face meeting permitted. For two weeks immediately following the initial meeting, teams in the CC treatment worked asynchronously using the FirstClass computer conferencing system. Each group submitted a case analysis report two weeks after the initial face-toface meeting.
Table 2. Number of hours that each FtF group met Section
Group 1
Group 2
Group 3
Group 4
Group 5
Total hours
1 2
2 2 1/4
5 2 3/4
2 1 3/4
8 6
1 1/4 3 1/4
18 1/4 16
434
OCKER AND YAVERBAUM
Debriefing. All groups were debriefed after the end of the second two-week experimental session, where the research question and design was explained. All student questions regarding experimental design and method were answered.
5.1. Measures of dependent variables
5.1.1. Subjective measures. Self-report data regarding satisfaction with the case study solution (i.e., solution satisfaction), satisfaction with the process used to reach a solution (i.e., process satisfaction), and perceptions regarding discussion quality were collected via the post-experiment questionnaire administered to subjects after the completion of each case study assignment. Solution satisfaction and process satisfaction were each measured using a five question scale which was developed and tested for validity and reliability by Green and Taber (1980). Perceptions regarding discussion quality were measured using a seven question scale which was developed and tested for validity and reliability by Gouran et al. (1978). The quality of each group’s written case analysis was rated by two expert judges. Each judge had an advanced degree in the information systems field and previous experience with case studies. Each judge rated the overall quality of each written case analysis, with consideration given to the quality of the content and writing style (e.g. organization, clarity, grammar). 5.1.2. Objective measures. To assess individual learning, a multiple choice quiz was administered to subjects at the end of each two-week experimental period. Each quiz contained 10 questions which were developed to measure understanding of the fundamental aspects of each case study; the questions were developed in light of major points discussed in the Harvard case teaching notes. As an objective measure of quality, for each of the two case studies, an expert rating form outlining the content of each case was developed based on the Harvard case teaching notes. For the Kodak case, 48 elements were included in the content coding form; the Chemical Bank case form included 50 elements. Each of the two expert judges used these forms to track the content of each case analysis. Both judges met to review the content forms before conducting the content coding. After each judge had individually completed the content coding forms for each case, both judges met and reviewed the coding results for each case study. All differences between judge’s coding were discussed and resolved.
6. Results Paired t-tests were run to test the effects of mode of communication on the dependent variables. Table 3 presents the means, paired differences and significance levels for the dependent variables. There were no effects due to the order of treatments. The expert judges had a high level of agreement when rating the quality of solution for case 1 (Kodak)
435
ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION
Table 3. Statistical findings
FtF Quiz scores 57.62 Solution quality 7.77 Content 15.60 Process satisfaction 4.15 Perceived discussion quality 4.12 Solution satisfaction 4.08
Asynch
DF
Paired Differences Std. Std. Error Deviation Mean
Significance (2-tailed)
56.43 7.55 16.10 3.80 3.66 4.01
41 9 9 42 42 41
39.52 0.5584 9.2400 0.9855 0.9842 0.9624
0.846 0.235 0.868 0.023 0.004 0.656
6.10 0.1766 2.9200 0.1503 0.1501 0.1485
(Cronbach’s alpha = 0.9485) and for case 2 (Chemical Bank) (Cronbach’s alpha = 0.8347). Learning measure. The percentage of correct answers on the 10 question, 100 point multiple-choice quiz was compared for each subject. Although subjects in the FtF treatment scored slightly higher (57.62) than those in the asynchronous treatment (56.43), the difference was not significant (p = 0.846), giving support to hypothesis H1. Quality Measure. The expert judges rated the quality of each group’s written case analysis on a scale from one (poor) to ten (excellent). FtF groups were rated slightly higher (7.77) than asynchronous groups (7.55), although the difference was not significant (p = 0.235). Therefore, hypothesis H2 was supported. Content Measure. The content of each group’s written case analyses were coded by the expert judges. The number of comments that each group included in their case writeup was summed to form the score. Scores could range from a minimum of zero to a maximum of 50. The written analyses of FtF groups contained slightly less content than asynchronous groups, but the difference was not significant (15.60 vs. 16.10, p = 0.868). Hypothesis H3 was supported. Process satisfaction measure. Process satisfaction was measured using a five question scale where responses could range from one (low) to five (high). As expected, subjects using FtF collaboration were more satisfied with the group interaction process than subjects in the asynchronous treatment (4.15 vs. 3.80, p = 0.023), providing support for hypothesis H4. Discussion quality measure. Perceived discussion quality was measured using a seven question scale where responses could range from one (low) to five (high). As predicted, subjects collaborating FtF were more satisfied with the discussion quality than subjects collaborating asynchronously (4.12 vs. 3.66, p = 0.004). Therefore, hypothesis H5 was supported. Solution satisfaction measure. Solution satisfaction was measured using a five question scale where responses could range from a one (low) to five (high). There was no significant difference between the rankings of subjects in the FtF and asynchronous treatments (4.08 vs. 4.01, p = 0.656), providing support for hypothesis H6.
7. Discussion and Conclusions Although there has been more than a decade of research on computer-mediated communication in education, the research has been unclear as to whether it is an effective
436
OCKER AND YAVERBAUM
replacement for FtF collaboration. This empirical study sought to add to this research area by exploring the effects of two modes of collaboration on student groups working on case analyses and write-ups. In particular, this research focused on the usefulness of using asynchronous computer conferencing as a means for groups to collaborate outside of the classroom. Both subjective and objective measures of dependent variables were used in this study. The portrayal of computer conferencing in this study is different from the distance learning application that is sometimes referred to as a “virtual classroom” (Hiltz 1994). It is also different from research regarding the electronic classroom where technology supports students working FtF in a synchronous manner (Alavi 1994; Leidner and Jarvenpaa 1993; Alavi et al. 1995). In the context of this experiment, students met during regularly scheduled classes and computer conferencing was used to continue collaboration outside of class. Table 4 contains a summary of empirical findings. Is asynchronous computer conferencing an effective substitute for FtF collaboration? It depends on your focus. If you are most concerned with the amount of learning that takes place and the quality of the end-product that is produced as a result of the learning experience, then the answer appears to be yes. We found no significant differences between FtF and asynchronous CC groups on measures of learning and quality. However, if you are most interested in the satisfaction of students with the collaborative learning experience, then the traditional FtF collaboration mode appears superior to asynchronous CC. Students in the FtF treatment reported enjoying interacting with their group members more than the asynchronous groups and also thought that their FtF discussions were of higher quality to the asynchronous ones. Although asynchronous groups were just as satisfied with the end-product (i.e., the case write-up) as FtF groups, they were significantly less satisfied with the asynchronous form of group interaction. This is no surprise, as this finding is consistently supported in the CMC literature. Furthermore, this finding occurred even though asynchronous groups had an initial FtF meeting to “kick-off” their group, where they could plan and organize their group – tasks often difficult to conduct efficiently in an asynchronous environment (Galegher 1990; Zack 1993). In light of the needs of the increasing population of nontraditional students, and given that the level of learning and quality of the end-product is equal under conditions of FtF and asynchronous collaboration, what can be done to increase students’ level of satisfaction with the process of asynchronous group collaboration? One might think that merely making this technology available to nontraditional students would be enough to ensure their satisfaction with the collaborative learning process, however that assumption is incorrect. Although asynchronous collaboration is increasingly being used in the workplace within Table 4. Summary of empirical findings Dependent variables
Hypothesis
Prediction
Results
Learning Quality Content Process Satisfaction Perceived Discussion Quality Solution Satisfaction
H1. H2. H3. H4. H5. H6.
Asynchronous = FtF Asynchronous = FtF Asynchronous = FtF FtF > Asynchronous FtF > Asynchronous Asynchronous = FtF
Supported Supported Supported Supported Supported Supported
ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION
437
“networked” organizations, it remains foreign to the vast majority of students. Of course, most students are quite adept at communicating via email, but that type of communication is very different from accomplishing complex collaborative group work. The obvious suggestion is to increase students’ exposure to the use of asynchronous technologies for collaborative tasks. Although students in this experiment were exposed to the computer conferencing technology throughout the semester, they did not use it to complete collaborative assignments prior to the experiment. Rather, it was used as an electronic bulletin-board. Students commented regarding this, making statements such as “Perhaps a small assignment weeks before the `big’ project on FirstClass would help us feel more comfortable and get us over the `get familiar’ phase.” Students are accustomed to conducting collaboration by means of FtF meetings, and have much more experience with this mode of communication than with using asynchronous technology to accomplish complex tasks. Another suggestion is to better educate students as to the benefits of this form of collaboration. For many students, dealing with technology is viewed as an unnecessary “hassle” – just another hoop to jump through on their long quest for a diploma. They perhaps fail to see the benefits provided from working this way. Although we presented an information session prior to the start of the experiment describing computer conferencing, we did not stress the positive (or negative) aspects, so as not to confound experimental results. However, as educators, we can work to “retrain” the thinking of students in this area by providing students with more electronic collaborative experiences and incorporating the use of computer conferencing throughout our courses. Finally, asynchronous collaborative learning involves technology and must be supported by technically competent personnel. It takes some know-how to run the necessary hardware and software. But more than that, someone must be available to “hold the hands” of students as they delve into this very new, and often fearful, way of learning. Software must be loaded onto PCs, Internet connections must be made, bugs must be worked out. This hand-holding takes valuable time and is perhaps best done by someone other than the instructor. During this experiment, students were provided technical support by a student assistant. This assistant worked with various students (some much more than others) to solve technical problems. From the students’ point of view, this technical assistance is essential to the success of the process. There is a learning curve associated with computer conferencing collaboration. Only after students become accustomed to this new form of collaboration can we view data regarding students’ perceptions with any degree of confidence. In spite of being less satisfied with the group interaction process, students generally felt that asynchronous collaboration was beneficial. In response to the question, “Overall, did you find that using the FirstClass computer conferencing system was beneficial to accomplishing your project work for this class? Why or why not?,” typical student comments include: “Yes, anytime a group needs to collaborate, it is useful. This is especially true since most of us are part-time and living off-campus.” “Yes. With work schedules as they are this is a great way to get ‘the group’ together regardless of the distance between members and the hours of the campus buildings.”
438
OCKER AND YAVERBAUM
The results of this experiment indicate that a technology-supported collaborative environment is an effective means of learning and conducting complex group work. However, it also shows us that people prefer to interact in a FtF manner. Social change is a slow process. Whether we can change students’ perceptions over the long run remains to be seen.
8. Limitations and Future Research The relatively small size of this study hampers the generalization of the experimental findings. More and larger studies should be conducted to test the effectiveness of~cmC for completing complex collaborative assignments. Additionally, longitudinal studies should be conducted to ascertain if perceptions change as students become more familiar with the technology-supported learning environment. Students had to use the FirstClass conferencing system. They could not use other forms of electronic communication and were not permitted to meet FtF other than the initial group meeting. Some students found these requirements too constraining as evidenced by comments such as: “The only drawback to using FC in the Kodak case was that we had to use it without any other communication.” “I think we should have had a choice to use face-to-face, FC or both, instead of being forced to use one or the other.”
Another condition should be added to the experimental design which allows students to freely choose how and when to use FtF and CMC to complete their assignments.
Notes 1. See Leider and Jarvenpaa (1995) for an overview of the various models of learning. 2. Fjermestad and Hiltz (1999) compare and contrast the results of over 175 experiments concerning CMC and group support systems. They summarize results on such dependent variables as quality and creativity of the group outcome, process satisfaction, and solution satisfaction. 3. Eastman Kodak: Managing Information Systems through Strategic Alliances (#193-037) and Chemical Bank: Technology Support for Cooperative Work (#193-131).
References Abrami, P.C. and Bures, E.M. (1996). “Computer-supported Collaborative Learning and Distance Education,” The American Journal of Distance Education, 10, (2), 37–42. Aiken, M.W. (1993). “Advantages of Group Decision Support Systems,” Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 1, 3, July; [journal on-line]; available from http:// www.helsinki.fi/science/optek/1993/n3/aiken.txt:Internet; accessed 15 February, 1997. Alavi, M. (1994). “Computer-Mediated Collaborative Learning: An Empirical Evaluation,” MIS Quarterly, June, 150–174.
ASYNCHRONOUS LEARNING, QUALITY AND SATISFACTION
439
Alavi, M., Wheeler, B.C., and Valacich, J.S. (1995.) “Using IT to Reengineer Business Education: An Exploratory Investigation of Collaborative Telelearning,” MIS Quarterly, September, 293–311. Althaus, S. L. (1997.) “Computer-Mediated Communication in the University Classroom: An Experiment with Online Discussions,” Communication Education, 46, (3), 158–174. Arunachalam, V. (1991). “Decision Aiding in Multi-party Transfer Pricing Negotiation: The Effects of Computermediated Communication and Structured Interaction,” Unpublished doctoral dissertation, University of Illinois, Urbana-Champagne. Benbunan-Fich, R. (1997.) “Effects of Computer-Mediated Communication Systems on Learning, Performance and Satisfaction: A Comparison of Groups and Individuals Solving Ethical Scenarios,” Unpublished doctoral dissertation. Rutgers University, Newark, NJ. Benbunan-Fich, R. and Hiltz, S.R. (1999). “Impacts of Asynchronous Learning Networks on Individual and Group Problem Solving: A Field Experiment,” Group Decision and Negotiation, 8, 409–426. Berge, Z. and Collins, M. (1993). “Computer Conferencing and OnLine Education,” The Arachnet Electronic Journal on Virtual Culture, 1 (3), http://cac.psu.edu/~mauri/papers/bergev1n3.html Bonk, C.J., Appleman, R., and Hay, K.E. (1996). “Electronic Conferencing Tools for Student Apprenticeship and Perspective Taking,” Educational Technology 37, 8–17. Bouton, C. and Garth, R.Y. (1983). Learning in Groups, San Francisco: Jossey-Bass. Daly, B.L. (1993). “The Influence of Face-to-Face Versus Computer-Mediated Communication Channels on Collective Induction,” Accounting, Management, and Information Technologies 3, 1–22. Eastmond, D.V. (1995). Alone But Together: Adult Distance Study Through Computer Conferencing. Cresskill, New Jersey: Hampton Press, Inc. Elder, J. (1991). “A learned response,” In Education Life, New York Times Special Report, January 6, 23. Fjermestad, J., and Hiltz, S.R. (1999). “AnAssessment of Group Support Systems Experimental Research: Methodology and Results,” Journal of Management Information Systems, in press. Galegher, J.(1990). “Intellectual Teamwork and Information Technology: the Role of Information Systems in Collaborative Intellectual Work,” In J. Carroll (ed.), Applied Social Psychology and Organizational Settings. Hillsdale, NJ: Lawrence Erlbaum Associates, 193–216. Glasser, R. and Bassok, M. (1989). “Learning Theory and the Study of Instruction,” Annual Review of Psychology 40, 631–666. Golub, J. (1988). Focus on Collaborative Learning. Urbana, IL: National Council of Teachers of English. Gouran, D., Brown, C., and Henry, D. (1978). “Behavioral Correlates of Perceptions of Quality in Decision-Making Discussions,” Communication Monographs 45, 51–63. Green, S.G., and Taber, T.D. (1980). “The Effects of Three Social Decision Schemes on Decision Group Processes,” Organizational Behavior and Human Performance 25, 97–106. Harasim, L. (1987). “Teaching and Learning On-Line: Issues in Computer-Mediated Graduate Courses,” Canadian Journal for Educational Communication 16, (3), 117–135. Hiltz, S. R. (1986). “The Virtual Classroom: Using Computer-Mediated Communication for University Teaching,” Journal of Communication 36, (2), 95–104. Hiltz, S.R. (1988). “Collaborative learning in a virtual classroom: Highlights of findings,” Paper presented at the Computer Supported Cooperative Work Conference (ERIC Document Reproduction Service No. ED 305 895). Hiltz, S.R. (1994). The Virtual Classroom: Learning without Limits via Computer Networks. Norwood, New Jersey: Ablex Publishing Corporation. Hiltz, S. R. and Wellman, B. (1997). “Asynchronous Learning Networks as a Virtual Classroom,” Communications of the ACM 40, (9), 44–49. Johnson, D.W. (1981). “Student-Student Interaction: The Neglected Variable in Education,” Educational Research, 10, (1), 5–10. Johnson, D.W. and Johnson, R.T. (1975). Learning Together and Alone: Cooperation, Competition, and individualization. Englewood Cliffs, NJ: Prentice-Hall. Johnson, D.W. and Johnson, R.T. (1989). “Toward a Cooperative Effort: A Response to Slavin,” Educational Leadership, 46, (7), 80–81.
440
OCKER AND YAVERBAUM
Karp, D.A and Yoels, W.C. (1983). “The College Classroom: Some Observations on the Meanings of Student Participation,” In Rube, H. and Clark, C. (Eds.), Social Interaction: Reading in Sociology. New York: St. Martin’s Press, 195–210. Leidner, D.E. and Jarvenpaa, S.L. (1993). “The Information Age Confronts Education: Case Studies on Electronic classrooms,” Information Systems Research 4, (1), 24–54. Leidner, D.E. and Jarvenpaa, S.L. (1995). “The Use of Information Technology to Enhance Management School Education: A Theoretical View,” MIS Quarterly September, 265–291. Levine, A. and Cureton, J.S. (1998). When Hope and Fear Collide: A Portrait of Today’s College Student. San Francisco: Jossey-Bass. Morrison, J.L. (1992). “The Computer Conference: Adaptive Problem-Solving within a Spontaneous Technological Framework,” Educational Technology December, 46–47. Scrifes, E. L. (1998). “An Empirical Investigation of Electronic Groups in the Classroom,” Journal of Education for Business 73, (4), 247–250. Short, J., Williams, E., and Christie, B. (1976). The Social Psychology of Telecommunications. London: Wiley. Slavin, R.E. (1987). Cooperative Learning: Student Teams (2nd ed). Washington, D.C.: National Educational Association. Straus, S.G. and McGrath, J. E. (1994). “Does the Medium Matter? The Interaction of Task Type and Technology on Group Performance and Member Reactions,” Journal of Applied Psychology 79, (1), 87–97. Tooey, M.J. and Wester, B.R. (1989). “Computer Conferencing: A Campus Meets Online,” Online July, 54–60. U.S. Department of Education.(1996). Condition of Education. National Center for Education Statistics (NCES 96304), Washington, DC: U.S. Government Printing Office. U.S. Department of Education. (1997). Digest of Education Statistics. National Center for Education Statistics (NCES 98-015), Washington, DC: U.S. Government Printing Office. Warkentin, M.E., Sayeed, L., and Hightower, R. (1997). “Virtual Teams Versus Face-to-Face Teams: An Exploratory Study of a Web-based Conference System,” Decision Sciences 28, (4), 975–996. Wells, R.A. (1990). Computer-Mediated Communications for Distance Education and Training: Literature Review and International Resources, Alexandria, VA.: U.S. Army Research Institute for the Behavioral and Social Sciences (ERIC Document Reproduction Service No. ED 343 569). Whipple, E.R. (1987). “Collaborative Learning: Recognizing It When We See It,” AAHE Bulletin 40, (2), 3–7. Zack, M. (1993). “Interactivity and Communication Mode Choice in Ongoing Management Groups,” Information Systems Research 4, (3), 207–239.
