Session S1J
Defining the Requirements for the Next Generation Asynchronous Computer Mediated Communications System for Use in Online Education: A Faculty Perspective Maxine S. Cohen1, Timothy J. Ellis2
Index Terms – asynchronous communications, threaded discussion forums, distance education, online learning environments, online pedagogy.
What seems to still be missing from online learning environments are standardized, systematic, and reproducible procedures needed to create the next generation online learning environment. Much of the present work in design is either ad hoc or being driven by the available technology. As these new systems become available, each iteration has everything from the previous version, plus a number of new and enhanced features. This “feature creep” makes the software bloated and hard to manage. The needs and expectations of the users of the ALN are often not even considered as new products appear on the market. The requirements are often driven by the vendors of these products [11]. There are three major classes of users for these systems: faculty, administrators, and students. Although the administrators are often the ones that budget and select these systems, this research concentrates strictly on the academic and pedagogical dimensions of these systems. The goal of this paper is to anchor the desired design features by generating a requirements specification using a standardized procedure from the faculty perspective. Previous work [4] has discussed designing a specification from the student perspective. This work is a continuation of that work, but now concentrating on faculty needs and perspectives. Others [6,15] have shown that there is still a need for effective instructional design for online courses. In addressing this goal, the study attempted to answer the following research questions: 1. What, in the opinion of experienced online instructors, is inadequate with the asynchronous communication tools currently in use in distributed education? 2. What features should an asynchronous computer mediated communication system have?
INTRODUCTION
BACKGROUND
The viability of distributed educational services delivered via asynchronous learning networks (ALN) has been well established both in practice and through the literature. Numerous articles have been published comparing face-to-face instruction to online instruction and analyzing the strengths and weaknesses of various pedagogical approaches.
The internet has turned into an excellent medium to communicate with a wide ranging audience. Professors can use the internet to distribute information about their courses and students can submit questions and course products [10]. Gill [7] defines some of the strengths of the internet to be: convenience and flexibility and easier access to administrative
Abstract - Distance education courses at all academic levels and throughout all disciplines are increasingly being offered by educational institutions. Over 1300 colleges and universities offer degree programs using an online modality. The literature contains several thousand articles comparing online to face-to-face education or discussing various pedagogical strategies for online education. Despite this widespread acceptance, the online learning environment is still quite immature. The software constituting the environment has largely been developed in an ad hoc fashion. Design decisions were largely based on the limitations inherent in the technology rather than user needs and requirements. The development and implementation of most commercial learning management systems have followed a one-size-fits-all approach. This rather casual methodology is inadequate for the broad range of courses now being offered online. This paper is a follow-up of work presented at FIE 2005 in which the development of a requirements specification for an effective asynchronous computer mediated communications system from the student’s perspective was reported. This research employed a similar Delphi Process using eight experienced online instructors. An initial list of 39 potential characteristics for the ideal environment, brainstormed by the expert panel in the first round of the process was refined to 37 criteria split into five broad categories.
1
Maxine S. Cohen, Professor, Nova Southeastern University, Graduate School of Computer and Information Sciences, Fort Lauderdale, FL 33314,
[email protected] 2 Timothy J. Ellis, Associate Professor, Nova Southeastern University, Graduate School of Computer and Information Sciences, Fort Lauderdale, FL 33314,
[email protected]
1-4244-0257-3/06/$20.00 © 2006 IEEE October 28 – 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference S1J-7
Session S1J information. Of course, there are weaknesses of the internet as an instructional environment as well. Hicks, Reid, and Rigmor [9] stated that online learning environments have the following eight characteristics: computer-mediation, the potential for accessing large amounts of dynamic information through the WWW, the use of hypertext and working with materials in a non-linear way, access to real world contexts via the internet, the capacity to communicate via email and other electronic technologies with lecturers and other students, new methods for the administration of learning such as submitting assignments and getting results, networking, and internationalizing the curriculum. These eight characteristics constitute a solid theoretical basis, but do not represent a checklist for an effective learning environment. The years of research and experience teaching online must be combined with this theoretical basis in analyzing and designing the “next generation” communication environment for internet-based learning. Asynchronous computer mediated communications systems as defined for this research refers only to the hardware, software, policies, and procedures that promote discourse within the learning environment and should not be confused with a comprehensive course management system. This type of system can directly affect the quality of learning. Learners can respond to the material in this communications system immediately or in minutes, hours, or days [14]. Designers need to be aware of the technological capabilities of existing computer mediated communication systems, but not lose sight of creating environments that are educationally sound. These systems need to be flexible and usable in different learning contexts and across the curriculum. Learning styles and adaptive environments need to be considered in the mix. A lack of theoretical grounding of webbased communication tools has been noted in the literature [12,13]. Ismail’s e-learning systems framework [11] is a conceptual model that captures the information flow. It is based on the Learning Technologies Systems Architecture (Architecture and Reference Model Working Group of IEEE) as well as other standards. The goal of the model is to combine the learning goals with pedagogy. His framework allows organizations to: capture the knowledge within their organization, structure the knowledge into focused, directed learning programs, incorporate third party content, and achieve rapid updates, dissemination, management, and utilization of that knowledge throughout the organization. Coffey and Canas’ Organizer [3] is a software suite of tools. It gives flexibility to both the designer and the student. The Organizer is a mathematical graph. It provides two views: the more global (contextual) view and a local (focus) view. The tool uses a client-server architecture. Its goal is to help the instructional designer organize the learning environment. The student can take different paths through the Organizer. There is some concern about the loss of educational quality in online learning systems [10]. At the same time there is a tremendous opportunity to make these systems more
flexible [5]. The systems need to be more than a one size fits all, in terms of class size, class content, student preparation, and instructor capability. A grassroots intervention is needed by both teachers and learners. The administration can not make some of these decisions in isolation [6]. Certainly, students have expectations about online learning environments [15]. Many have now been exposed to the major players in the market (BlackBoard, WebCT, etc.) as well as an assortment of home grown tools. Although there is a strong leaning towards student-centered environments, faculty still control what goes into their courses and set the tone for the course environment. We, as faculty, are still learning how best to integrate these systems from many different directions. The technology drives some of this. The vendors keep coming out with more feature-laden systems. Students’ expectations are getting stronger. Faculty now realize that some part of online education is in their future as well. Gill [8] sums this up well in his article about the five hard things about using the internet: lack of models from our own experience, constant disruptions precipitated by evolving technology, explaining our courses to others, adjusting to a new rhythm of life, and adjusting to our new role. Although these challenges are formidable, being on the leading edge of this revolution in education is exciting.
METHODOLOGY The research questions underlying this study entailed use of an Expert Panel engaged in an online version of a Delphi Method [1]. The Expert Panel was comprised of eight instructors teaching in the computer technology curriculum at either the undergraduate or graduate level. The expert status of the panel was based upon the following criteria: an average of 13.5 years teaching experience, 5.9 years of which included teaching in an online environment; an average of 45 courses taught in either an entirely online or hybrid environment; five of the members had taken courses in an online environment; all members described themselves as at least partially “selftaught” in online pedagogy, but four also had formal training and five had received formal mentoring. The Delphi Method [1] is a well-established process for determining group consensus on a topic. Traditionally, the process entails a series of questionnaires on which the participants vote anonymously. Each round of voting is followed by structured feedback on the results of the previous round. Typically, the process continues until a document that all find, at the least, acceptable is generated. The online Delphi Method [2] implemented in this study entailed the following rounds: Round 1: Brainstorming. Each participant was provided a link to a Web-form that described the goals of the study and solicited up to 10 characteristics, features, or functions that would constitute an effective asynchronous collaborative learning environment. The brainstorming process produced a total of 39 unduplicated suggestions that appeared to fall into five categories: assessment of students, organizational issues,
1-4244-0257-3/06/$20.00 © 2006 IEEE October 28 – 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference S1J-8
Session S1J synchronous features, multimedia features, and management utilities. Round 2: Initial anonymous vote. Each member was given a link to a Web-form that listed the 39 criteria candidates developed in Round 1, ordered by the five categories that emerged in that round. Each item was to be rated on its importance in a well-designed asynchronous communication environment using a three-point scale: No! Cannot accept a product if it has this feature; Don’t care: Can accept a product with or without this feature; and Vital! Cannot accept a product without this feature. The participants were also offered the opportunity to suggest rewording for an item or present new items for consideration. Figure 1 presents a sample of an item from this second round.
FIGURE 1: ROUND 2 SAMPLE ITEM
Round 3: Second anonymous vote. Round 2 produced no new items or recommendations for rewording. Thirteen items did receive both at least one “No! Cannot accept a product if it has this feature” and one “Vital! Cannot accept a product without this feature” vote. In this round the members were given the results from Round 2 and asked to resolve these “irreconcilable differences” by categorizing the thirteen items in question using a more granular, five-point scale: • No! = I would not "buy into" using a product with this feature • Prefer not = I would rather not see the feature in the product, but would accept it if present • Don't care = I don't care if the feature is present or not • Nice to have = I would rather see the feature in the product, but would accept it if absent • Vital! = I would not "buy into" using a product without this feature Figure 2 presents an example of a Round 3 item. Since Round 3 produced no suggestions for rewording and all “irreconcilable differences” were resolved, it was determined that the panel had reached consensus. The Delphi process was concluded, producing the criteria list displayed in Table 1.
FIGURE 2:ROUND 3 SAMPLE ITEM
RESULTS Table 1 lists the criteria for an effective asynchronous communication network for use in an educational setting developed by the panel of expert educators that participated in the study. The first column of the table lists the 39 items
developed by the panel. The next five columns present the distribution of ratings for each item from the second and third rounds of the Delphi Process. The final column presents the weighted score for each item, as calculated by summing the products of the number of No! responses by 1, Prefer not responses by 2, Don’t care responses by 3, Nice to have responses by 4, and Vital responses by 5. The criteria are grouped by the five categories that emerged after the first round of the Delphi Process; within each category, the criteria are listed by weighted score.
CONCLUSIONS The asynchronous communication facilities currently available in online learning networks are clearly inadequate. The faculty who participated in this study had no difficulty identifying a number of features, functionalities, and characteristics that could greatly enhance the learning environment. This list should, of course, be viewed as an initial set of requirements for the next generation asynchronous learning environment. Further research in two areas is definitely indicated: replicating the process executed in this study using a wider range of faculty in terms of both discipline and academic level.
REFERENCES [1]
Adler, M. & Ziglio, E. “Gazing into the Oracle: the Delphi Method and its application to social policy and public health”. London, Jessica Kingsley Publishers, 1996.
[2]
Chou, Turoff & Hiltz,. “Examining Drivers of Course Performance: An Exploratory Examination of an Introductory CIS Applications Course”. Decision Sciences Journal of Innovative Education. 4(1), pp. 51-65, 2002.
[3]
Coffey, J.W. & Cañas, A.J. “A Learning Environment Organizer for Asynchronous Distance Learning Systems”. Proceedings of the Twelfth IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 2000). November 6-9, 2000. Las Vegas, Nevada.
[4]
Ellis, T.J. & Cohen, M.S. “Building the better asynchronous computer mediated communication system for use in distributed education”. Proceedings of the 35th ASEE/IEEE Frontiers in Education Conference, 2005, pp. T3E15-T3E20.
[5]
Feldstein, M. & Masson, P. “Unbolting the chairs: making learning management systems more flexible”. eLearn Magazine. January 30, 2006.
[6]
Feldstein, M. “A call to arms”. eLearn Magazine, January 30, 2006.
[7]
Gill, G. “13 (Educational) Things Id Rather Do Over the Internet”. eLearn Magazine. February 28, 2006.
[8]
Gill, G. “5 (Really) hard things about using the internet in higher education”. eLearn Magazine. March 6, 2006.
[9]
Hicks, M., Reid, I.C. & Rigmore, G. “Designing responsive online learning environments: approaches to supporting student”. AARE Conference, 1999.
1-4244-0257-3/06/$20.00 © 2006 IEEE October 28 – 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference S1J-9
Session S1J [10] Hirshheim, R. “The Internet-Based Education Bandwagon: Look before you leap”. Communications of the ACM. 48(7). 97-101. 2005. [11] Ismail, J. “The design of an e-learning system beyond the hype”. The Internet and Higher Education, 4(2002), 329-336. [12] Jarvela, S., & Hakkinen, P. “The levels of web-based discussions: using perspective-taking theory as an analytical tool”. In H.v. Oostendorp (Ed.), Cognition in a digital world, (pp. 77-95). Mahwah, NJ: Lawrence Erlbaum Associates, 2003. [13] Koschmann, T. D. “Toward a theory of computer support for collaborative learning”. Journal of Learning Sciences, 3(3), 1994, 219-225. [14] Li, J. & Yang, F. “Perspectives on evaluating asynchronous learning networks (ALN) system”. In R. Cheung, R. Lau, Q. Li (Eds.), New Horizon in Web-Based Learning, (206-213). 2004. [15] Song, L., Singleton, E. Hill, J. & Koh, M.H. “Improving online learning: student perceptions of useful and challenging characteristics”. The Internet and Higher Education, 7(2004), 5970.
1-4244-0257-3/06/$20.00 © 2006 IEEE October 28 – 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference S1J-10
Session S1J Weighted Score
No!
Prefer not
Don't care
Nice to have
Vital!
Table I Criteria List
Assessment of Students To facilitate assessment of the students' participation in the assignment, capability of displaying the postings in multiple formats. Some of the desired formats would be: 1.Threaded, full text, with the message indented to display responses to postings 2.Sorted by date of contribution 3.Sorted/grouped by author of the contribution 4.Sorted/grouped by responses to a student's contributions 5.Sorted/grouped by the students to which each student responded 50% 38% 13% 0% 0% 4.38 Automated diagnostics that identify the lost and lonely student -- lost and lonely not in the traditional sense, but in the sense of the solitary learner. 0% 63% 38% 0% 0% 3.63 Facilitating better discourse between instructor and students is the most important feature to improve. Next, I would choose class discourse, and lastly student-to-student discourse. 38% 25% 38% 0% 0% 4.00 Quick access to student grades-to-date across all courses 0% 50% 38% 0% 13% 3.25 25% 63% 13% 0% 0% 4.13 To permit assessment of the student's participation, provide statistical analysis of participation Tracking of individual responses, including the time, location within the instructional sequence, and opportunity to manually or automatically qualify the response 56% 0% 44% 0% 0% 4.11 Capacity for the instructor to specify rating criteria and a rating scale for evaluating or categorizing postings. This capacity should enable the instructor to actively rate postings from students as the instructor reads the posting, with the rating information retained in a database for subsequent analysis. 50% 0% 50% 0% 0% 4.00 Portfolio repository of student work, which I add to, that I can quickly access to view student progress 78% 0% 22% 0% 0% 4.56 Membership status- a better way of tracking who has read and done what assignments. 67% 0% 33% 0% 0% 4.33 Attendance- the ability to easily pull weekly attendance 56% 0% 44% 0% 0% 4.11 Organizational Issues A clear organizational template that encourages a structure for courses organization that relates directly to interaction. this template should be clear and obvious, but modifiable. 75% 0% 25% 0% 0% 4.50 Ability to schedule the appearance of threaded discussions during the class duration (Week One discussion, Week Two discussion, etc) 67% 0% 33% 0% 0% 4.33 Group support - enable the instructor to break the students into small groups and provide a group-only 100% 0% 0% 0% 0% 5.00 100% 0% 0% 0% 0% 5.00 Shared documents for easier collaborative work and peer review. Repository where instructors can add comments about student. This repository would be easily accessible by me from the course. 22% 0% 78% 0% 0% 3.44 A collaborative environment in which students may add/edit/delete assignment content within the LMS 56% 0% 44% 0% 0% 4.11 Synchronous Features An integrated synchronous chat feature that would allow the students to do real-time communication online, but would then parse that chat session into a series of threaded discussion forum postings that could be reviewed 33% 0% 67% 0% 0% 3.67 Ideally, a feature that could facilitate peer communication in an asynchronous environment would be well received. Blackboard's Discussion Board works fairly well, but I wish for some improvement in this area. I am unsure whether a synchronous "class communication" would be beneficial or end up being a zone for 33% 0% 67% 0% 0% 3.67 IM like utility that fellow students & instructor may interact when logged into the LMS 56% 0% 44% 0% 0% 4.11 Virtual classroom / chat function to use at a mutually convenient scheduled time with the student or students 25% 25% 38% 13% 0% 3.63 Frankly, I have been content with the interactive, asynchronous features of Blackboard while teaching online for the past eight years. As you request information on what additional features would benefit my students and me, I would like a better vehicle for synchronous communication, first, between instructor and student. 25% 25% 25% 25% 0% 3.50 Multimedia Features 25% 50% 13% 13% 0% 3.88 Integrated sound recorder to incorporating voice recordings and other sounds in forum posting Easy access to pictures of students in the course 13% 50% 38% 0% 0% 3.75 Video and audio storage of lectures including any presentation materials, demonstrations, or computer aided lectures for playback by the student. Complete control by the student is required. Utilizing separate windows for components would allow for personal configurations 38% 25% 38% 0% 0% 4.00 A video/audio streaming application imbedded in the LMS which can provide (1 to 1) or (1 to many) capabilities for online office hours /chats and other instructional purposes. With the ability to playback any session for students at a future time. Although this is a synchronous modality use would still be beneficial to students who 13% 38% 38% 13% 0% 3.50 Support for audio (podcasting) 56% 0% 44% 0% 0% 4.11 Support for video 56% 0% 44% 0% 0% 4.11 Capability for multi media, pre-recorded media of all types, and rapidly developed media identified for use on the 78% 0% 22% 0% 0% 4.56 spur of the moment Management Utilities 71% 29% 0% 0% 0% 4.71 Exportability so the instructional content, the intellectual property, is transportable to other platforms or to other WYSIWYG type of HTML editor with full html editing features. More times than not what I write on the screen 38% 63% 0% 0% 0% 4.38 does not appear in the same format Polling - where you can have a "show of hands" for response to questions 13% 38% 25% 25% 0% 3.38 Integrated "white board" that would contain basic graphic tools and allow the student and instructor to 67% 0% 33% 0% 0% 4.33 incorporate basic drawings in either jpg or scalable vector graphic format into a forum posting Online access to textbooks and handouts with the capability to take notes directly into the text, including storage and retrieval processes with a graphical interface 56% 0% 44% 0% 0% 4.11 Automatic/native Spell/Grammar / Thesaurus checking much like MS Word provides 89% 0% 11% 0% 0% 4.78 Instructional Materials tailored to using the existing LMS environments without the need to make extensive changes to them For example Interactive Textbook Chapter Notes 44% 0% 56% 0% 0% 3.89 78% 0% 22% 0% 0% 4.56 An Integrated database for identifying plagiarized documents. For example integrate TurnItIn.com into WebCt A robust help/ tutorial system to aid students with LMS related issues and questions 78% 0% 22% 0% 0% 4.56 The ability for instructors to easily forward and RESPOND to email from the LMS 78% 0% 22% 0% 0% 4.56
1-4244-0257-3/06/$20.00 © 2006 IEEE October 28 – 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference S1J-11
