Educational Technology & Society 3(3) 2000 ISSN 1436-4522
Increasing Access to Learning With Hybrid Audio-Data Collaboration Michael W. Freeman, EdD President, InnovaLearn Consulting 240 Oak Shores Drive McDonough, GA 30253 USA
[email protected]
Lawrence W. Grimes, PhD Professor Experimental Statistics F-148 Poole Agricultural Sciences Building Clemson University, Clemson, SC 29634-0367 USA
[email protected]
J. Ray Holliday, EdD Instructor, Biology Program 205 Barre Hall Clemson University, Clemson, SC 29634-0367 USA
[email protected] ABSTRACT Internet enabled hybrid audio-data collaboration delivers high quality audio over telephone lines and data interaction over packet switched Internet connections, thus distributing the transmission load between two highly accessible but limited bandwidth media. This paper explores the need for hybrid audio-data collaboration and describes two complementary studies comparing the performance and satisfaction of groups of graduate students taking an introductory course in statistics via the following modes: (1) hybrid audio-data collaboration, (2) satellite delivered instructional television, (3) face-to-face in the television studio, and (4) face-to-face in a traditional classroom. The results of the studies suggest there is no difference in student learning performance between the hybrid audio-data collaboration and instructional television or face-to-face modes for the graduate level introductory statistics course. Students in the Internet enabled hybrid audio-data collaboration group were also more satisfied than those in the instructional television group with the technical aspects of the medium. Compared to instructional television, hybrid audio-data collaboration can be a viable method of dramatically increasing access to learners while maintaining educational effectiveness and student satisfaction.
Keywords Computer mediated communication, Data conferencing, Collaboration, Accessibility, Distance learning
Introduction Distance learning is rapidly growing throughout industry, government and academia. It is widely recognized as the method of choice for reducing costs, increasing flexibility, increasing access and increasing number of learners reached. Since distance learning by definition relies on technology for delivery (Sauve, 1993), there is also a challenge to provide service to areas and learners with limited access to technology. Distance learning has the power to level the playing field for rural and disadvantaged learners by providing a learning experience without walls or barriers. In order to do this, distance education technologies must go beyond the Information Superhighway onramps to reach learners down the two lane streets and digital dirt roads. The more courseware designers and program managers rely on high cost, high bandwidth, exotic equipment and dedicated facilities, the more difficulty in reaching the learners who can benefit the most from distance learning. To traverse these digital dirt roads, distance learning must embrace scalability, use common infrastructure and provide effective learning. It must also be affordable and available. According to the U.S. National Center for Education Statistics (1998) , the three factors most cited as preventing institutions from starting or expanding distance education programs are: program development costs (43%) limited technological infrastructure (31%), equipment failures and cost of maintaining equipment (23%). All
112
three of these factors are exacerbated by selecting high bandwidth systems requiring dedicated, unique equipment and facilities.
Information superhiway or digital dirt roads? The transportation analogy of the Information Superhighway evokes images of knowledge traveling at high speeds from the world's great repositories to eager learners in organizations occupying choice real estate with quick access to on and off ramps. If we follow this analogy we see there are also many potential classrooms and learners in organizations on the periphery far from the high-speed system. They may be there because they can't afford the close-in real estate, their family ties are in the area or they prefer the lifestyle. For whatever reason, these customer organizations are down the digital equivalent of dirt roads. This challenge is also inhibiting the growth of electronic commerce (Clinton, 1998).
Imperative to increase access and control costs The major challenge of distance education is increasing access while controlling the cost of delivery. Meeting this challenge while maintaining effective instruction is crucial to achieving the return on investment necessary to insure the viability of distance learning programs. The imperative of access and cost effectiveness requires the selection of the least expensive alternative that meets the course objectives and reaches the intended audience. All things equal, the more access and less expensive the method, the better. Courseware designers and managers are expected to be good stewards of resources entrusted to them and expected to seek efficiency in their designs.
Technology determines access and costs For distance learners, access to the technologies used by the providing institution for delivery determines access to education and training. Therefore, access to education and training is limited by the requirement for high bandwidth systems requiring dedicated, unique equipment and facilities specially trained support perssonnel. Schools and students in rural and developing areas are least likely to have access to broadband educational technology (Parrott, 1995). The attention and efforts of distance education professionals are often diverted from stable, affordable, effective solutions by the allure of the latest, highest cost technologies (Sherry, 1996) (Russell, 1997). The U.S. National Institute of Standards and Technology (1998) states: Many educational technologies are high in cost, low in reliability, and difficult to adapt to special usability needs. Better use of information technology, including the Web and other networks, could reach more learners with educational material tailored precisely to their needs. The costs of producing and disseminating educational content would drop. The user community for instructional systems would expand and become more diverse. There is great potential to increase access to quality education through the convergence of emerging network technologies and proven techniques. The demand to provide global access at affordable prices is especially important for developing areas and other geographically separated organizations and learners.
Role of synchronous delivery Distance education delivery systems are categorized as either synchronous or asynchronous. Synchronous delivery requires instructor and students to participate at the same time while asynchronous delivery allows participation at differing times. The main advantage of synchronous delivery is the provision for live interaction and the possibility of more natural group processes. The disadvantage is the requirement to adhere to a specific time frame that may not be convenient for all participants, especially those in other time zones (Steiner, 1995). Asynchronous delivery systems are characterized by the separation of the instructor and student in both place and time. Asynchronous systems allow anytime-anywhere learning, but are limited in student and instructor interaction.
113
As Chute pointed out, synchronous events are desirable in distributed learning programs in order to provide student-to-student interaction for peer learning and student to instructor interaction for mentored learning. Additionally, synchronous events provide a framework of calibration and expectations to keep students on track. Even programs that depend primarily on asynchronous learning benefit from periodic synchronous events. Synchronous learning activities leverage the great depth of expertise and qualities of the traditional classroom while expanding the physical reach to geographically separated learners (as cited in Christensen & CowleyDurst, 1998).
Synchronous methods Many of the current synchronous delivery methods employed tend to use primarily text based computer mediated communications such as chat sessions or video based tele-training applications. Both of these have significant limitations. Text Text based communications require the participants to type their comments using a keyboard. These texts lack the rich, multimedia dimension of communications and can also lack spontaneity. The greatest advantages to this type of delivery are the low communications bandwidth required and the ability to save the text discussions for review later. However, very often the germane response to a probing question is composed, keyed and sent too late to flow with the discussion. Text messages are relatively shallow in respect to transferring the meaning of statements, especially those that are ambiguous or sarcastic. Except for the use of special symbols called emoticons, there are few provisions for word inflection or emphasis. Video Full motion video based communications require expensive, dedicated equipment and infrastructure as well as technical support personnel with specialized skills. The transmission of communications requires dedicated, high-speed lines and system operation requires extensive, specialized training. All of these considerations combine to dramatically increase the cost of operation and ownership. No single factor drives the costs of synchronous distributed learning more than the reliance on full motion video. Along with all these costs, video is also limited in quality of graphics due to the requirement to compress the video images. Limitations in interactivity are caused by the latency of the compressed audio and video signal, which results in participants talking over each other. The primary advantage of video is the social aspects of seeing the movement of participants. Although internet protocol based video systems show promise to reduce costs, the current state of the technology and lack of quality of service connectivity severely limits their effective use across the wide area network. Audio Although audio conferencing using the telephone is simple, inexpensive and extremely accessible, it is used very little for education. The most significant drawback of audio conferencing is the lack of a shared graphical workspace that can be manipulated in instructionally significant ways by the instructor and students. (Wisher, 1998) The switched telephone network is the most cost-effective system currently available for transmitting voice communications because of guaranteed quality of service, impact of open competition and governmental policies of universal service. Audiographics Audiographics is defined by Willis (1993) as …a sophisticated computer application relying on graphic computer interaction augmented by two-way, real time audio communication. Audio, data and graphics are shared over telephone lines, allowing users in different locations to simultaneously work on the same application."
114
Audiographics is an effective and low-cost solution for synchronous interactive distance education in groups. This is the distance education model most like the current classroom approach, therefore course adaptation costs are reduced and the guiding role of the instructor is preserved (Sherry 1996). Audiographics has been recognized in the literature as one of the least costly interactive methods of delivering synchronous distance education (Bradshaw & Desser, 1990).
The hybrid approach The future of communications and distance education is in networks and connectivity. However, the current Internet bandwidth and access speeds are inadequate for the delivery of true multimedia instruction combining sound, video, graphics and data (Kerka, 1996). Hybrid approaches to distance education use a combination of technologies to increase capacity and choice in designing and delivering instruction. (Kidwell, 1998). Hybrid audiographics uses the public switched telephone network for voice transmission and the packet switched Internet for graphical data transmission. This combination distributes the transmission load between two highly accessible but limited bandwidth media, thus improving performance and eliminating the requirement for broadband connections to support real-time interaction. Essentially every site capable of simultaneous voice and Internet communications over telephone or network connections, or a combination of the two, is a potential distance learning station.
Hybrid audio-data collaboration The proliferation of low cost personal computers capable of rendering high quality graphics, adoption of international standards for multimedia conferencing, ubiquity of Internet access and universal telephone service have created the opportunity to deliver scaleable, low cost multimedia instruction with the hybrid audio-data collaboration approach. This consists of delivering high quality audio over telephone lines and data interaction over packet switched Internet connections. Costs are very low and it is accessible to any site with telephone service, a Pentium class personal computer and Internet connection. With U.S. national information infrastructure initiatives, Internet access is available in essentially every community in the country and the switched telephone network is universally accessible. Hybrid audio-data collaboration is much the same in results as older telephone based audiographic applications. Both use the standard switched telephone system for audio. However, audiographics originally required a dedicated phone line for graphics. Hybrid audio-data collaboration uses packet switched communications such as Local Area Networks (LAN) or the Internet for transmission of visual information and application controls. Multi-point conferencing can be accomplished for hybrid audiographic teleconferencing without the hardware requirements of older versions of audiographics (see Figure 1). With hybrid audio-data collaboration, teachers and learners can communicate by voice while interactively sharing and annotating visual information. Graphics can be simply and quickly prepared, even during delivery. Multiple locations can be reached at the same time. Instruction can originate from any of the participating locations therefore empowering decentralized distance education programs. Rural areas and less affluent schools especially benefit from a cost effective, lower bandwidth solution (Gooley, nd).
International standards Hybrid audio-data collaboration uses the International Telecommunications Union (ITU) data conferencing standard known as T.120, which is a component of the family of multimedia conferencing standards known as H.32X. These standards insure worldwide interoperability. The H.32X standards promise to provide rich multimedia conferencing interoperability with integrated audio-video and data communication. However, the only component with guaranteed reliable transmission over the Internet is the T.120 data conference. Therefore, data conferencing is the most dependable method of shared visual digital interaction currently available over the Internet (The International Multimedia Teleconferencing Consortium, 1997). Using the telephone provides quality of service for the audio channel by creating a virtual direct circuit in which all information is carried along the same route for the duration of the call. This insures high fidelity, real-time voice interaction at a relatively low cost. This approach to providing the voice channel results in more effective
115
and interactive teleconferences since the quality of audio is the highest determinant of participant satisfaction (Tang & Isaacs, 1992). Analog telephone service is also essentially universal in the United States due to governmental policy and subsidies.
Figure 1. Typical Hybrid Audio-Data Connectivity Using the packet switched Internet for data transmission allows graphics and application controls to travel in digital format along toll-free routes worldwide. The variable latency of the Internet is offset by error-free, guaranteed, full resolution transmissions (IMTC, 1997). While latency in audio would severely hamper natural communication, latency in graphics transmission is a relatively minor issue.
Effectiveness The majority of research on the effectiveness of distance education over the last 50 years has shown there is no significant difference in learner performance due to technology or media. The most important factors are welldesigned lessons and courseware, not the technology used for delivery (Russell, 1998). This is because the core knowledge content of the course should remain unchanged by the decision to deliver at a distance, although new strategies for presentation and additional preparation time for instructors may be required (Willis, 1992). Video and audiographics have also been recognized to be the most preferred technologies for synchronous learning in groups (Duning, et al, 1993). This is due to the shared nature of the visual workspace and the ability to focus and involve geographically separated groups. In a series of studies on teleconferencing effectiveness for group problem solving, audio-data collaboration was at least as effective as video conferencing. There was no significant difference between the quality or timeliness of decisions between study groups using video with audio compared with groups using graphics and audio (Tang & Isaacs, 1992).
116
In a study comparing the performance of groups of military students taking an introductory level network protection course via (1)hybrid audio-data collaboration, (2) hybrid audio-data collaboration with video of instructor and (3) traditional face-to-face classroom, there was no significant difference in student performance as measured by self-assessment or actual test grades. Just as important, the typical transmission costs associated with delivering hybrid audiographics were calculated as approximately 4% of the cost of two-way full motion video. (Freeman et al, 1999).
Study approach The experiment used hybrid audio-data collaboration, satellite television and face-to-face lecture to deliver a 14 week introductory course on experimental statistics. The study involved comparisons of students' academic performance and satisfaction with the delivery medium. An overall assessment of the effectiveness of audiodata collaboration was also conducted. The basic approach of the study was to instruct three classes of students simultaneously with the same instructor and graphics using hybrid audiographics for one class, instructional television for a second and live instructor face-to-face for the studio audience. Additionally, the same instructor taught another section of the course with the same courseware using traditional face-to-face lecture. All students were administered the same traditional pencil and paper tests and questionnaires to measure satisfaction. Data on past academic performance in the form of cumulative grade point average and aptitude for quantitative graduate coursework in the form of Graduate Record Examination Quantitative scores were gathered from student records to serve as surrogate pretests and indicators of the equivalency of groups. The traditional face-to-face section met twice a week for 90 minutes and the three simultaneous sections met once each week for three hours. An alpha level of .05 was used for all statistical tests. Course Description The formal title of the course was Statistical Methods and the course number was EX ST 801. The course carried four graduate credit hours for the equivalent of three hours of classroom contact and three hours of laboratory work each week. The Clemson University Graduate Announcements entry describes the course as follows: EX ST 801: Statistical Methods, 4 cr. (3 and 3) F, S Role and application of statistics in research; estimation, test of significance, analysis of variance, multiple comparison techniques, basic designs, mean square expectations, variance components analysis, simple and multiple linear regression and correlation, and nonparametric procedures. Prerequisite: Permission of instructor. (Clemson University Graduate School, 1998) The stated primary objectives of the Experimental Statistics 801 course were: ! to learn how to summarize and interpret research data. ! to learn how to draw appropriate conclusions and inferences from data. ! to develop an understanding of several statistical techniques and to know when and how to apply them. ! to learn about a few basic experimental designs and how to select an appropriate design for research. In addition to the primary objectives above, the following secondary course objectives were also stated: ! to learn to use computer and statistical computing packages. ! to develop an awareness of the power and capability of statistical software to aid in performing statistical analyses. ! to review the importance of the scientific method in research. ! to learn to read texts, periodicals, the newspaper, etc. with greater understanding of the statistical information presented. Description of Groups Forty-seven graduate students participated in the study. These students were enrolled in Experimental Statistics and self-selected into one of four intact study groups representing methods of instructional delivery. Those groups were distance education via hybrid audiographics (HA)(n=5), distance education via satellite television
117
(ITV)(n=7), face-to-face instruction in a studio audience (F2F-S)(n=7) and face-to-face instruction in a traditional classroom (F2F-T)(n=26). The two distance education groups and the face-to-face studio groups were all taught simultaneously meeting at night once each week by the same instructor with the same audio and course materials with the only difference that the HA group did not see the moving images of the instructor. The face-to-face traditional classroom group was taught by the same instructor with the same course materials meeting during the day twice each week. Each group's academic achievement was measured using the same written tests, homework assignments and laboratory projects. An alpha level of .05 was used for all statistical tests. The experimental group for this study was the intact group of students enrolling in the section of Experimental Statistics listed as taught via the Internet for the Spring semester 1999. A total of five students were enrolled in this class. Four were located in a computer laboratory on campus and one participated from home. Course Conduct The course was structured with instructor led classroom discussions, directed homework, and hands-on statistical analysis laboratories. The distance and face-to-face studio groups met on campus three Saturdays during the semester to complete the laboratories while the traditional class conducted the laboratories in weekly sessions. Three instructor designed, objective referenced tests were used to measure student learning along with the results of assigned homework and laboratory projects. The instructor maintained a student oriented World Wide Web site (current course site available at http://virtual.clemson.edu/exst/web801) with course administrative data, background information and instructional content. The instructor also posted the presentation graphics used and whiteboard annotations made during each lecture to create an an asynchronously accessible record of the synchronously delivered courseware.
Figure 2. Example Instructional Whiteboard
118
While the hybrid audio-data collaboration followed lectures on computers, a scan converter enabled images from the instructor’s host computer to be televised to the television remote group. As the instructor utilized the whiteboard and application sharing functions of T.120 data collaboration, those images were broadcast to the television audience (see Figure 2). Except for intermittent shots of the instructor not seen by the computer group, duplicate images were transmitted to all groups. To speak to the instructor, television remote students called a toll-free telephone number. Although the television studio group was in the same room with the instructor, these students watched a television monitor to see the instructor’s computer generated lecture materials and demonstrations. A site coordinator at each remote television site was responsible for unlocking room doors and readying the equipment. Likewise, a coordinator was responsible for readying the lab used by the computer group and in setting up the audio equipment. However, individual users were responsible for entering and participating in the data collaboration conference. The instructor served as the site coordinator for the television studio. Demographics The hybrid audio-data collaboration, instructional television and face-to-face studio audience groups were all older and included more part-time students than the face-to-face traditional classroom group. This was expected since these groups met in the evening and were attractive to non-traditional students. The hybrid audio-data collaboration, instructional television and face-to-face studio audience groups also contained more doctoral students than the face-to-face traditional classroom group. The traditional class had more females than the evening classes. This led to the face-to-face aggregate also being predominately female while the distance aggregate was evenly distributed between male and female students. Further examination of data concerning equivalency yielded mixed results. Although none of the differences were statistically significant at the .05 level, the hybrid audiographics group had the lowest scores in past academic performance as measured by grade point average (GPA) and only slightly higher scores than the instructional television group in aptitude for quantitative subjects as measured by score on the quantitative section of the Graduate Record Examination (GRE-Q). The instructional television group had the highest GPA but the second lowest GRE-Q. The aggregated distance education groups scored lower on both GPA and GREQ than the aggregated face-to-face groups. Evaluation Performance In order to test the hypothesis that students taught using hybrid Internet-enabled audiographics would perform as well as students taught using other delivery methods, the course results were tested using analysis of variance. Type of delivery was used as the independent variable and cumulative score for the course was used as the dependent variable. There was no significant difference among any of the groups in final course grades (p=0.7778). After further analysis of the data using the analysis of variance procedure, the performance of the two distance delivered groups, hybrid audiographics and instructional television, were shown to not be significantly different (p=0.3215). Therefore, hybrid audiographics appears to be an effective alternative delivery method to the current standard of instructional television. Additionally, since the statistical analysis of variance showed no significant difference between any of the groups, the learning achievement of graduate students taught via hybrid audiographics appears to not be significantly different from the learning achievement of graduate students taught face-to-face in a studio (F2F-S) and those taught face-to-face in a traditional classroom (F2F-T). The final course grades for the distance and face-to-face groups were essentially the same at 87.53 and 88.33 respectively. This was borne out by the analysis of variance (p=0.7786). This further suggests that hybrid audiographics is an effective alternative to face-to-face delivery of instruction. Satisfaction While positive student reactions to a distance education course may not guarantee that learning has taken place, negative reactions are likely to undermine support for the program and negatively affect learning (Biner 1993). Biner (1993) recommends that evaluations of student learning and achievement occur only after an assessment of
119
satisfaction of the distance education program has allowed for those components causing dissatisfaction to be corrected. General course satisfaction, measured by Biner's Teleconference Evaluation Questionnaire (TEQ) items dealing with overall course satisfaction, comparison of the course to other courses, and workload, indicated that students in each group were pleased with the course. While differences between the computer group and other groups for measures of general course satisfaction were not statistically significant, hybrid audio-data collaboration students rated their course higher than other groups in comparisons to conventional courses, and reported the second highest score for the question of overall satisfaction. Hybrid audio-data collaboration students also rated the workload required in the course as lighter than students in other groups. However, the level of satisfaction by students taught via hybrid audio-data collaboration exceeded the satisfaction level of students taught via instructional television for the technology characteristics of the course. Screen picture quality, quality of audio from the instructor and adequacy of screen size appear to have contributed most heavily to these differences. Several factors likely contributed to the lower satisfaction scores for screen size and quality given by instructional television students. Computer monitors and televisions have different aspect ratios (Silverthorne, 1996). This fact was recognized prior to the beginning of the study and steps were taken to ensure that pictures originally fitted within a computer screen were not cut off on the television. Problems with resolution were more difficult to satisfactorily address. Standard NTSC (National Television System Committee) television signal is roughly equivalent to a 640 by 480 pixel image on a computer screen (Machrone, 1997). Perceived quality is further impacted by the increased flicker of the television image, which is caused by the rendering of every other horizontal line during each pass of the cathode ray in a process called interlacing. Also, television resolution was further degraded by the carrier (South Carolina Educational Television) in order to provide more channel capacity on the satellite. Comparatively, hybrid audio-data collaboration participants, including the host computer, operated with a much higher resolution of 800 by 600 pixels, noninterlaced.
Summary Distance learning has the potential to enhance individual competency and graduate education by delivering learning where needed and when needed. However, this requires the distance learning systems to be highly accessible. Since the technology selected for delivery in large part determines accessibility and costs, great care must go into selecting the most accessible, least expensive method that will meet learning objectives. There is a great challenge to provide increased distance learning access for organizations and learners that do not have high bandwidth on-ramps to the Information Superhighway. Hybrid audio-data collaboration is a viable alternative to instructional television for reaching far down the digital dirt roads to dramatically increase access to learners while maintaining educational effectiveness and student satisfaction.
References Biner, P. M. (1993). The development of an instrument to measure student attitudes toward televised courses. The American Journal of Distance Education, 7 (1), 62-73. Bradshaw, D. H. & Desser, K. (1990). Audiographics Distance Learning: A Resource Handbook, San Francisco, CA: Far West Laboratory for Educational, Research and Development. Christensen, H. D. & Cowley-Durst, B. (1998). Thoughts on Distance Learning, An Interview with Alan Chute. Performance Improvement, November/December. Clinton, W. J. (1998). Remarks by the President and the Vice President at Electronic Commerce Event, November 30, 1998, Washington, DC: Office of the Press Secretary, http://www.pub.whitehouse.gov/. Clinton, W. J. (1999). Memorandum for the Heads of Executive Departments and Agencies, Enhancing Learning and Education Through Technology, January 30, 1999, Washington, DC: Office of the Press Secretary,
120
http://www.pub.whitehouse.gov/. The International Multimedia Teleconferencing Consortium, Inc (1997). T.120 OVERVIEW, http://www.imtc.org/t120.htm. Freeman, M. (1999). Effectiveness and Costs of Hybrid Audiographics and Instructional Television for Delivering College Graduate Distance Education, Unpublished Doctoral Dissertation, Clemson SC: Clemson University. Freeman, M., Wisher, R., Curnow, C. & Morris, K. (1999). Down the Digital Dirt Roads: Increasing Access to Distance Learning with Hybrid Audiographics. Proceedings of the 1999 Interservice/Industry Training Simulations and Education Conference, Arlington, VA: National Training Systems Association, 52-61. Gooley, A. (nd). Queensland Open Learning Network. Review of Higher Education Financing and Policy, South Brisbane: Australian Department of Education, Training and Youth Affairs, http://www.deetya.gov.au/highered/hereview/submissions/submissions/q/qldopenlearn.htm Kerka, S. (1996). Distance Learning, the Internet, and the World Wide Web. ERIC Digest, ED395214 96, Columbus, Ohio: ERIC Clearinghouse on Adult, Career, and Vocational Education. Kidwell, R. J. (1998). Integrating Training Through Hybrid Distance Learning Environments. Proceedings of the 1998 Interservice/Industry Training, Simulation and Education Conference, Arlington, VA: National Training Systems Association, 102-113. Machrone, B. (1997). Convergence is still a source of confusion. PC Week, 14 (7), 63. McGreal, R. (1994). Report Of The Subcommittee Of The Advisory Board Of The New Brusnwick Distance Education Network On Choosing An Audiographic Teleconferencing System. The Online Chronicle of Distance Education and Communication, North Miami Beach: Nova Southeastern University, http://www.fcae.nova.edu/disted/fall94/article.html#summary National Institute of Standards and Technology (1998). ATP FOCUSED PROGRAM: Adaptive Learning Systems, http://www.atp.nist.gov/atp/focus/als.htm National Center for Education Statistics (1998). Issue Brief, Distance Education in Higher Education Institutions: Incidence, Audiences, and Plans to Expand, Washington, D.C.: U.S. Department of Education. Parrott, S. (1995). Future Learning: Distance Education in Community Colleges. ERIC Digest, ED385311, Los Angeles, CA.: ERIC Clearinghouse for Community Colleges. Russell, T. L. (1997). Technology Wars: Winners & Losers. Educom Review, 32 (2), March/April. Russell, T. L. (1998). The “No Significant Difference” Phenomenon as Reported in 248 Research Reports, Summaries, and Papers, 4th Edition, Raleigh, N.C.: North Carolina State University. Sherry, L. (1996). Issues in Distance Learning. International Journal of Educational Telecommunications, 1 (4), 337-365. Sauve’, L. (1993). What’s behind the development of a course on the concept of distance education? In D. Keegan (Ed.) Theoretical principles of distance education, NY: Routledge, 102-104. Silverthorne, S. (1996). TV’s fuzzy web sight. PC Week, 13 (23), A1-A2. Steiner, V. (1995). What is Distance Education? Distance Learning Resources Network, http://www.wested.org/tie/dlrn/distance.html. Wisher, R. A., Priest, A. & Glover, E. (1997). Audio Teletraining for Unit Clerks: A Cost Effectiveness Analysis, Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
121
